# Partitioned Interleaved Bloom filters using Optane DC Persistent Memory

- Institution: CERN
- Department: other events or meetings
- Speaker: Enrico Seiler
- Year: 2019
- Language: English
- Source: [CERN](https://videos.cern.ch/record/3016118)
- Licence: [CERN](http://copyright.web.cern.ch)
- Who can watch: Teens profiles and older (sources give no ratings; placed by Invisible Network from the source's own words)

The recent improvements of full genome sequencing technologies, commonly subsumed under the term NGS (Next Generation Sequencing), have tremendously increased the sequencing throughput. Within 10 years it rose from 21 billion base pairs collected over months to about 400 billion base pairs per day (current throughput of Illumina's HiSeq 4000). The costs for producing one million base pairs could also be reduced from 140,000 dollars to a few cents. As a result of this dramatic development, the number of new data submissions, generated by various biotechnological protocols (ChIP-Seq, RNA-Seq, etc.), to genomic databases has grown dramatically and is expected to continue to increase faster than the cost and capacity of storage devices will decrease.

The main task in analyzing NGS data is to search sequencing reads or short sequence patterns (i.e. exon/intron boundary read-through patterns) or expression profiles in large collections of sequences (i.e. a database). Searching the entirety of such databases mentioned above is usually only possible by searching the metadata or a set of results initially obtained from the experiment. Searching (approximately) for specific genomic sequence in all the data has not been possible in reasonable computational time.

In this work we describe results of our new data structure, called binning directory that can distribute approximate search queries based on an extension of our recently introduced Interleaved Bloom Filters (IBF) called x-partitioned IBF (x-PIBF). The results presented here make use of Intel's Optane DC Persistent Memory architecture and achieves significant speedups compared to a disk based solution..

0:00:00 Slide 1

0:00:52 Slide 2

0:00:59 Slide 3

0:01:05 Slide 4

0:01:15 Slide 5

0:01:22 Slide 6

0:01:37 Slide 7

0:01:51 Slide 8

0:02:01 Slide 9

0:02:06 Slide 10

0:02:40 Slide 11

0:03:16 Slide 12

0:03:24 Slide 13

0:03:37 Slide 14

0:03:49 Slide 15

0:04:06 Slide 16

0:04:25 Slide 17

0:05:01 Slide 18…

## Lectures

1. [Partitioned Interleaved Bloom filters using Optane DC Persistent Memory](https://invisiblenetwork.ai/learn/cern/3016118/0ddcdb08cec5), 25 min (CERN's player)