# The one-million table partitions challenge in an ATLAS experiment DB application

- Institution: CERN
- Department: IT Lightning Talks (ITLT)
- Speaker: Gancho Dimitrov
- Year: 2019
- Language: English
- Source: [CERN](https://videos.cern.ch/record/3006501)
- Licence: [CERN](http://copyright.web.cern.ch)
- Who can watch: Teens profiles and older (sources give no ratings; placed by Invisible Network from the source's own words)

How can a new database system which manages billions of rows with relatively short lifetime (weeks to months) be designed ? The system should resemble a Whiteboard: - data, grouped by given properties form data collections, are added when requested - data collections are consumed by the requestor - when not needed any more, data collections are removed by a "Whiteboard sponge" process.

I will show you what options I explored and which one is potentially the best in terms of efficiency..

0:00:00 Slide 1

0:00:06 Slide 2

0:00:11 Slide 3

0:00:38 Slide 4

0:01:13 Slide 5

0:01:23 Slide 6

0:02:08 Slide 7

0:02:25 Slide 8

0:02:31 Slide 9

0:02:40 Slide 10

0:03:02 Slide 11

0:03:18 Slide 12

0:03:42 Slide 13

0:04:31 Slide 14

0:05:08 Slide 15

0:05:24 Slide 16

0:06:07 Slide 17

## Lectures

1. [The one-million table partitions challenge in an ATLAS experiment DB application](https://invisiblenetwork.ai/learn/cern/3006501/9407cc17e1a2), 10 min (CERN's player)