# Things you can do dumping your Invenio database into a flat file

- Institution: CERN
- Department: Invenio User Group Workshops
- Speaker: Ferran Jorba
- Year: 2017
- Language: English
- Source: [CERN](https://videos.cern.ch/record/3019282)
- Licence: [CERN](http://copyright.web.cern.ch)
- Who can watch: Teens profiles and older (sources give no ratings; placed by Invisible Network from the source's own words)

Invenio database design and interfaces are optimized for fast end user search and retrieval. As administrators, we can add indexes at will and use them via web or API. However, many maintenance tasks are not well covered with those indexes.

For most of those cases, reading the records sequentialy is the optimal solution. However, if the database is large enough, reading them via Invenio API may take hours, while the system slows down and it may become unresponsive.

In this presentation I'll show a small Python tool that uses Invenio API and a SQLite database as cache to keep an up to date flat file with your bibliographic records.

We'll see how whith this flat file it is much faster and easier to do tasks like generate specialised statistics, quality control, automatic record enrichment or cleaning, or even creating exotic indexes or counters..

0:00:00 Slide 1

0:00:25 Slide 2

0:01:24 Slide 3

0:02:20 Slide 4

0:02:36 Slide 5

0:03:35 Slide 6

0:04:07 Slide 7

0:04:45 Slide 8

0:05:45 Slide 9

0:06:37 Slide 10

## Lectures

1. [Things you can do dumping your Invenio database into a flat file](https://invisiblenetwork.ai/learn/cern/3019282/20b42baf3dfb), 12 min (CERN's player)