About
File formats for generic data structures, such as ROOT, Avro, and Parquet, pose a problem for deserialization: it must be fast, but its code depends on the type of the data structure, not known at compile-time. Just-in-time compilation can satisfy both constraints, but we propose a more portable solution: specialized virtual machines. AwkwardForth is a Forth-driven virtual machine for deserializing data into Awkward Arrays. As a language, it is not intended for humans to write, but it loosens the coupling between Uproot and Awkward Array. AwkwardForth programs for deserializing record-oriented formats (ROOT and Avro) are about as fast as C++ ROOT and 10‒80× faster than fastavro. Columnar formats (simple TTrees, RNTuple, and Parquet) only require specialization to interpret metadata and are therefore faster with precompiled code..
0:00:00 Slide 1
0:01:12 Slide 2
0:02:24 Slide 3
0:03:36 Slide 4
0:04:47 Slide 5
0:05:59 Slide 6
0:07:11 Slide 7
0:08:22 Slide 8
0:09:34 Slide 9
0:10:46 Slide 10
0:11:58 Slide 11
The talk
Credits and licence
- Institution
- CERN
- Department
- Conferences
- Speaker
- Jim Pivarski
- Year
- 2021
- Language
- English
- Subjects
- Physics, Computer science, Earth and environment
- Source
- CERN
- Licence
- CERN, as CERN states it. CERN's terms
- Attribution
- © CERN. CERN provides its audiovisual media free of charge for educational and informational use, credited to CERN; this lecture is shown in CERN's own player.
- Who can watch
- CERN gives no rating. Invisible Network shows it to Tweens profiles and older, judged from what the source says about it. How courses are placed