Published 2021 | Version v1
Journal article

AwkwardForth: accelerating Uproot with an internal DSL

  • 1. Princeton University, Princeton, New Jersey (United States)
  • 2. Institute of Engineering and Management, Kolkata, West Bengal (India)

Description

File formats for generic data structures, such as ROOT, Avro, and Parquet, pose a problem for deserialization: it must be fast, but its code depends on the type of the data structure, not known at compile-time. Just-in-time compilation can satisfy both constraints, but we propose a more portable solution: specialized virtual machines. AwkwardForth is a Forth-driven virtual machine for deserializing data into Awkward Arrays. As a language, it is not intended for humans to write, but it loosens the coupling between Uproot and Awkward Array. AwkwardForth programs for deserializing record-oriented formats (ROOT and Avro) are about as fast as C++ ROOT and 10–80× faster than fastavro. Columnar formats (simple TTrees, RNTuple, and Parquet) only require specialization to interpret metadata and are therefore faster with precompiled code.

Availability note (English)

Available from https://www.epj-conferences.org/articles/epjconf/pdf/2021/05/epjconf_chep2021_03002.pdf; https://doaj.org/article/f6557b42090f449084eae9cef42177af

Additional details

Publishing Information

Journal Title
EPJ. Web of Conferences
Journal Volume
251
Journal Page Range
vp.
ISSN
2100-014X

Conference

Title
25. International Conference on Computing in High Energy and Nuclear Physics
Acronym
CHEP 2021
Dates
17-21 May 2021
Place
Geneva (Switzerland)

INIS

Country of Publication
France
Country of Input or Organization
France
INIS RN
53090138
Subject category
S97: MATHEMATICAL METHODS AND COMPUTING;
Resource subtype / Literary indicator
Conference
Descriptors DEI
COMPUTERIZED SIMULATION; DATA ANALYSIS; PROGRAMMING LANGUAGES
Descriptors DEC
DATA PROCESSING; PROCESSING; SIMULATION