hep_tables: Heterogeneous Array Programming for HEP
Description
Array operations are one of the most concise ways of expressing common filtering and simple aggregation operations that are the hallmark of a particle physics analysis: selection, filtering, basic vector operations, and filling histograms. The High Luminosity run of the Large Hadron Collider (HL-LHC), scheduled to start in 2026, will require physicists to regularly skim datasets that are over a PB in size, and repeatedly run over datasets that are 100's of TB's – too big to fit in memory. Declarative programming techniques are a way of separating the intent of the physicist from the mechanics of finding the data and using distributed computing to process and make histograms. This paper describes a library that implements a declarative distributed framework based on array programming. This prototype library provides a framework for different sub-systems to cooperate in producing plots via plug-in's. This prototype has a ServiceX data-delivery sub-system and an awkward array sub-system cooperating to generate requested data or plots. The ServiceX system runs against ATLAS xAOD data and flat ROOT TTree's and awkward on the columnar data produced by ServiceX. * This work was supported by the National Science Foundation under Cooperative Agreement OAC-1836650.
Availability note (English)
Available from https://www.epj-conferences.org/articles/epjconf/pdf/2021/05/epjconf_chep2021_03061.pdf; https://doaj.org/article/67214cf956394f7c8350ebc13f6a1608Additional details
Identifiers
Publishing Information
- Journal Title
- EPJ. Web of Conferences
- Journal Volume
- 251
- Journal Page Range
- vp.
- ISSN
- 2100-014X
Conference
- Title
- 25. International Conference on Computing in High Energy and Nuclear Physics
- Acronym
- CHEP 2021
- Dates
- 17-21 May 2021
- Place
- Geneva (Switzerland)
INIS
- Country of Publication
- France
- Country of Input or Organization
- France
- INIS RN
- 53090263
- Subject category
- S46: INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND TECHNOLOGY; S97: MATHEMATICAL METHODS AND COMPUTING;
- Resource subtype / Literary indicator
- Conference
- Descriptors DEI
- AGGLOMERATION; CERN LHC; FILTERS; LUMINOSITY; MECHANICS; PROGRAMMING
- Descriptors DEC
- ACCELERATORS; CYCLIC ACCELERATORS; OPTICAL PROPERTIES; PHYSICAL PROPERTIES; STORAGE RINGS; SYNCHROTRONS