Published 2019
| Version v1
Journal article
Columnar data processing for HEP analysis
- 1. Princeton University (United States)
- 2. National Institute of Technology, Silchar (India)
Description
In the last stages of data analysis, physicists are often forced to choose between simplicity and execution speed. In High Energy Physics (HEP), high-level languages like Python are known for ease of use but also very slow execution. However, Python is used in speed-critical data analysis in other fields of science and industry. In those fields, most operations are performed on Numpy arrays in an array programming style; this style can be adopted for HEP by introducing variable-sized, nested data structures. We describe how array programming may be extended for HEP use-cases and an implementation known as awkward-array. We also present integration with ROOT, Apache Arrow, and Parquet, as well as preliminary performance results.
Availability note (English)
Available from https://www.epj-conferences.org/articles/epjconf/pdf/2019/19/epjconf_chep2018_06026.pdf; https://doaj.org/article/f9f0e3c141c04becb657778b3f1ffb04Additional details
Identifiers
Publishing Information
- Journal Title
- EPJ. Web of Conferences
- Journal Volume
- 214
- Journal Page Range
- vp.
- ISSN
- 2100-014X
Conference
- Title
- 23. International Conference on Computing in High Energy and Nuclear Physics
- Acronym
- CHEP 2018
- Dates
- 9-13 Jul 2018
- Place
- Sofia (Bulgaria)
INIS
- Country of Publication
- France
- Country of Input or Organization
- France
- INIS RN
- 53095595
- Subject category
- S97: MATHEMATICAL METHODS AND COMPUTING;
- Resource subtype / Literary indicator
- Conference
- Descriptors DEI
- DATA ANALYSIS; HIGH ENERGY PHYSICS; PERFORMANCE; PYTHON
- Descriptors DEC
- DATA PROCESSING; PHYSICS; PROCESSING; PROGRAMMING LANGUAGES