Site Logo
Site Logo
Heliophysics Software Search Interface
HSSI

dbprocessing

0.1.0
Code Docs DOI
Citation

Software

Concise Description
Automated processing controller for heliophysics data
Description
dbprocessing is a Python-based, database-driven process controller which automates the production of derived data products upon the arrival of new input data. Although originally written for Heliophysics data, it is intended to be flexible enough to manage most forms of digital time-series data. Documentation is available at https://spacepy.github.io/dbprocessing/.
Authors
Lorna Ellis
Ken Fairchild
Reinhard Friedel ( Los Alamos National Laboratory )
Myles Johnson
Brian Larsen ( Los Alamos National Laboratory )
Nathan Masi
Steven K. Morley ( Los Alamos National Laboratory )
Denis Nadeau ( Los Alamos National Laboratory )
Jonathan T. Niehof ( University of New Hampshire )
Evin O'Shea
Anthony Rogers
Elizabeth Vandegriff
Andrew Walker ( Los Alamos National Laboratory )
Meilin Yan
Xiaoguang Yang ( Los Alamos National Laboratory )
Functionality
Data Processing and Analysis Processing Data Visualization 2D Graphics Mission-related Archive Distribution/Access Ingest Inventory Monitoring Orchestration Processing Science Data Processing
Technical Details
Programming Languages Python 2.x Python 3.x
Operating Systems Linux
CPU Architecture CPU Independent
Development Status Inactive
Publication Date Feb 10, 2022
Publisher Zenodo
Data & File Formats
Data Sources Observatory/Mission-specific Other
Input Formats ascii CDF
Output Formats ascii Other
Science Context
Related Instruments PSP Integrated Science Investigation of the Sun, ISOIS, Instrument RBSP ECT RBSP ECT
Related Observatories Parker Solar Probe Van Allen Probes
Related Regions Earth Magnetosphere Earth Inner Magnetosphere Interplanetary Space
Keywords
database data management data processing heliophysics metadata parker solar probe provenance pyhc package python van allen probes
Related Items

Related Publications

https://ui.adsabs.harvard.edu/abs/2020hdee.prop...17N/abstract

Related Software

https://github.com/lanl/dbprocessing

Interoperable Software

https://github.com/spacepy/spacepy
Licensing & Funding
License BSD 3-Clause "New" or "Revised" License
Funders Johns Hopkins University Applied Physics Laboratory National Aeronautics and Space Administration
Funding Awards
dbprocessing: Space Science Data Processing Controller in Python (80NSSC21K0307) - National Aeronautics and Space Administration
Parker Solar Probe Integrated Science Investigation of the Sun (136435) - Johns Hopkins University Applied Physics Laboratory
Van Allen Probes Radiation Belt Storm Probe, Energetic particle, Composition, and Thermal plasma suite (967399) - Johns Hopkins University Applied Physics Laboratory
Version Information
Current Version 0.1.0
Release Date Feb 10, 2022
Version DOI https://doi.org/10.5281/zenodo.6047134
Release Notes
First public packaged release of dbprocessing. Adds Python 3 support; initial PostgreSQL support; support for processes that take no input, with related DBRunner enhancements; a new linkUningested script that finds files matching a product format but absent from the database and symlinks them into the incoming directory for ingestion; and new printProcessQueue options (--count, --exist, --quiet, --product, --sort). Fixes ProcessQueue.buildChildren on older databases lacking the yesterday/tomorrow columns, corrects runMe's handling of the output interface version so that quality versions increment appropriately, and restricts input selection to files recorded in the database as existing on disk. SQLAlchemy and dateutil are required.
Back to Software Catalog