<i>PvaPy</i> streaming framework for real-time data processing

User facility upgrades, new measurement techniques, advances in data analysis algorithms as well as advances in detector capabilities result in an increasing amount of data collected at X-ray beamlines. Some of these data must be analyzed and reconstructed on demand to help execute experiments dynamically and modify them in real time. In turn, this requires a computing framework for real-time processing capable of moving data quickly from the detector to local or remote computing resources, processing data, and returning results to users. In this paper, we discuss the streaming framework built on top of PvaPy, a Python API for the EPICS pvAccess protocol. We describe the framework architecture and capabilities, and discuss scientific use cases and applications that benefit from streaming workflows implemented on top of this framework. We also illustrate the framework's performance in terms of achievable data-processing rates for various detector image sizes.

Authors

Institutions

Publication Details

Journal
Journal of Synchrotron Radiation
Published
2025-04-25
DOI
https://doi.org/10.1107/s1600577525002115
Citations
1
Primary Topic
Scientific Computing and Data Management
Type
article
Field-Weighted Citation Impact
4.17

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

PvaPy streaming framework for real-time data processing

Siniša Veseli, John Hammonds, Steven Henke, Nicholas Schwarz et al.
1 citations
Journal of Synchrotron Radiation
Scientific Computing and Data Management
4.17
article

PvaPy streaming framework for real-time data processing

Siniša Veseli, John Hammonds, Steven Henke, Nicholas Schwarz, Barbara Frosik, Hannah Parraga
article en
1 citations

Abstract

User facility upgrades, new measurement techniques, advances in data analysis algorithms as well as advances in detector capabilities result in an increasing amount of data collected at X-ray beamlines. Some of these data must be analyzed and reconstructed on demand to help execute experiments dynamically and modify them in real time. In turn, this requires a computing framework for real-time processing capable of moving data quickly from the detector to local or remote computing resources, processing data, and returning results to users. In this paper, we discuss the streaming framework built on top of PvaPy, a Python API for the EPICS pvAccess protocol. We describe the framework architecture and capabilities, and discuss scientific use cases and applications that benefit from streaming workflows implemented on top of this framework. We also illustrate the framework's performance in terms of achievable data-processing rates for various detector image sizes.

Journal of Synchrotron RadiationVol. 32(3)
Argonne National Laboratory (US)
Office of Science
Industry, innovation and infrastructure
Openalex Percentile: Top 7%
Scientific Computing and Data Management
4.17
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.