TS-Zarr: a cloud-optimized Zarr layout for viewing and analyzing biomedical time series

Background: Biomedical time series are recorded at high sampling rates for hours to days across tens to hundreds of channels. Each recording occupies gigabytes to terabytes of storage. FAIR data sharing requirements often place recordings in cloud storage. Visual analysis requires downsampling windows wider than a few seconds to the pixel width of the screen. Visualization tools therefore either read a local copy of the recording or query a dedicated resampling server in front of the data. Findings: We present TS-Zarr, a Zarr v3 layout for time series data. It stores the samples of each channel alongside a pyramid of summary statistics at progressively coarser levels. Zarr on cloud storage offers low-latency, high-throughput parallel reads. The layout lets a reader fetch only the channels and windows it needs. These targeted reads scale better than the same reads from HDF5 and EDF. Precomputed levels trade additional storage and error-bound signal transformation for faster and more scalable visual analysis that moves from the whole recording to narrow windows. As users grow and windows widen, precomputed reads from cloud storage scale beyond the point at which a resampling server fails. Conclusions: Precomputed levels let researchers visualize a recording without downloading a local copy or querying a dedicated resampling server. The time to read a view remains bounded as the window under view widens. Zarr on cloud storage also provides efficient, highly parallelizable, targeted reads of raw sample data for analysis.

Authors

Institutions

Publication Details

Journal
bioRxiv (Cold Spring Harbor Laboratory)
Published
2026-11-18
DOI
https://doi.org/10.64898/2026.11.16.767094
Primary Topic
Scientific Computing and Data Management
Type
preprint

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

TS-Zarr: a cloud-optimized Zarr layout for viewing and analyzing biomedical time series

Wolfgang Ganglberger, Sumita Garai, Michael Brandon Westover, Van Q. Truong et al.
1 links
bioRxiv (Cold Spring Harbor Laboratory)
Scientific Computing and Data Management
preprint

TS-Zarr: a cloud-optimized Zarr layout for viewing and analyzing biomedical time series

Wolfgang Ganglberger, Sumita Garai, Michael Brandon Westover, Van Q. Truong, Joost Wagenaar, Tucker Paron, Rohan Shah, Darrell De Freitas, Nishant Sinha
preprint en

Abstract

Background: Biomedical time series are recorded at high sampling rates for hours to days across tens to hundreds of channels. Each recording occupies gigabytes to terabytes of storage. FAIR data sharing requirements often place recordings in cloud storage. Visual analysis requires downsampling windows wider than a few seconds to the pixel width of the screen. Visualization tools therefore either read a local copy of the recording or query a dedicated resampling server in front of the data. Findings: We present TS-Zarr, a Zarr v3 layout for time series data. It stores the samples of each channel alongside a pyramid of summary statistics at progressively coarser levels. Zarr on cloud storage offers low-latency, high-throughput parallel reads. The layout lets a reader fetch only the channels and windows it needs. These targeted reads scale better than the same reads from HDF5 and EDF. Precomputed levels trade additional storage and error-bound signal transformation for faster and more scalable visual analysis that moves from the whole recording to narrow windows. As users grow and windows widen, precomputed reads from cloud storage scale beyond the point at which a resampling server fails. Conclusions: Precomputed levels let researchers visualize a recording without downloading a local copy or querying a dedicated resampling server. The time to read a view remains bounded as the window under view widens. Zarr on cloud storage also provides efficient, highly parallelizable, targeted reads of raw sample data for analysis.

bioRxiv (Cold Spring Harbor Laboratory)
Harvard University (US), University of Pennsylvania (US), Stanford University (US)
U.S. Department of Defense, National Institutes of Health, Advanced Research Projects Agency, National Institute of Neurological Disorders and Stroke
Scientific Computing and Data Management
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon