TS-Zarr: a cloud-optimized Zarr layout for viewing and analyzing biomedical time series
Background: Biomedical time series are recorded at high sampling rates for hours to days across tens to hundreds of channels. Each recording occupies gigabytes to terabytes of storage. FAIR data sharing requirements often place recordings in cloud storage. Visual analysis requires downsampling windows wider than a few seconds to the pixel width of the screen. Visualization tools therefore either read a local copy of the recording or query a dedicated resampling server in front of the data. Findings: We present TS-Zarr, a Zarr v3 layout for time series data. It stores the samples of each channel alongside a pyramid of summary statistics at progressively coarser levels. Zarr on cloud storage offers low-latency, high-throughput parallel reads. The layout lets a reader fetch only the channels and windows it needs. These targeted reads scale better than the same reads from HDF5 and EDF. Precomputed levels trade additional storage and error-bound signal transformation for faster and more scalable visual analysis that moves from the whole recording to narrow windows. As users grow and windows widen, precomputed reads from cloud storage scale beyond the point at which a resampling server fails. Conclusions: Precomputed levels let researchers visualize a recording without downloading a local copy or querying a dedicated resampling server. The time to read a view remains bounded as the window under view widens. Zarr on cloud storage also provides efficient, highly parallelizable, targeted reads of raw sample data for analysis.
Authors
- Wolfgang Ganglberger (ORCID: https://orcid.org/0000-0002-6029-2450)
- Sumita Garai (ORCID: https://orcid.org/0000-0001-6035-4837)
- Michael Brandon Westover (ORCID: https://orcid.org/0000-0003-4803-312X)
- Van Q. Truong (ORCID: https://orcid.org/0000-0002-5485-1818)
- Joost Wagenaar (ORCID: https://orcid.org/0000-0003-0837-7120)
- Tucker Paron
- Rohan Shah (ORCID: https://orcid.org/0009-0003-8056-3443)
- Darrell De Freitas (ORCID: https://orcid.org/0000-0002-3717-3007)
- Nishant Sinha
Institutions
- Harvard University (US)
- University of Pennsylvania (US)
- Stanford University (US)
Publication Details
- Journal
- bioRxiv (Cold Spring Harbor Laboratory)
- Published
- 2026-11-18
- DOI
- https://doi.org/10.64898/2026.11.16.767094
- Primary Topic
- Scientific Computing and Data Management
- Type
- preprint
Funders
- U.S. Department of Defense
- National Institutes of Health
- Advanced Research Projects Agency
- National Institute of Neurological Disorders and Stroke