pyfive: A pure-Python HDF5 reader

pyfive is an open-source and thread-safe pure Python package for reading data stored in HDF5.While it is not a complete implementation of all the specifications and capabilities of HDF5, it includes all the core functionality necessary to read gridded datasets, whether stored contiguously or with chunks (with or without standard compression options).All data access is fully lazy as the data is only read from storage when the numpy data arrays are manipulated.Originally developed some years ago, the package has recently been expanded to support lazy data access, and to add missing features necessary for handling all the HDF5-based environmental data known to the authors.It is now a realistic option for production data access in environmental science and more broadly across other domains.The API is based on that of h5py (https://github.com/h5py/h5py,a Python shimmy over the HDF5 C-library which itself is not thread-safe), with some API extensions to help optimise remote access.With these extensions, coupled with thread safety, many of the limitations precluding the efficient use of HDF5 (and netCDF4) on cloud storage have been removed.

Authors

Institutions

Publication Details

Journal
The Journal of Open Source Software
Published
2026-02-12
DOI
https://doi.org/10.21105/joss.09688
Primary Topic
Computational Physics and Python Applications
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

pyfive: A pure-Python HDF5 reader

Brian B. Maranville, Benjamin J. Hodel, Wout De Nolf, David Hassell et al.
The Journal of Open Source Software
Computational Physics and Python Applications
article

pyfive: A pure-Python HDF5 reader

Brian B. Maranville, Benjamin J. Hodel, Wout De Nolf, David Hassell, Valeriu Predoi, Helmus, Ezequiel Cimadevilla, Kai Mühlbauer
article en

Abstract

pyfive is an open-source and thread-safe pure Python package for reading data stored in HDF5.While it is not a complete implementation of all the specifications and capabilities of HDF5, it includes all the core functionality necessary to read gridded datasets, whether stored contiguously or with chunks (with or without standard compression options).All data access is fully lazy as the data is only read from storage when the numpy data arrays are manipulated.Originally developed some years ago, the package has recently been expanded to support lazy data access, and to add missing features necessary for handling all the HDF5-based environmental data known to the authors.It is now a realistic option for production data access in environmental science and more broadly across other domains.The API is based on that of h5py (https://github.com/h5py/h5py,a Python shimmy over the HDF5 C-library which itself is not thread-safe), with some API extensions to help optimise remote access.With these extensions, coupled with thread safety, many of the limitations precluding the efficient use of HDF5 (and netCDF4) on cloud storage have been removed.

The Journal of Open Source SoftwareVol. 11(118)
Universidad de Cantabria (ES), University of Bonn (DE), European Synchrotron Radiation Facility (FR), National Centre for Atmospheric Science (GB), Astrobiology Center (JP), Film Independent (US), Instituto de Física de Cantabria (ES), NIST Center for Neutron Research (US), University of Reading (GB)
UK Research and Innovation, Met Office
Openalex Percentile: Top 59%
Computational Physics and Python Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

pyfive: A pure-Python HDF5 reader — Brian B. Maranville, Benjamin J. Hodel, et al. · The Journal of Open Source Software (2026) | TGRS Research Map | TGRS