Combining TIFF, HDF5, and Zarr into a Single Image File Format

“Situation: There are 3 competing standards” the last card of a popular XKCD cartoon (#927) might read if applied to recent microscopy bioimaging formats. TIFF, HDF5, and Zarr have all been used to store images as part of popular standards and formats (OME-TIFF, BDV-HDF5, OME-Zarr). The cartoon humorously points out the tendency to create new standards while discounting prior efforts. To combat the proliferation of formats I examine similarities between TIFF, HDF5, and Zarr shard containers. I then exploit them to create a combined data container that is simultaneously a TIFF file, a HDF5 file, and a Zarr version 3 shard without duplicating the image pixel or volumetric voxel data. This combined format is compatible with multiple viewers and image analysis pipelines. Additionally, the techniques involved provide a path to convert between the formats with minimal processing or overhead. In practice, the combined format avoids redundant copies of data while providing great utility to the microscope user. The combined format is a great candidate for a microscope acquisition format as it satisfies both short term needs to view microscope output in traditional viewers while integrating into next generation image analysis pipelines.