Efficient storage of sparse image data in a human-readable format

Abstract The modern image formats used in recent position sensitive X-ray detectors are not suited to handling sparse image data, with only a few photons per frame, and then often must store vast amounts of data (~ terabyte), even if the meaningful information is only a few MB. Additionally, many formats are binary and lack human-readability. Here we introduce a straightforward, human-readable format for collecting and storing sparse image data called “sitf” (sparse image text format). This format compresses data without losing information while improving human readability and contributing to more efficient use of storage. The source code developed in Python to handle sitf is available at https://github.com/t-manjo/sitf_code. For one recent case, continuous collection of data at 100 Hz, the sitf data volume was only ~36 MB / day, a compression rate is 4 × 10 −6 compared to the image format, while improving human readability.