Lossless compression of simulated radio interferometric visibilities

Context. Processing radio interferometric data often requires storing forward-predicted model data. In direction-dependent calibration, these data may have a volume an order of magnitude larger than the original data. Existing lossy compression techniques work well for observed, noisy data, but cause issues in calibration when applied to forward-predicted model data. Aims. To reduce the volume of forward-predicted model data, we present a lossless compression method called Simulated Signal Compression (Sisco) for noiseless data that integrates seamlessly with existing workflows. We show that Sisco can be combined with baseline-dependent averaging for further size reduction. Methods. Sisco decomposes complex floating-point visibility values and uses polynomial extrapolation in time and frequency to predict values, groups bytes for efficient encoding, and compresses residuals using the <ns0:sc>DEFLATE</ns0:sc> algorithm. We evaluated Sisco on diverse LOFAR, MeerKAT, and MWA datasets with various extrapolation functions. Implemented as an open-source Casacore storage manager, it can directly be used by any observatory that makes use of this format. Results. We find that a combination of linear and quadratic prediction yields optimal compression, reducing noiseless forward-predicted model data to 24% of its original volume on average. Compression varies by dataset, ranging from 13% for smooth data to 38% for less predictable data. For pure noise data, compression achieves just a size of 84% due to the unpredictability of such data. With the current implementation, the achieved compression throughput is with 534 MB/s mostly dominated by I/O on our testing platform, but occupies the processor during compression or decompression. Finally, we discuss the extension to a lossy algorithm.