Preference-Based Algorithm Selection for Data Compression

The proliferation of compression algorithms has introduced a significant obstacle: how should users select the optimal algorithm for their specific needs? Traditional benchmarking studies focus primarily on compression ratios while neglecting equally critical objectives such as speed. Furthermore, most studies lack statistical validation of performance differences, and evaluation is often limited to narrow benchmark sets. We address these limitations through a comprehensive multi—objective evaluation of five widely—deployed compression algorithms (gzip, bzip2, lzma, zstandard, brotli) across 391 files (387 real—world files from 6 diverse corpora plus 4 artificial stress—test files excluded from statistical inference) measuring compression ratio and compression speed. Our rigorous statistical analysis employing ANOVA with assumption testing, non—parametric validation, and paired t—tests with Bonferroni correction reveals a striking finding: while compression speed exhibits highly significant differences across algorithms , compression ratio differences are statistically detectable under non—parametric testing but practically negligible due to a very small effect size . Effect size analysis demonstrates that algorithm choice explains only 0.3% of compression ratio variance while explaining 5.0% of speed variance, with file entropy characteristics dominating compression outcomes. Median compression ratios range narrowly from 2.71 to 3.51 across all algorithms, with interquartile ranges of approximately 1.0, indicating consistent compression effectiveness regardless of algorithm choice. Through Pareto frontier analysis, we identify four algorithms (gzip, bzip2, zstd, brotli) as optimal under different speed—ratio preferences. However, the marginal ratio improvement of brotli over gzip (+29%) comes at an 86% speed penalty, illustrating the trade—off between compression ratio and throughput in speed—sensitive real—world applications. This work demonstrates that algorithm selection must account for statistically validated trade—offs between compression ratio and throughput, questioning conventional algorithm selection practices.