Reference normalizations of Effort-To-Compress across length, alphabet size, and composition

Comparing finite symbolic complexity requires an explicit reference. Effort-To-Compress (ETC) counts the recursive pair substitutions needed to reduce a symbolic sequence to a constant. Its conventional normalization with L − 1 leaves systematic dependence on sequence length L and alphabet size k, confounding comparisons across datasets. We derive certified lower bounds for ETC and show that, for L ≥ k ≥ 2, the ceiling L − 1 is attainable if and only if L ≤ k2 + k + 1. For fixed k ≥ 2, the global ensemble mean and maximum ETC grow as Θ k (L/ log k L) as L → ∞. These support two normalized ETC coordinates. Global-ensemble normalized ETC (nETC) maps a certified lower bound to zero and the uniform ensemble mean at the same L and k to one. Composition-conditioned normalized ETC (cETC) uses corresponding references restricted to the observed symbol composition. In Potts chains, nETC substantially reduced dependence on length and alphabet size. cETC further reduced differences associated with unequal symbol frequencies, while retaining sensitivity to temporal dependence. In human chromosome 22, the exploratory predictor nETC − cETC outperformed LZ76 in distinguishing CpG islands and long interspersed nuclear elements from genomic background with AUROC of 0.947 and 0.848. Matched references therefore enable interpretable comparisons of symbolic complexity across sequence lengths, alphabet sizes, and compositions.

Authors

Institutions

Publication Details

Journal
bioRxiv (Cold Spring Harbor Laboratory)
Published
2026-10-28
DOI
https://doi.org/10.64898/2026.10.22.761192
Primary Topic
Algorithms and Data Compression
Type
preprint

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Reference normalizations of Effort-To-Compress across length, alphabet size, and composition

Zirui Huang, Jy-yong Sohn, Hyunwoo Jang
bioRxiv (Cold Spring Harbor Laboratory)
Algorithms and Data Compression
preprint

Reference normalizations of Effort-To-Compress across length, alphabet size, and composition

Zirui Huang, Jy-yong Sohn, Hyunwoo Jang
preprint en

Abstract

Comparing finite symbolic complexity requires an explicit reference. Effort-To-Compress (ETC) counts the recursive pair substitutions needed to reduce a symbolic sequence to a constant. Its conventional normalization with L − 1 leaves systematic dependence on sequence length L and alphabet size k, confounding comparisons across datasets. We derive certified lower bounds for ETC and show that, for L ≥ k ≥ 2, the ceiling L − 1 is attainable if and only if L ≤ k2 + k + 1. For fixed k ≥ 2, the global ensemble mean and maximum ETC grow as Θ k (L/ log k L) as L → ∞. These support two normalized ETC coordinates. Global-ensemble normalized ETC (nETC) maps a certified lower bound to zero and the uniform ensemble mean at the same L and k to one. Composition-conditioned normalized ETC (cETC) uses corresponding references restricted to the observed symbol composition. In Potts chains, nETC substantially reduced dependence on length and alphabet size. cETC further reduced differences associated with unequal symbol frequencies, while retaining sensitivity to temporal dependence. In human chromosome 22, the exploratory predictor nETC − cETC outperformed LZ76 in distinguishing CpG islands and long interspersed nuclear elements from genomic background with AUROC of 0.947 and 0.848. Matched references therefore enable interpretable comparisons of symbolic complexity across sequence lengths, alphabet sizes, and compositions.

bioRxiv (Cold Spring Harbor Laboratory)
Yonsei University (KR), University of Michigan (US)
National Institutes of Health, Medical School, University of Michigan, National Institute of General Medical Sciences
Algorithms and Data Compression
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon