Predicting study reproducibility with a metric of information compression - a pre-registered test in biomedical experiments

One of the fundamental goals of metascience is to identify and correct factors that hamper the ability to successfully replicate published findings. Using the framework of Information Compression Theory, we pre-registered a set of predictions about how the amount of information yielded by a study, $I$, and the descriptive complexity of the replication protocol, $D$, should correlate with replication success, and we tested these predictions on results of the Brazilian Reproducibility Initiative, a multicenter replication of experiments from Brazilian biomedical science using three common methods. To measure the complexity of protocols, we developed a multi-agent framework powered by large language models (LLMs) that translated each protocol text into a knowledge graph and calculated the graph's minimum description length. We found that $I$, $D$ and their ratio $K=I/D$, a principled measure of information compression, were significantly associated with replication success in the direction predicted only when method type was not controlled for. These results offer a partial support for our hypothesis that a study's expected reproducibility is modulated by the strength of the study's empirical signal and the complexity of its methods.