Bayesian estimation and regularization techniques in categorical data analysis

Abstract This paper explores Bayesian estimation for categorical data, focusing on simple yet effective models that provide a foundation for applying more advanced methods accurately and reliably in real-world applications. We begin by revisiting Bayesian estimators for the binomial distribution and investigating their properties. Next, we develop hypothesis tests for categorical data (sign test, homogeneity test, symmetry test) based on regularized maximum likelihood estimates of the probabilities. Finally, we formulate regularized versions of common association measures for contingency tables and study the regularized version of mutual information, particular for the situation where the regularized version can effectively handle zero counts.