A Bayesian Framework with Dirichlet Priors and Spatial Smoothing for Protein Rotamer Prediction

Accurate prediction of protein sidechain conformations is a fundamental challenge in structural biology, with diverse applications ranging from protein structure determination to computational drug design. The performance of backbone-dependent rotamer libraries is often limited by discrete binning artifacts and difficulties handling sparse conformational regions. In this work, we present a Bayesian framework for rotamer prediction that addresses these limitations through Dirichlet priors and spatial smoothing. Our approach models rotamer probabilities as continuous functions of backbone dihedral angles, using circular Gaussian convolution, to make the most of statistical strength from neighboring conformations while respecting the periodic nature of angular data. We constructed rotamer libraries through structural clustering of sidechain conformations rather than chi angle binning, ensuring that each rotamer represents a distinct three-dimensional geometry. We evaluated and compared our framework against the state-of-the-art libraries on two independent test sets. Our Dirichlet model achieved chi angle prediction accuracy of 59-60%. Notably, our method produced consistently lower angular errors, an approximate 13% reduction in mean deviation, suggesting that the continuous probability distributions better capture subtle conformational preferences. Further, we explored the incorporation of non-sequential context by including the identity of nearby non-neighboring residues as an example of extensibility of our framework.