Entropy-Regularized Diffusion-Policies in Offline Reinforcement Learning for Antibody Sequence Design

The discovery of therapeutic antibodies is traditionally performed through wet lab screening, which is costly and time-consuming. Generative models offer a data-driven alternative, however such methods become unreliable outside the training distribution. We present Sequential Diffusion + Q-Learning (SeqDiff+QL), which formulates antibody sequence design as a constrained offline Reinforcement Learning (RL) problem, enforcing proximity to the training distribution. SeqDiff+QL employs an entropy-regularized diffusion policy that, through policy improvement, is trained to sequentially generate Complementarity Determining Region (CDR) sequences with higher predicted binding affinity based on a variety of training distributions. Our novel entropy regularization thereby promotes diverse candidate generation, while the integration of biophysical priors through contrastive Variational Autoencoder (VAE) latent representations improves the stability of the generative process. The framework can learn from heterogeneous sequence sources across different training distributions. Using the Absolut! simulator and Rosetta energy function as in silico affinity evaluation oracles, we show that SeqDiff+QL produces candidate sequences with improved oracle-predicted affinity across two distinct antigen targets per oracle. We further document an explicit affinity-diversity tradeoff: Q-guided variants reduce diversity and novelty relative to unguided generators, and our entropy regularizer partially recovers diversity at high affinity.