The journey towards a specialized text embedding model for accelerator physics

We present PhysBERT and AccPhysBERT, specialized sentence-embedding models trained on 1.2 million arXiv physics papers and fine-tuned for accelerator physics, respectively. Evaluation across retrieval, clustering, and similarity tasks shows gains of up to 12\% over general-purpose models for physics corpora and 18\% for accelerator-specific tasks. Applications include semantic reviewer–paper matching, Retrieval-Augmented Generation for control-room logbooks, and rapid sub-domain adaptation. We analyze key design choices—data curation, masking objectives, and contrastive fine-tuning—and outline strategies for continual adaptation, providing a blueprint for domain-specific embeddings in the physical sciences.