Toward Robust Three-State Diabatization: A Systematic Benchmark of O(3) Transformation Parameterizations

While direct adiabatic-to-diabatic (ATD) transformations for two-state systems have been extensively studied, their extension to three-state systems remains less explored. Building upon the framework of the valence bond wave function-based automatic diabatization (VBADA) approach, this study investigates the application of various O(3) group unitary transformations for the construction of three-state diabatic representations. These include Householder and composite Givens transformations, Euler angles, Tait–Bryan angles, quaternions, and the Rodrigues’ rotation formula. Using the dissociation of lithium hydride and the hydrogen abstraction reaction H + HCl → Cl + H2 as test cases, we evaluate the performance of these different unitary matrices in the diabatization process. Our results reveal a clear performance hierarchy among the O(3) parametrizations. While transformations such as Householder reflections and composite Givens rotations show numerical instability in intermediate geometric regions, and Euler/Tait–Bryan angles are prone to gimbal-lock singularities, both the Rodrigues’ rotation formula and quaternion-based parametrizations consistently generate smooth, stable diabatic states with well-localized chemical character.