Comfort zones of stars: A limit on orbital tightening via stable mass transfer shapes the properties of binary black hole mergers

Mass transfer in binary systems is the key process in the formation of various classes of objects, including merging binary black holes (BBHs) and neutron stars. The orbital evolution that occurs during mass transfer depends on how much mass is accreted and how much angular momentum is lost – two of the main uncertainties in binary evolution. This poses a challenge for obtaining reliable predictions from binary channels. Here, we demonstrate that despite these unknowns, a fundamental limit exists to how close binary systems can become via stable mass transfer (SMT) that is robust against uncertainties in orbital evolution. Based on detailed evolutionary models of interacting systems with a BH accretor and a massive-star companion, we show that the post-interaction orbit is always wider than ∼10 R ⊙ , even when extreme shrinkage due to L2 outflows is assumed. Systems evolving toward tighter orbits become dynamically unstable and result in stellar mergers. This separation limit has direct implications for the properties of BBH mergers, including long delay times (≳1 Gyr) and an absence of high BH spins from the tidal spin-up of helium stars. At high metallicity, the SMT channel may be severely quenched due to Wolf-Rayet winds. We predict BBH mergers from ∼10 M ⊙ to 90 M ⊙ , with case A mass transfer dominating above 40 M ⊙ . The reason for the separation limit lies in the stellar structure, not in binary physics. If the orbit becomes too narrow during mass transfer, a dynamical instability is triggered by a rapid expansion of the remaining donor envelope due to its near-flat entropy profile. The closest separations can be achieved from core-He burning (∼8−15 R ⊙ ) and Main Sequence donors (∼15−30 R ⊙ ), while Hertzsprung gap donors lead to wider orbits (≳30−50 R ⊙ ) and non-merging BBHs. These outcomes and mass transfer stability are determined by the entropy structures, which are governed by internal composition profiles. Consequently, the formation of BBH mergers and other compact binaries via SMT is a sensitive probe of chemical mixing in stars, and it may help address open questions of stellar astrophysics, such as the blue supergiant problem. Finally, we propose a new simplified treatment of mass transfer stability that more accurately reproduces detailed results and remains flexible under varying assumptions for orbital evolution.