Neighboring and polarization effects on line shape of the modulation transfer spectroscopy in lower ground hyperfine state of Rb atoms

We have investigated the detailed line shapes of modulation transfer spectroscopy (MTS) for the [Formula: see text], and 3 transitions of ⁸⁵Rb due to neighboring hyperfine transitions. This analysis has been conducted under four different polarization configurations: linear parallel, linear perpendicular, circular parallel, and circular perpendicular configurations. To the best of our knowledge the effects of neighboring transitions have been applied to MTS for the first time. The dependencies in the asymmetric MTS spectra resulting from the neighboring effects on the [Formula: see text] transition of ⁸⁵Rb are accounted for by artificially adjusting the hyperfine energy separations in the excited state. We also investigate the detailed MTS line shapes for the [Formula: see text] transitions of ⁸⁷Rb, considering circular polarization configurations. The significant strength of the MTS spectra for the crossover lines involving the [Formula: see text] and 1 states of ⁸⁷Rb in the [Formula: see text] polarization configuration can be explained through a MTS signal mediated by spontaneous emission. Theoretical line shapes of these transitions, calculated using the density matrix equations, show good agreement with the observed spectra.