When X-ray Features Fail to Identify Intrinsic Emitters: Label Noise and Luminosity Overlap in Machine Learning Classification of AGN and Star-forming Galaxies

Abstract Previously, we found that adding an X-ray flux feature to a Random Forest classifier of active galactic nuclei (AGN) and star-forming galaxies (SFGs) coincided with a decrease in classification accuracy from 97.51% to 89.26%, a counterintuitive result given the prevailing theory that X-rays are a reliable AGN diagnostic. We investigate this discrepancy through both astrophysical and machine learning lenses. On the astrophysical side, we show that the X-ray luminosities of AGN and SFGs in the sample substantially overlap across the canonical 1042 erg/s threshold, reflecting the moderate luminosity characteristic of an optically-selected sample and the contribution of high-mass X-ray binaries (HMXBs) to SFG emission. On the machine learning side, we argue that BPT-derived training labels carry instance-dependent label noise concentrated near the BPT demarcation line, where X-ray data would be most useful as a discriminator. We show that objects misclassified in 5-fold cross-validation cluster near the Kewley demarcation curve, with a median distance of 0.123 dex versus 0.743 dex for correctly classified objects (Kolmogorov-Smirnov D = 0.745, p = 1.67 × 10−9). A controlled cross-validation decomposition shows that the previously reported decrease reflects predominantly sample selection: on a fixed sample, adding the X-ray feature leaves class-resolved metrics unchanged, and its permutation importance is consistent with zero across resampled splits. The classifier therefore makes negligible use of a physically informative feature, consistent with training labels coupled to the optical features. We conclude that future classifiers operating on optically-selected samples should employ independent label sources or noise-robust training methods.