Extreme deconvolution reimagined: conditional densities via neural networks and an application in quasar classification

ABSTRACT Density estimation is a fundamental problem that arises in many areas of astronomy, with applications such as selecting quasars via colour distributions and characterizing stellar abundances. Astronomical observations are inevitably noisy, while the density of a noise-free feature is often the desired outcome. The extreme-deconvolution (XD) method can be used to deconvolve the noise and estimate the underlying density distribution by fitting a mixture of Gaussians to data with heteroscedastic Gaussian noise. However, XD does not generalize to cases where some feature dimensions have distributions far away from Gaussian, and no established method exists to overcome this limitation. Requiring negligible noise in these non-Gaussian features, we introduce a possible solution that separates out the non-Gaussian features and models the Gaussian-like dimensions conditioned on the non-Gaussian features using a neural network and Gaussian mixture model. The result is the CondXD algorithm, a generalization of XD that takes in the non-Gaussian features and outputs the deconvolved conditional distribution of the Gaussian-like features on the input features. We apply CondXD to a toy model, and compare it with an existing method that divides the samples into bins of conditioning variables and applies XD separately to each bin. We find that CondXD is more accurate than the classical approach. We further test CondXD on a real-world high-redshift quasar versus contaminant classification problem. It achieves comparable results to the binning method but is roughly 10 times faster. Overall, our method has the potential to significantly improve the deconvolution of non-Gaussian distributions and enable new discoveries in astronomy.