Estimation of age and metallicity for galaxies based on multi-modal deep learning

Aims. This study is aimed at deriving the age and metallicity of galaxies by proposing a novel multi-modal deep learning framework. This multi-modal framework integrates spectral and photometric data, offering advantages in cases where spectra are incomplete or unavailable. Methods. We propose a multi-modal learning method for estimating the age and metallicity of galaxies (MMLforGalAM). This method uses two modalities: spectra and photometric images as training samples. Its architecture consists of four models: a spectral feature extraction model ( ℳ 1 ), a simulated spectral feature generation model ( ℳ 2 ), an image feature extraction model ( ℳ 3 ), and a multi-modal attention regression model ( ℳ 4 ). Specifically, ℳ 1 extracts spectral features associated with age and metallicity from spectra observed by the Sloan Digital Sky Survey (SDSS). These features are then used as labels to train ℳ 2 , which generates simulated spectral features for photometric images to address the challenge of missing observed spectra for some images. Overall, ℳ 1 and ℳ 2 provide a transformation from photometric to spectral features, with the goal of constructing a spectral representation of data pairs (photometric and spectral features) for multi-modal learning. Once ℳ 2 is trained, MMLforGalAM can then be applied to scenarios with only images, even in the absence of spectra. Then, ℳ 3 processes SDSS photometric images to extract features related to age and metallicity. Finally, ℳ 4 combines the simulated spectral features from ℳ 2 with the extracted image features from ℳ 3 to predict the age and metallicity of galaxies. Results. Trained on 36278 galaxies from SDSS, our model predicts the stellar age and metallicity, with a scatter of 1 σ = 0.1506 dex for age and 1 σ = 0.1402 dex for metallicity. Compared to a single-modal model trained using only images, the multi-modal approach reduces the scatter by 27% for age and 15% for metallicity.