DES to HSC: Detecting low-surface-brightness galaxies in the Abell 194 cluster using transfer learning

Context. Low-surface-brightness galaxies (LSBGs) are important for understanding galaxy evolution and cosmological models. Nevertheless, the physical properties of these objects remain unknown, as even the detection of LSBGs can be challenging. Upcoming large-scale surveys are expected to uncover a large number of LSBGs, which will require accurate automated or machine learningbased methods for their detection. Aims. We study the scope of transfer learning for the identification of LSBGs. We used transformer models trained on Dark Energy Survey (DES) data to identify LSBGs from dedicated Hyper Suprime-Cam (HSC) observations of the Abell 194 cluster, which are two magnitudes deeper than DES. A new sample of LSBGs and ultra-diffuse galaxies (UDGs) around Abell 194 was compiled, and their properties were investigated. Methods. We used eight models, divided into two categories: LSBG Detection Transformer (LSBG DETR) and LSBG Vision Transformer (LSBG ViT). The data from DES and HSC were standardised based on the pixel-level surface brightness. We used an ensemble of four LSBG DETR models and another ensemble of four LSBG ViT models to detect LSBGs. This was followed by a singlecomponent Sérsic model fit and a final visual inspection to filter out potential false positives and improve sample purity. Results. We present a sample of 171 LSBGs in the Abell 194 cluster using HSC data, including 87 new discoveries. Of these, 159 were identified using transformer models, and 12 additional LSBGs were found through visual inspection. The transformer model achieves a true positive rate of 93% in HSC data without any fine-tuning. Among the LSBGs, 28 were classified as UDGs. The number of UDGs and the radial UDG number density suggests a linear relationship between UDG numbers and cluster mass on a log scale. The UDGs share similar Sérsic parameters with dwarf galaxies and occupy the extended end of the R eff − M g plane, suggesting they might be an extended sub-population of dwarf galaxies. We also found that LSBGs and UDGs near the cluster centre are brighter and redder than those in outer regions. Conclusions. We have demonstrated that transformer models trained on shallower surveys can be successfully applied to deeper surveys with appropriate data normalisation. This approach allows us to use existing data and apply the knowledge to upcoming and ongoing surveys, such as the Rubin Observatory Legacy Survey of Space and Time (LSST) and Euclid.