Automatic Fish Age Prediction Using Deep Machine Learning: Combining Otolith Image, FT-NIR Spectra and Metadata Features

Fish age is a crucial parameter for effective stock management, influencing the estimation of growth rates, mortality, age at maturity, and population trends. However, traditional methods for determining fish age through otolith analysis are labor-intensive and prone to low repeatability. Recently, Fourier transform near-infrared (FT-NIR) spectroscopy has emerged as a promising tool for more efficient age estimation. In this study, we explore an alternative approach using RGB imagery of whole fish otoliths for rapid and accurate age determination. Specifically, we evaluate the effectiveness of three data modalities-otolith images, FT-NIR spectra, and associated biological and geospatial data-in predicting fish age using deep learning techniques, both independently and in combination. Drawing inspiration from generative AI, which integrates diverse input modalities (e.g., image, text, masks), we propose two distinct methods for merging these data types, beyond simple concatenation of feature embeddings. Our experiments, conducted on otolith data from two commercially significant fish species-walleye pollock and red snapper-reveal that combining all three modalities yields the most accurate age predictions. Additionally, we demonstrate that conditioning the image feature extraction process on both spectral and metadata enhances model performance.