A Novel Framework for Automated Fish Behavior Monitoring Using Acoustic Images and Signals

This study introduces a novel framework for recognizing fish foraging behavior, utilizing a multimodal Variational Autoencoder (VAE) that concurrently processes acoustic images and signals to enhance recognition accuracy. The framework employs a multi-task learning architecture that integrates reconstruction, classification, and contrastive learning objectives, thereby improving the model's generalization capabilities and the quality of its learned representations. An online learning mechanism is incorporated, allowing the model to continuously update with new data streams through an experience replay buffer. Experimental evaluations conducted on the MRS-FFIA dataset reveal that the proposed method significantly surpasses unimodal approaches, achieving a 12% increase in accuracy. This framework presents an efficient acoustic monitoring solution for automated behavior tracking in aquaculture systems.