AquaGPT: A Multimodal Transformer Framework with Expert Knowledge Embedding for Intelligent Feeding Management in Aquaponics

Aquaponics integrates aquaculture with hydroponic crop cultivation, which offers a sustainable approach to food production. However, feeding management remains a critical bottleneck because it directly affects feed efficiency, water quality stability, and ecosystem balance. This paper proposes AquaGPT, a multimodal Transformer framework that embeds expert knowledge to address challenges in feeding management in aquaponic systems. The system employs a distributed sensing network, processed by modality-specific encoders and projected into a shared embedding space, to capture synchronized acoustic, sensor, environmental, and visual data. A spatiotemporal attention mechanism models complex fish–environment interactions, and a differentiable expert rule layer then aligns predictions with established aquaculture practice to improve interpretability. A dynamic weight allocation strategy further enhances robustness by prioritizing reliable modalities under noisy or incomplete input conditions. Experiments on the Full Fish Interaction Analysis (FFIA) dataset demonstrate that AquaGPT outperforms state-of-the-art multimodal fusion baselines by up to 4.5% accuracy under severe noise. Also, it achieves a 30% reduction in parameters compared to similar models, enabling real-time deployment on edge devices. These results highlight AquaGPT’s potential for sustainable, precision aquaponics, thereby improving resource efficiency and reducing environmental impact.