MTRD-Net: An RGB-NIR Multimodal Fusion Network for Tomato Ripeness Detection in Complex Greenhouse Environments

Tomato ripeness detection is critical for intelligent greenhouse management and automated harvesting. However, complex backgrounds, severe occlusion, scale variation with numerous small objects, and lighting-induced feature degradation make single-modality detection less robust. To overcome these obstacles, a multimodal tomato ripeness detection network (MTRD-Net) is proposed that fuses red-green-blue (RGB) and near-infrared (NIR) imagery (RGB-NIR) through a dual-path framework with feature fusion at the P4 and P5 scales. First, a C3k2-DEDR module combining Cross-Stage Partial with Kernel Size 2 (C3k2) and a Dynamic Extraction and Dual Refinement (DEDR) Block (DEDRBlock) enhances discriminative feature capture in complex backgrounds. Second, we design a Multi-Scale-Aware Cross-Modal Transformer Fusion (MACTF) module. It improves target representation against scale changes and occlusions, while simultaneously integrating cross-modal cues in spatial and semantic domains. Third, a Small-Object Feature Enhancement Neck (SOFEN) enhances fine-grained feature preservation for small tomatoes. Experimental results on the public multimodal tomato ripeness detection dataset validate the effectiveness of MTRD-Net, which surpasses several mainstream models by achieving 76.7% precision, 81.0% recall, and mAP values of 84.8% (at IoU = 0.5) and 70.2% (at IoU = 0.5:0.95). Additional evaluation on Multi-Scenario Multi-Modality Benchmark (M3FD) further demonstrates the general applicability of the proposed multimodal architecture to generic object detection tasks.