A data-centric multi-modal ai framework for weed detection and classification in smart agriculture

In the agricultural sector, weeds significantly influence productivity as they compete with crops for essential resources and attract pests, leading to substantial yield losses. Traditional weed management practices, including manual weeding or non-selective herbicide spraying, are labor-intensive, inefficient, and environmentally harmful. Precision weed management promotes sustainable farming through the integration of computational technologies such as artificial intelligence, computer vision, and deep learning. In this context, Unmanned Aerial Vehicle (UAV) and ground-based imaging technologies have significantly advanced automated weed detection and classification. However, most existing approaches rely on a single imaging modality, limiting their ability to capture complementary field-level and plant-level information required for accurate weed detection under scale variation, severe occlusions, overlapping vegetation, and complex field conditions. Furthermore, although recent studies have reported improvements through advanced model architectures, comparatively less attention has been given to dataset quality, annotation consistency, and the effective integration of UAV and ground-based imagery, which may influence the reliability and applicability of AI models in real-field environments. To address these limitations, this paper presents a data-centric, multi-modal, and interpretable AI framework for weed detection and classification. The proposed framework emphasizes the importance of high-quality annotated data while integrating the complementary strengths of UAV and ground-based imagery. Weed regions are localized using a U-Net backbone with attention mechanisms and edge-preserving strategies, enabling accurate segmentation under challenging field conditions. Deep feature representations extracted from segmented weed regions are fused and classified using an ensemble of machine learning models to improve discrimination among visually similar weed species. The proposed framework is evaluated on MH-Weed16, a self-constructed and rigorously annotated real-field dataset comprising 8,283 images with 18,395 annotated instances of 16 weed species. Experimental results achieve IoU scores of 0.80 for UAV imagery and 0.92 for ground-based imagery, with corresponding Dice scores of 0.89 and 0.96, respectively. The classification module achieves an overall accuracy of 91%. Overall, the proposed framework provides an effective data-centric multi-modal approach for automated weed detection and classification in agriculture real-fields.