End-to-end deep fusion of hyperspectral imaging and computer vision techniques for rapid detection of wheat seed quality

Seeds are essential to the agri-food industry. However, their quality is vulnerable to biotic and abiotic stresses during production and storage, leading to various types of deterioration. Real-time monitoring and pre-sowing screening offer substantial potential for improved storage management, field performance, and flour quality. This study investigated diverse deterioration patterns in wheat seeds by analyzing 1000 high-quality and 1098 deteriorated seeds encompassing mold, aging, mechanical damage, insect damage, and internal insect infestation. Hyperspectral imaging (HSI) and computer vision (CV) were employed to capture surface data from both the embryo (EM) and endosperm (EN). Internal seed quality was further assessed using scanning electron microscopy, dissection, and standard germination tests. Both conventional machine learning algorithms and deep convolutional neural networks (DCNN) were employed to develop discriminative models using independent datasets. Results revealed that each data source contributed valuable information for seed quality assessment (validation set accuracy: 65.1–89.2 %), with the integration of HSI and CV showing considerable promise. A comparison of early and late fusion strategies led to the development of an end-to-end deep fusion model. The decision fusion-based DCNN model, integrating HSI-EM, HSI-EN, CV-EM, and CV-EN data, achieved the highest accuracy in both training (94.3 %) and validation (93.8 %) sets. Applying this model to seed lot screening increased the proportion of high-quality seeds from 47.7 % to 93.4 %. These findings were further supported by external samples and visualizations. The proposed end-to-end decision fusion DCNN model simplifies the training process compared to traditional two-stage fusion methods. This study presents a potentially efficient alternative for rapid, individual kernel quality detection and control during wheat production. • Many types of deteriorated wheat seeds were studied to mimic complex production. • Data fusion of HSI and CV techniques outperformed that within a single technique. • Decision-level fusion outperformed data-level fusion in seed quality detection. • An end-to-end decision fusion DCNN model was built to detect wheat seed quality. • Model's robustness was further validated by external datasets and visualizations.