Predicting Operational Events in Mechanized Weed Control Operations by Offline Multi-Modal Data and Machine Learning Provides Highly Accurate Classification in Time Domain

Monitoring of operations has become a critical activity in forestry, aiming to provide the data required by planning and production management. Conventional methods, on the other hand, come at a high expense of resources. A neural network was trained, validated, and tested in this study based on multi-modal data to classify relevant operational events in mechanized weed control operations. The architecture of a neural network was tuned in terms of the number of hidden layers and neurons, and the regularization term was set at various values to obtain optimally tuned models for three data modalities: triaxial acceleration data coupled with speed extracted from GNSS signals (AS), triaxial acceleration (A), and speed alone (S). In the training and validation phase, the models based on AS and A achieved a very high classification accuracy, accounting for 92 to 93% when considering four relevant events. In the testing phase, which was run on unseen data, the classification accuracy reached figures of 91 to 92%, indicating a good generalization ability of the models. The results point out that multimodal data are able to provide the features for distinguishing events and add spatial context to the monitored operations, standing as a suitable solution for offline, partly automated monitoring. Future studies are required to see how the capabilities of online, real-time technologies such as deep learning coupled with computer vision can add more context and improve classification performance.