Using image recognition and machine learning to automatically identify feller-buncher activities in forest operations

Accurate work element classification is essential for estimating machine utilization, productivity, and operational costs in mechanized forest harvesting. Traditional time studies rely on field observations or manual annotation of operational videos, making productivity analysis labour-intensive, time-consuming, and expensive. Previous studies applied onboard computer records to automate harvesting time studies. However, these methods have limited capability to distinguish individual work elements without additional sensor data. To the best of the author’s knowledge, no previous research applied automatic classification of feller-buncher activities using operational video data collected in Canada. To address this gap, the application of machine learning techniques for the automatic classification of feller-buncher operational activities using multiple source data is investigated. The dataset consisted of more than 320 hours of operational time studies from 793 videos collected from four feller-bunchers operating in British Columbia, Canada. Two machine learning frameworks are developed. The MATLAB framework extracts video and audio features and evaluates multiple machine learning algorithms for cutting versus non-cutting and productive versus non-productive classification. A Python framework employs Convolutional Neural Networks to classify activities directly from image frames using both individual-machine and multi-machine datasets. The MATLAB framework demonstrated that video features consistently outperformed audio features and combined video-audio features. The Fine K-Nearest Neighbour model using video features achieved the best performance, with classification accuracies of 86.11% for cutting versus non-cutting activities and 86.95% for productive versus non-productive activities. Feature importance analysis further indicated that color histograms, edge density and brightness are the most influential factors of model performance. The generalized Python models achieved classification accuracies ranging from 67.78% and 85.50%, demonstrating the feasibility of automatically classifying feller-buncher activities directly from operational images across multiple machines. The relatively lower accuracy of the Python framework is partly due to the use of individual images instead of continuous videos, as single frames provide limited temporal information. Overall, this research demonstrated that machine learning and image recognition techniques could effectively automate work element classification for feller-buncher operations. The proposed frameworks provide a practical foundation for reducing reliance on manual time studies and supporting more efficient productivity analysis and operational decision-making in feller-buncher operations.