Automatic Filtering of Sugarcane Yield Data

Sugarcane mechanized harvesting generates large volumes of data that are used to monitor harvesters’ functionalities. The dynamic interaction of the machine-onboard instrumentation–crop system introduces discrepant and noisy values into the data, requiring outlier detectors to support this complex and empirical decision. This study proposes an automatic filtering technique for sugarcane harvesting data to automate the process. A three-step automated filtering algorithm based on a sliding window was developed and further evaluated with four configurations of the maximum variation factor f and six SW sizes. The performance of the proposed method was assessed by using artificial outliers in the datasets with an outlier magnitude (OM) of ±0.01 to ±1.00. Three case studies with real crop data were presented to demonstrate the effectiveness of the proposed filter in detecting outliers of different magnitudes, compared to filtering by another method in the literature. In each dataset, the proposed filter detected nearly 100% of larger (OM = ±1.00 and ±0.80) and medium (OM = ±0.50) magnitudes’ outliers, and approximately 26% of smaller outliers (OM = ±0.10, ±0.05, and ±0.01). The proposed algorithm preserved wider ranges of data compared to the comparative method and presented equivalent results in the identification of regions with different productive potentials of sugarcane in the field. Therefore, the proposed method retained data that reflect sugarcane yield variability at the row level and it can be used in practical application scenarios to deal with large datasets obtained from sugarcane harvesters.