Meta-Features Extracted from Use of kNN Regressor to Improve Sugarcane Crop Yield Prediction

Accurate crop yield prediction is essential for sugarcane growers, as it enables them to predict harvested biomass, guiding critical decisions regarding acquiring agricultural inputs such as fertilizers and pesticides, the timing and execution of harvest operations, and cane field renewal strategies. This study is based on an experiment conducted by researchers from the Commonwealth Scientific and Industrial Research Organisation (CSIRO), who employed a UAV-mounted LiDAR and multispectral imaging sensors to monitor two sugarcane field trials subjected to varying nitrogen (N) fertilization regimes in the Wet Tropics region of Australia. The predictive performance of models utilizing multispectral features, LiDAR-derived features, and a fusion of both modalities was evaluated against a benchmark model based on the Normalized Difference Vegetation Index (NDVI). This work utilizes the dataset produced by this experiment, incorporating other regressors and features derived from those collected in the field. Typically, crop yield prediction relies on features derived from direct field observations, either gathered through sensor measurements or manual data collection. However, enhancing prediction models by incorporating new features extracted through regressions executed on the original dataset features can potentially improve predictive outcomes. These extracted features, nominated in this work as meta-features (MFs), extracted through regressions with different regressors on original features, and incorporated into the dataset as new feature predictors, can be utilized in further regression analyses to optimize crop yield prediction. This study investigates the potential of generating MFs as an innovation to enhance sugarcane crop yield predictions. MFs were generated based on the values obtained by different regressors applied to the features collected in the field, allowing for evaluating which approaches offered superior predictive performance within the dataset. The kNN meta-regressor outperforms other regressors because it takes advantage of the proximity of MFs, which was checked through a projection where the dispersion of points can be measured. A comparative analysis is presented with a projection based on the Uniform Manifold Approximation and Projection (UMAP) algorithm, showing that MFs had more proximity than the original features when projected, which demonstrates that MFs revealed a clear formation of well-defined clusters, with most points within each group sharing the same color, suggesting greater uniformity in the predicted values. Incorporating these MFs into subsequent regression models demonstrated improved performance, with R¯2 values higher than 0.9 for MF Grad Boost M3, MF GradientBoost M5, and all kNN MFs and reduced error margins compared to field-measured yield values. The R¯2 values obtained in this work ranged above 0.98 for the AdaBoost meta-regressor applied to MFs, which were obtained from kNN regression on five models created by the researchers of CSIRO, and around 0.99 for the kNN meta-regressor applied to MFs obtained from kNN regression on these five models.