Using conformal prediction to quantify uncertainty in machine learning models for monitoring animal communities

Passive acoustic monitoring is increasingly used to survey biodiversity. However, studies on acoustic indices and their effectiveness for monitoring animal communities have produced mixed results, particularly in assessing bird species richness. To improve performance, experts have recommended combining multiple indices, and researchers are increasingly turning to machine learning methods, such as Random Forest regression, for this purpose. However, most machine learning algorithms have a limitation in that they do not provide well-calibrated measures of uncertainty for their predictions. Quantifying uncertainty with reliable and accurate prediction intervals is essential for making informed ecological and environmental management decisions. To address this issue, we propose extending existing machine learning approaches by applying a framework known as Conformal Prediction, which offers guaranteed coverage prediction intervals under the exchangeability assumption. We demonstrate the utility of the proposed workflow using acoustic and bird data collected at twenty-four biodiverse sites in the lowland subtropical rainforest of the Big Scrub region in New South Wales, Australia. Specifically, we apply GPR-CP, a recently developed approach that combines Conformal Prediction with Gaussian Process Regression, a popular non-parametric supervised machine learning technique. GPR-CP is one of the few full conformal prediction approaches currently available for regression, offering higher statistical efficiency, especially for smaller datasets, which are common in ecology. We demonstrate that GPR-CP can reliably quantify uncertainty in predictions of bird species richness derived from acoustic indices. In addition, we discuss how integrating the CP framework into broader ecological applications that use machine learning can support more informed management decisions.