Trusting Predictions of Acoustic Classification

Trusting the predictions of acoustic classifiers is crucial in today's military. Having confidence in the predictions of machine learning models used by military systems for surveillance, reconnaissance, or on the battlefield is essential for human adoption. If the system is unreliable, soldiers who could potentially benefit from its assistance may experience adverse effects. In light of this, we present an algorithm that calculates neural network classifier confidence scores, enabling us to assess the trustworthiness of classifiers. In this inquiry, we utilize the Latent Space-Statistical (LS-Stat) confidence score technique to compute the confidence score of an acoustic signal classifier. The LS-Stat approach utilizes the latent features extracted from the penultimate layer of the trained classifier model to compute the confidence score. The classifier model employs the Acoustic-Seismic Classification Identification Data Set (ACIDS) for training and validation. Using the trained classifier, the confidence scores of the classifier predictions are compared using LS-Stat, Monte Carlo dropout, and prediction probability values or SoftMax algorithms. Our findings indicate that the LS-Stat method outperforms the other methods in terms of accuracy, prediction, and confidence score.