Evaluation of Machine Learning algorithm’s selection criteria for satellite image classification, Leveraging accuracy and execution times

Assessing the effectiveness of Machine Learning algorithms for a particular application is essential for establishing selection criteria. This study presents a new evaluation metric, execution time, which improves the algorithm selection process for a specific application. In this study, five Machine Learning algorithms, which are Decision Tree, Random Forest, Support Vector Machine (SVM), Artificial Neural Network (ANN), and Normal Bayes, were utilized for classifying moderate and higher spatial resolution satellite images in two geographical environments, arid and semi-arid regions of Africa. Overall accuracy and the kappa coefficient were used as accuracy metrics alongside the algorithm’s execution time for measuring latency. The results suggest that the Support Vector Machine algorithm is not a cost-effective option for classifying high-resolution satellite images in terms of performance accuracy and execution time, scoring a kappa of 0.82 and requiring more than 12 h. Conversely, Artificial Neural Networks showed better performance in these aspects as it scored a kappa of 0.97 and completed the classification task in under 15 min. Normal Bayes algorithm showed superior performance compared to other Machine Learning algorithms in classification of moderate spatial resolution satellite images, achieving the best balance between high accuracy (kappa ≈ 0.95–0.98) and low latency, requiring less than 2 min. This study introduces and emphasizes execution time as a metric measure to be considered alongside classification accuracy and provides a comprehensive basis for the algorithm selection process for real-time satellite image classification systems under limited hardware environments.