Optimized Assertiveness-Cost Evaluation: An Innovative Performance Measuring Method for Machine Learning Models

The increasing use of Machine Learning (ML) across various sectors has rendered model evaluation a progressively complex task. Ensuring models exhibit both high performance and usability necessitates in-depth evaluations of both assertiveness and computational cost. To address these needs, this study introduces the Optimized Assertiveness-Cost Evaluation (OACE) method, a novel general performance evaluation framework for ML that integrates joint analyses of assertiveness and cost metrics, based on a holistic evaluation that balances both aspects. The methodology and implementation extends to different ML contexts, allowing for comprehensive and consistent replication of results from a reproducible algorithm. In the case study, Deep Learning models were developed for the classification of three-dimensional shapes in real-world environments. The results indicated that the ResNet-50 architecture achieved the highest score with the proposed method, being 20% more better than the other architectures, demonstrating its effectiveness in identifying models that maximize assertiveness while minimizing computational cost.