Comparison of AutoML frameworks for effective analysis in MAPE-K self adaptive systems

Abstract Typically, the MAPE-K (Monitoring, Analysis, Planning, Executing–Knowledge) loop represents the cornerstone of Self-Adaptive Systems (SASs). These systems require highly intelligent analysis mechanisms to ensure reliable performance under changing conditions. Recently, Automated Machine Learning (AutoML) has been applied sparingly to mitigate the labor- and time-intensive challenges associated with the Machine Learning (ML) integration in the MAPE-K analysis phase. However, there is still no clear guideline to identify the most suitable AutoML solutions. In this paper, we evaluate six state-of-the-art AutoML frameworks–TPOT, GAMA, H2O, NaiveAutoML, TabPFN, and AutoGluon–on the DeltaIoT SAS multi-class datasets, using standard performance and runtime metrics. We aim to determine the best AutoML solutions for the MAPE-K Analyzer, comparing their results with those of conventional ML. To our knowledge, this paper is the first AutoML benchmark applied for SASs. Our results indicate that AutoML frameworks can deliver strong predictive performance for AI-driven intelligent SASs. Notably, NaiveAutoML and AutoGluon provide a satisfactory balance of speed and accuracy, making them more suitable for both offline and online use.