IntelliMAD: a Framework for Secure Machine Learning Models Evaluation and Fine-tuning in Federated Setting

In contemporary artificial intelligence (AI) appli- cations, Machine Learning (ML) models are core components enabling AI-driven functionalities, yet selecting and fine-tuning a model and its hyperparameters remains challenging. ML model architecture, as well as key model training parameters, such as the number of training epochs, batch size, and learning rate, are highly dependent on both the dataset modalities and the specific task resolved in a particular application. More sophisticated execution setups may require determination of additional environment-related parameters, such as identifica- tion of computational capabilities required for execution of a particular AI-driven task, or discovery and establishment of desired security-related parameters. The accurate configuration of these parameters is crucial for a robust and secure end-user application, as their selection influences the performance and reliability of the resulting Foundation Model (FM). In order to investigate the set of the most suitable parameters, the iterative and systematic experimentation is required. IntelliMAD is a comprehensive framework that enables FM evaluation and fine- tuning in a Federated Learning manner. The framework facil- itates the investigation and determination of execution environ- ment parameters and security mechanisms. It provides a unified entry point for experiment settings, where each aspect of model training and aggregation is handled as a configurable parameter. In this work, the application of IntelliMAD is demonstrated in the case of implementation of the Model Anomaly Detection mechanism in Federated Learning.