Scalable MLOps Pipeline with Complexity-Driven Model Selection Using Microservices

The increasing complexity of integrating modern convolutional neural networks into software systems imposes significant computational demands on machine learning infrastructures. Existing MLOps systems lack mechanisms for dynamic model selection based on dataset complexity, leading to inefficient resource utilization and limited scalability under high-load conditions. This study employs convolutional neural network-based machine learning algorithms for image classification and ensemble methods for quantitative feature classification. The paper presents a self-optimizing machine learning pipeline that integrates a microservices-based architecture with a formal process for estimating image complexity and an optimization-based model selection strategy. The proposed methodology is based on designing an adaptive microservice-based ML pipeline that dynamically reconfigures its computation graph at runtime. The results confirm the effectiveness of the proposed approach for building resilient and high-performance distributed systems. The mechanism proposed in this work enables the adaptive use of modern deep learning algorithms, leading to improved result quality. A comparative analysis with existing approaches demonstrates superiority in model selection complexity, pipeline overhead, and scalability. The outcome of the proposed mechanism is an adaptive algorithm selection process based on bias-related parameters, enabling the selection of the most suitable module for data processing.