A Quantitative Framework for Evaluating Architectural Patterns in ML Systems

Contemporary intelligent systems include both software and machine learning (ML) components. As they grow in complexity and data volume, such ML systems face unique challenges in terms of quality, such as scalability and performance. To overcome them, engineers often use specific architectural patterns; however, their impact on ML systems is difficult to quantify. While software architecture’s impact on traditional systems is well understood, more research is needed for ML systems. This study proposes a framework for quantitative assessment of architectural patterns in ML systems, focusing on scalability and performance metrics for cost-effective CPU-based inference. We integrate these metrics into a systematic evaluation process for the selection of architectural patterns and demonstrate their application through a case study. The approach shown in the paper should enable software architects to objectively analyze and select optimal patterns, addressing key challenges in ML system design.