A checklist of quality concerns for architecting ML-intensive systems

Machine learning components are being deployed across nearly every business sector and their importance is continually growing. However, the engineering practices for building these systems remain poorly understood compared to those for conventional software systems. This work provides practical guidance to support architects in designing and implementing machine learning-intensive systems, and identifies areas where there are gaps in understanding and achievement. Building on our prior research, we developed a checklist of quality concerns for architects of machine learning-intensive systems. This checklist was iteratively refined through expert interviews and subsequently validated in a workshop with experienced architects. The main result of this work is a comprehensive list of 40 checks, organized into two main categories and 16 subcategories. Also, we present the results of a workshop where the importance and degree of achievement of each check was assessed by 25 practicing architects of ML-intensive systems. The findings of this study contribute to a better understanding of the unique challenges of ML-intensive systems and offer initial guidance to practitioners, and researchers on areas where future work should be directed. The findings of this study offer valuable support to architects in addressing the unique challenges of ML-intensive systems and provide guidance to practitioners and researchers in terms of where future work should be focused.