Allocating distributed AI/ML applications to cloud–edge continuum based on privacy, regulatory, and ethical constraints

There is an increasing need for practitioners to address legislative and ethical issues in both the development and deployment of data-driven applications with AI/ML due to growing concerns and regulations, such as GDPR and the EU AI Act. Thus, the field needs a systematic framework for assessing risks and helping to stay compliant with regulations in designing and deploying software systems. Clear and concise descriptions of risks associated with each model and data source are needed to guide the design without acquiring deep knowledge of the regulations. In this paper, we propose a reference architecture for an ethical orchestration system that manages distributed AI/ML applications on the cloud–edge continuum and present a proof-of-concept implementation of the main ideas of the architecture. Our starting point is the methods already in use in the industry, such as model cards, and we extend the idea of model cards to data source cards and software component cards, which provide practitioners and the automated system with relevant information in actionable form. With the metadata card based orchestration system and information about the risk levels of the target infrastructure, the users can create deployments of distributed AI/ML systems that fulfill the regulatory and other requirements.