Cross-platform edge deployment of machine learning models: a model-driven approach

Abstract Deploying machine learning (ML) models on edge devices presents unique challenges, arising from the different environments used for developing ML models and those required for their deployment, leading to a gray area of competence and expertise between ML engineers and application developers. In this paper, we explore the use of model-driven engineering to simplify the deployment of ML models on edge devices, specifically smartphones. We present a DSL for the specification of the ML serving pipelines (pre- and postprocessing of data before and after inference), together with a model interpretation approach that allows to make changes to the pipeline during runtime, thus removing the need to re-release an application upon changes to a pipeline. We followed a design science approach, in which we elicited requirements through an initial artifact study and interviews with engineers at an industrial partner. This was followed by the design and implementation of a lightweight, JSON-based domain-specific language designed to describe ML serving pipelines, along with an accompanying Flutter library to execute the pipelines during runtime. A preliminary evaluation with four developers shows the potential of this approach to increase development speed, decrease the amount of code required to make changes to an ML serving pipeline, and make less-experienced engineers more confident contributing to the domain.