From Model to Value: Engineering End-to-End AI Systems with Scalable Data Infrastructure and Continuous ML Delivery

Modeling seldom leads to value. To generate value, data-driven models and their deployments must be engineered as systems. For this, we draw upon years of experience in the space, spanning work at both Research and Development organizations. Value generation comes from otherwise traditionally engineered software systems with an added layer of learning with data. Continuous ML delivery treats ML models as the software they are, embedding ML capabilities within the larger systems. However, due to the unique complexities of ML and special needs, we need an ecosystem that will move the needle on model value generation via scaling ML team productivity on feature engineering, data centricity, and pipeline orchestration. This is made possible through accounts of AI systems deployed in production across different domains, the challenges faced in doing so, and how some of them were overcome. Changing the paradigm to pay attention to the engineering of the entire system with learning as an added capability distills into a set of validated core principles that guide success - comprehensively define the problem, build modular generalizable and reusable systems, be data-centric, leverage constraints to decrease complexity, set performance as the main goal via pipelines over processes, act and optimize for the whole. In this work, we outline the scope of building, training, deploying, and curating the lifetime of machine learning models that ensure continual generation of value. Further, how related research can move conversational assistance and production readiness efforts forward? While the principles have broad applicability across many domains of AI, we discuss in detail the deployment of models for the areas of Natural Language Processing i.e. question answering, entity linking, uncrossable, and visual question answering, due to the presence of production systems in these areas.