MLOps and LLMOps: A Systematic Survey of Deploying, Monitoring, and Operating Large Language Models in Production
Machine learning operations (MLOps) emerged to close the gap between a model that works in a notebook and a system that works in production, and the arrival of large language models (LLMs) has forced the field to evolve again. Language models introduce new deployment problems — enormous model sizes, autoregressive inference, prompt-dependent behavior, and outputs that cannot be checked against a fixed ground truth — that classical MLOps tooling was never designed to handle. This paper presents a systematic survey of MLOps and its language-model successor, LLMOps. It traces the principles of MLOps from continuous integration and delivery for machine learning through model registries, data validation, and monitoring; it presents a taxonomy of production machine-learning systems organized by deployment pattern, serving architecture, and operational risk; it examines how LLM inference is actually served, from memory-efficient attention and paged key-value caches to quantization and speculative decoding; it analyzes monitoring and evaluation for systems whose outputs are free text, including drift detection, LLM-as-a-judge evaluation, retrieval-augmented generation evaluation, and tracing; it surveys the operational layer of LLM applications — prompt versioning, retrieval pipelines, and autonomous agents; and it discusses cost, energy, safety, governance, and compliance as first-class operational concerns. The survey concludes that LLMOps is not a replacement for MLOps but an extension of it: the classical disciplines of versioning, testing, and monitoring remain necessary, and the new work is to extend them to artifacts — prompts, retrieved context, and agent trajectories — that have no counterpart in classical machine learning.
Authors
- Aref Mahmoudpour
Institutions
- Islamic Azad University, Tehran (IR)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-16
- DOI
- https://doi.org/10.5281/zenodo.24268760
- Primary Topic
- Software System Performance and Reliability
- Type
- preprint