Declarative MLOps Pipelines for Enterprise Platforms: A Domain-Specific Language Approach
This work presents a declarative Domain-Specific Language (DSL) for defining Machine Learning Operations pipelines across heterogeneous enterprise platforms. The DSL abstracts away platform-specific implementation details while preserving semantic intent, enabling portable, reproducible, and maintainable ML workflows. Built on a multi-stage compiler architecture with sophisticated semantic mapping, the language translates high-level pipeline definitions into optimized platform-specific code for AWS SageMaker, Google Vertex AI, and Databricks. Integrated experiment tracking, versioning, and compliance features address enterprise governance requirements. The type system of the language offers strong validation features, identifying mistakes early in the development process and aiding communication between technical and non-technical participants. The DSL's clear semantics greatly lessen the cognitive burden on ML professionals, and its composability allows for building modular pipelines that fit enterprise architecture patterns. Assessment shows notable decreases in development complexity and improved cross-platform compatibility while preserving performance equivalence with native solutions. The declarative method fundamentally changes how ML pipelines are constructed and overseen, creating a basis for reproducible, scalable, and compliant MLOps within corporate settings.
Institutions
- Anna University, Chennai (IN)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2025-08-04
- DOI
- https://doi.org/10.5281/zenodo.16735207
- Primary Topic
- Software System Performance and Reliability
- Type
- article
- Field-Weighted Citation Impact
- 0.00