INTEGRATING ARTIFICIAL INTELLIGENCE WITH CLOUD-NATIVE ARCHITECTURES: DESIGN PATTERNS, CHALLENGES, AND FUTURE DIRECTIONS FOCUS: HOW AI MODELS ARE DEPLOYED IN SCALABLE CLOUD ENVIRONMENTS, INCLUDING MICROSERVICES, CONTAINERS, AND SERVERLESS SYSTEMS

Abstract The exponential increase in AI inference workloads has necessitated integration with cloud-native architectures to deliver scalable, low-latency, and cost-effective production environments. The paper discusses design trends for deploying advanced AI models, such as LLMs, computer vision systems, and agentic workflows, across microservices, Kubernetes clusters, and serverless platforms. Among the most important are KServe, a declarative serving system with InferenceGraph; vLLM, a paged, attention-based, high-throughput inference system; Triton Inference Server, a multi-framework GPU optimization; and Ray Serve, a dynamic allocation system. These enable elastic scaling with GPU-affinity Horizontal Pod Autoscalers and model versioning in multi-tenant environments. Cold-starts lasting more than 60 seconds in 95 percent of serverless applications, GPU fragmentation with scheduling conflicts, lack of observability in opaque inference pipelines, security vulnerabilities in shared clusters, and increased energy consumption from sustained inference are all quantitatively analyzed. Future directions all point to AI-native cloud platforms that self-heal autonomously, the edge-cloud continuum of privacy-preserving real-time applications, and sustainable AI through quantization and carbon-conscious scheduling. The definitions of the symbols are as follows: L is the end-to-end inference latency (in milliseconds), T is the throughput (in requests per second), and alpha is the replica scaling factor. The research offers practical lessons for bringing experimental AI to robust enterprise implementations.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-04-10
DOI
https://doi.org/10.5281/zenodo.19606649
Primary Topic
Cloud Computing and Resource Management
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

INTEGRATING ARTIFICIAL INTELLIGENCE WITH CLOUD-NATIVE ARCHITECTURES: DESIGN PATTERNS, CHALLENGES, AND FUTURE DIRECTIONS FOCUS: HOW AI MODELS ARE DEPLOYED IN SCALABLE CLOUD ENVIRONMENTS, INCLUDING MICROSERVICES, CONTAINERS, AND SERVERLESS SYSTEMS

BANGAR RAJU CHERUKURI
Zenodo (CERN European Organization for Nuclear Research)
Cloud Computing and Resource Management
article

INTEGRATING ARTIFICIAL INTELLIGENCE WITH CLOUD-NATIVE ARCHITECTURES: DESIGN PATTERNS, CHALLENGES, AND FUTURE DIRECTIONS FOCUS: HOW AI MODELS ARE DEPLOYED IN SCALABLE CLOUD ENVIRONMENTS, INCLUDING MICROSERVICES, CONTAINERS, AND SERVERLESS SYSTEMS

BANGAR RAJU CHERUKURI
article en

Abstract

Abstract The exponential increase in AI inference workloads has necessitated integration with cloud-native architectures to deliver scalable, low-latency, and cost-effective production environments. The paper discusses design trends for deploying advanced AI models, such as LLMs, computer vision systems, and agentic workflows, across microservices, Kubernetes clusters, and serverless platforms. Among the most important are KServe, a declarative serving system with InferenceGraph; vLLM, a paged, attention-based, high-throughput inference system; Triton Inference Server, a multi-framework GPU optimization; and Ray Serve, a dynamic allocation system. These enable elastic scaling with GPU-affinity Horizontal Pod Autoscalers and model versioning in multi-tenant environments. Cold-starts lasting more than 60 seconds in 95 percent of serverless applications, GPU fragmentation with scheduling conflicts, lack of observability in opaque inference pipelines, security vulnerabilities in shared clusters, and increased energy consumption from sustained inference are all quantitatively analyzed. Future directions all point to AI-native cloud platforms that self-heal autonomously, the edge-cloud continuum of privacy-preserving real-time applications, and sustainable AI through quantization and carbon-conscious scheduling. The definitions of the symbols are as follows: L is the end-to-end inference latency (in milliseconds), T is the throughput (in requests per second), and alpha is the replica scaling factor. The research offers practical lessons for bringing experimental AI to robust enterprise implementations.

Zenodo (CERN European Organization for Nuclear Research)
Analysis Group (United States) (US)
Affordable and clean energy
Openalex Percentile: Top 4%
Cloud Computing and Resource Management
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.