Unified Data Engineering Frameworks for LLM-Powered Cloud Applications at Scale

Large Language Models (LLMs) now dominate the field of intelligent cloud-native applications by providing strong generative functionality with context-aware automated operations. The achievement of large-scale benefits from LLMs depends on implementing an integrated data engineering solution which combines multi-source data pipelines and live data processing while supporting operational governance measures and maximizing scalability and cost effectiveness. This paper develops an integrated data engineering system designed for cloud applications using LLMs which applies modular design patterns with distributed control systems and automated pipeline optimization algorithms. The framework combines data ingestion along with feature transformation and metadata management capabilities and LLM-centric model services that incorporate Mops and Limos practices. Here it is explained both the design structure of the system alongside deployment tactics and measurement standards which support higher operational output and processing capacity. The system provides multiple cloud platform capabilities including AWS Azure and GCP so businesses can deploy LLMs in multi-tenant environments with standardized data management and security features. Experimental assessments prove the framework supplies scalability while minimizing latency and achieving operational stability which prepares it as a base for future-generation enterprise AI applications.

Authors

Publication Details

Published
2025-05-09
DOI
https://doi.org/10.1109/ginotech63460.2025.11076719
Citations
1
Primary Topic
Cloud Computing and Resource Management
Type
article
Field-Weighted Citation Impact
3.25
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Unified Data Engineering Frameworks for LLM-Powered Cloud Applications at Scale

Balkrishna Patil, Balaji Krishnan, Srinivas Reddy Kosna, Arshiya Shirdi
1 citations
Cloud Computing and Resource Management
3.25
article

Unified Data Engineering Frameworks for LLM-Powered Cloud Applications at Scale

Balkrishna Patil, Balaji Krishnan, Srinivas Reddy Kosna, Arshiya Shirdi
article en
1 citations

Abstract

Large Language Models (LLMs) now dominate the field of intelligent cloud-native applications by providing strong generative functionality with context-aware automated operations. The achievement of large-scale benefits from LLMs depends on implementing an integrated data engineering solution which combines multi-source data pipelines and live data processing while supporting operational governance measures and maximizing scalability and cost effectiveness. This paper develops an integrated data engineering system designed for cloud applications using LLMs which applies modular design patterns with distributed control systems and automated pipeline optimization algorithms. The framework combines data ingestion along with feature transformation and metadata management capabilities and LLM-centric model services that incorporate Mops and Limos practices. Here it is explained both the design structure of the system alongside deployment tactics and measurement standards which support higher operational output and processing capacity. The system provides multiple cloud platform capabilities including AWS Azure and GCP so businesses can deploy LLMs in multi-tenant environments with standardized data management and security features. Experimental assessments prove the framework supplies scalability while minimizing latency and achieving operational stability which prepares it as a base for future-generation enterprise AI applications.

Industry, innovation and infrastructure
Openalex Percentile: Top 7%
Cloud Computing and Resource Management
3.25
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Unified Data Engineering Frameworks for LLM-Powered Cloud Applications at Scale — Balkrishna Patil, Balaji Krishnan, et al. · (2025) | TGRS Research Map | TGRS