CADAEC: Content-Aware Deployment of AI Workloads in Edge-Cloud Ecosystem

The rapid growth of edge devices has revolutionized industrial AI applications, including robotics, autonomous systems, and IoT, where real-time processing is essential. These systems face the challenge of managing concurrent, high-volume workloads across resource-constrained edge devices and cloud infrastructure. A major hurdle is optimizing deep learning model deployment across edge-cloud environments in dynamic conditions, particularly where input quality and noise fluctuate under concurrent demands. This paper introduces a novel optimization framework, that addressed these challenges and dynamically selects the most suitable models from a diverse model zoo and determines optimal deployment locations (edge or cloud). The proposed framework leverages content-aware approach to minimize both communication and computation latency while considering hardware limitations and environmental factors. Using a binary linear programming (BILP) approach, our method efficiently balances model distribution of an AI pipeline, maximizing end-to-end performance. We validate this framework on a robotic AI pipeline in real-world, noise-variant environments, comparing content-aware and content-agnostic deployment strategies. Our results demonstrate significant optimization in deployment latency and system performance under high-concurrency conditions, using both content-agnostic and content-aware approaches, highlighting the framework's robustness and scalability. Additionally, we showed the effectiveness of the content-aware approach over the content-agnostic method in optimizing deployment choices and reducing latency, while maintaining the desired qualitative outcomes of the AI pipeline with different communication set up. This makes the content-aware strategy more suitable for complex, real-world environments where input quality and noise vary significantly. Overall, The proposed method presents a compelling solution for optimizing AI pipelines in edge-cloud ecosystems, offering potential for broader applications domains.