Uniform Memory Architecture for GPU-CPU Resource Sharing in Edge AI Systems

To achieve all the requirements of a real-time system that demands low latency and high-throughput performances, edge AI systems must share resources efficiently over CPUs and GPUs. Explicit data transfers between different memory regions are thus the bottlenecks that also forbid higher performance while moving that complexity on a traditional discrete memory system. In this paper, the study introduces a new UMA that enables edge AI systems to share resources across GPUs and CPUs without any hitch. The proposed UMA reduces data transport overhead, enhances processing performance, and simplifies programming models by incorporating a unified address space, intelligent data placement, coherent caching, and improved memory allocation. This also guarantees optimal data placement and consistency between processing units as the technique dynamically analyzes access patterns of memory. Validating the effectiveness of the suggested strategy, experimental evaluations across multiple edge AI workloads bring about a latency reduction of up to 45% and throughput of 30%. This work lays the foundation for advancements in next-gen edge AI architectures, ushering in AI-powered edge applications that are both responsive and efficient.