A Unified and Resource-Aware Framework for Adaptive Inference Acceleration on Edge and Embedded Platforms

Efficient and scalable inference is essential for deploying large-scale generative models across diverse hardware platforms, especially in real-time or resource-constrained scenarios. To address this, we propose a novel unified and resource-aware inference optimization framework that uniquely integrates three complementary techniques: sensitivity-aware mixed-precision quantization, heterogeneous sparse attention for reducing attention complexity, and capacity-aware dynamic expert routing for input-adaptive computation. This framework distinctively achieves fine-grained adaptivity by dynamically adjusting computation paths based on token complexity and hardware conditions, offering substantial performance gains and execution flexibility across diverse platforms, including edge devices like Jetson Orin. Implemented using PyTorch 1.13 and ONNX Runtime, our framework demonstrates significant reductions in inference latency and memory usage, alongside substantial throughput improvements in language and image generation tasks, outperforming existing baselines even under constrained GPU environments. Qualitative analyses reveal its fine-grained adaptivity, while robustness tests confirm stable behavior under resource fluctuation and input noise, offering an interpretable optimization approach suitable for heterogeneous deployments. Future work will explore reinforcement-based routing and multimodal inference.