E2E energy monitoring for AI inference in mobile networks

The increasing adoption of Artificial Intelligence (AI), particularly large language (LLMs) and vision-language models (VLMs) has led to a sharp rise in energy demand. While most of the studies predominantly assess energy consumption during training and inference, they often neglect the energy required to transport contextual data---such as text, images, or video---from far-edge devices to AI models, especially over mobile networks. We measure and analyze energy consumption for AI inference both on model-level and network-level. Our approach leverages a combined cross-layer and in-band network telemetry approach to estimate application-level energy usage. Our experiments show that the energy used by the network can be on par with that used by energy efficient AI models for certain tasks. Furthermore, we also estimate the total CO2 emissions of these inference workflows. These results highlight the critical need to incorporate network consumed energy into sustainable AI system design.