Energy consumption assessment in embedded AI: Metrological improvements of benchmarks for edge devices

This manuscript proposes a new method to improve the MLCommons protocol for measuring power consumption on Microcontroller Units (MCUs) when running edge Artificial Intelligence (AI). In particular, the proposed approach (i) selectively measures the power consumption attributable to the inferences (namely, the predictions performed by Artificial Neural Networks - ANN), preventing the impact of other operations, (ii) accurately identifies the time window for acquiring the sample of the current thanks to the simultaneous measurement of power consumption and inference duration, and (iii) precisely synchronize the measurement windows and the inferences. The method is validated on three use cases: (i) Rockchip RV1106, a neural MCU that implements ANN via hardware neural processing unit through a dedicated accelerator, (ii) STM32 H7, and (iii) STM32 U5, high-performance and ultra-low-power general-purpose microcontroller, respectively. The proposed method returns higher power consumption for the two devices with respect to the MLCommons approach. This result is compatible with an improvement of selectivity and accuracy. Furthermore, the method reduces measurement uncertainty on the Rockchip RV1106 and STM32 boards by factors of 6 and 12, respectively. • A new method to improve the MLCommons protocol for measuring power consumption on Microcontroller Units when running edge Artificial Intelligence. • The proposed approach selectively measures the power consumption attributable to the inferences preventing the impact of other operations. • The proposed approach accurately identifies the time window for acquiring the sample of the current thanks to the simultaneous measurement of power consumption and inference duration. • Compared to the MLCommons protocol, the proposed method significantly reduces measurement uncertainty by a factor of 6 on the Rockchip RV1106 and 12 on the STM32 boards.