Improving Efficiency in Multi-Modal Autonomous Embedded Systems Through Adaptive Gating

The parallel advancement of AI and IoT technologies has recently boosted the development of multi-modal computing (<inline-formula><tex-math notation="LaTeX">$M^{2}C$</tex-math></inline-formula>) on pervasive autonomous embedded systems (AES). <inline-formula><tex-math notation="LaTeX">$M^{2}C$</tex-math></inline-formula> takes advantage of data from different modalities such as images, audio, and text and is able to achieve notable improvements in accuracy. However, achieving these accuracy gains often comes at the cost of increased computational complexity and energy consumption. Furthermore, the presence of numerous advanced sensors in these systems significantly contributes to power consumption, exacerbating the issue of limited power resources. Collectively, these challenges pose difficulties in deploying <inline-formula><tex-math notation="LaTeX">$M^{2}C$</tex-math></inline-formula> on small embedded devices with scarce energy resources. In this article, we propose an <b>A</b>daptive <b>M</b>odality <b>G</b>ating technique called <i>AMG</i> for in-situ <inline-formula><tex-math notation="LaTeX">$M^{2}C$</tex-math></inline-formula> applications. The primary objective of <i>AMG</i> is to conserve energy while preserving the accuracy advantages of <inline-formula><tex-math notation="LaTeX">$M^{2}C$</tex-math></inline-formula>. To achieve this goal, <i>AMG</i> incorporates two first-of-its-kind designs. Firstly, it introduces a novel semi-gating architecture that enables partial modality sensor power gating. Specifically, we devise the de-centralized <i>AMG</i> (<i>D-AMG</i>) and centralized <i>AMG</i> (<i>C-AMG</i>) architecture. The former buffers raw data on sensors while the latter buffers raw data on the computing board, which are suitable for different edge scenarios respectively. Secondly, it facilitates a self-initialization/tuning process on the AES, which is supported by carefully-built analytical model. Extensive evaluations demonstrate the effectiveness of <i>AMG</i>. It achieves a 1.6x to 3.8x throughput higher than other power management methods and improves the lifespan of AES by 10% to 280% longer within the same energy budget, while satisfying all performance and latency requirements across various scenarios.