A Study on MRAM Across Memory Hierarchies for Edge KWS Adaptive Power Cycling

To process Keyword Spotting (KWS) at the edge, achieving high energy efficiency is crucial for long-term operation in battery-powered devices. Since speech occurs intermittently, adaptive power cycling can be used to conserve energy by powering the circuit off during inactive periods. An ultra-low power Voice Activity Detection (VAD) monitors for speech while the SoC remains off. Only once speech is detected again, the processing System-on-Chip (SoC) is booted, loading neural network (NN) parameters into volatile memory. Non-volatile Flash memory, especially off-chip SPI-Flash, leads to energy-intensive data access/booting. Additionally, the unpredictable timing and duration of speech events creates a trade-off between costly booting from off-chip Flash and continuous power consumption if the system continues processing (always-on). For always-on processing, SRAM memory leakage consumes significant power, draining the battery. Hence, this study investigates the impact of using on-chip Magnetoresistive Random Access Memory (MRAM) as a) non-volatile memory (NVM) for booting and b) as read-only memory for NN parameters, directly serving a MAC engine. Using a set of Recurrent Neural Networks (RNNs), our results demonstrate that MRAM reduces the booting energy consumption for parameter loading by 345x compared to off-chip Flash. Overall, on-chip MRAM-based operation enhances energy efficiency by up to 5x compared to always-on processing and by 280x compared to power cycling with off-chip Flash. Thus, on-chip MRAM significantly reduces the energy penalty associated with booting, making MRAM-based power cycling a viable option. Employing on-chip MRAM for NN weight storage improves energy efficiency by 20% over SRAM-based memory for a 60 kB RNN, while larger RNNs benefit more from scratch-padonly computation. These findings highlight the potential of on-chip MRAM in enhancing energy efficiency for edge speech processing, particularly in scenarios with intermittent speech activity.