Multimodal AI-Driven Edge Framework for Energy-Efficient Human Activity Monitoring in Smart Grids

This publication presents a Multimodal AI-Driven Edge Framework designed to provide energy-efficient monitoring of human activity in Smart Grids. Current approaches for monitoring energy networks utilize Centralized Cloud Infrastructures that place heavy demands on computers and create very high latencies in communication as well as limiting the ability to create real-time recommendations to optimize energy consumption. The proposed framework will utilize a combination of compressed and full deep learning pipelines combined with quantized inference models, and tracking methods based on TinyML technology; allowing for the utilization of multimodal data (occupancy, movement, and usage) directly on edge devices (NVIDIA Jetson Nano, Raspberry Pi 5, ARM-based Processors) therefore minimizing memory footprint, computational load, and power consumption while generating very accurate analysis across all )modality types tested. Test results demonstrate 42%-70% decreased latency, 30%-55% decreased memory footprint and that they operate in an energy-efficient manner making them ideal for continued field deployment. Our Multimodal AI-Driven Edge Framework enables the collection of activity data to develop predictive energy management systems, implement adaptive demand responses and support real-time energy optimization for Smart Grids without dependence on cloud connectivity. We have demonstrated lightweight, multimodal AI technology that will augment the energy efficiency, scalability, and intelligence of future Smart Grids.