Flexible Edge-AI Software Execution Architecture Based on Cloud-Connected Incremental Learning

Edge AI computing transcends the passive execution of predefined software by making inferences in response to inputs. However, AI inference at the edge is only matrix operations with statically loaded weights defined in on-chip code flash memory. The narrow weights configured in the offline learning stage do not produce accurate inferences for various input conditions. Therefore, AI-embedded edge systems have to be efficiently reconfigured in runtime to update the pre-defined weights and models to handle various scenarios. To address these issues this paper proposes flexible edge AI architecture with cloud-connected incremental learning. The proposed architecture updates weights and models through the cloud to adapt to external changes in edge AI in runtime. Also, offloading resource-intensive learning stages to the cloud with an event-driven approach reduces the load on edge. Cloud communication minimizes overhead by updating weights intermittently, based on accuracy under changing environmental conditions. The proposed system demonstrates an average accuracy improvement of over 10% in biased input scenarios, a 49% reduction in training time compared to the Jetson Nano board with learning capability, and a 70% reduction in communication volume compared to cloud code-streaming edge devices. This research focuses on integrating cloud and edge computing to overcome the limitations of embedded devices and develop flexible edge AI systems.