In‐Sensor Reservoir Computing System Based on Multifunctional Optoelectronic Synaptic Transistor for Efficient Human Activity Recognition

Abstract With the increasing demand for intelligent systems in transportation and healthcare, conventional vision‐based human activity monitoring and recognition (HAMR) systems face challenges. The physical separation of sensing, storage, and computation within these systems leads to exorbitant power consumption and significant latency. Moreover, they struggle to handle multidimensional data effectively and are severely limited when dealing with visual occlusions. In this research, a multifunctional opto‐electronic synaptic transistor (OEST) is developed. The OEST can seamlessly switch between volatile and non‐volatile functions via optical input and gate modulation. Leveraging the unique characteristics of the multifunctional OEST, an intra–sensor reservoir computation (ISRC) system is constructed. Under optical stimulation, OEST exhibits nonlinear short‐term memory characteristics of 4‐bit reservoir states, which can effectively map multi‐dimensional sequence information to the reservoir state space as a reservoir; under gate programming/erasure, 200 non‐volatile conductance states are realized for constructing the readout layer unit. Based on multidimensional information acquisition from multiple sensors, this ISRC system achieves 96.2% accuracy in human activity recognition, providing a new solution for future high‐performance intra‐sensor computing applications.