Multimodal perception system with optical feedback based on triboelectric nanogenerator and quantum dot light-emitting synaptic device

Abstract Artificial intelligence is developing towards multimodal perception, and display technology is evolving into intelligent human-computer interaction. Owing to the intuitive and anti-interference advantages of optical outputs, it is essential to apply them to artificial multisensory systems. Herein, we propose a multimodal perception system with optical feedback that utilizes an integrated triboelectric nanogenerator (TENG) in conjunction with a quantum dot light-emitting synaptic device (QLESD), where TENG serves as a receiver for pressure signals and QLESD functions as both ultraviolet (UV) light and temperature receptor. Three distinct signals were memorized and processed in QLESD, which ultimately outputs light and electrical signals that combined these three stimuli. The excitatory postsynaptic current (EPSC) and EP brightness (EPSB) of QLESD stimulated by pressure signal from TENG were systematically investigated. Notably, EPSC and EPSB of the QLESD were enhanced with increasing contact frequency. Furthermore, as both the temperature and UV light intensity increased gradually, the suppression effect on synaptic signal transmission and memory became more pronounced. The successfully integration of temperature and UV light in collaborative modulation of pressure signals has been achieved, showcasing remarkable potential applications in robotics and human-computer interaction.