Noncontact Neuromorphic Emotion Inference via Gaze Interaction

Abstract Understanding social emotion through physiological signals remains highly challenging, as it requires bulky devices and computer systems, limiting its practicality in real-world applications. Here, we propose a neuromorphic approach that decodes affective states from pupillary responses by integrating physiological feature extraction, spiking neural computation, and memristive hardware simulation. Pupil dynamics provide a noncontact physiological signal for studying social-interaction and attentional demands. We derived individual and dyadic features from infrared pupil-diameter recordings from 108 participants and encoded them as spikes for classification by a spiking neural network (SNN). The framework was evaluated in four independent binary tasks defined by the experimental protocol. These tasks examined familiarity and an empathy-related contrast, as well as competition versus cooperation and low versus high attentional demands. With solid spike encoding and a spike-event loss, the SNN trained on combined individual and interactive features achieved performance comparable to the strongest conventional machine learning baselines with an accuracy of 95.4%. We also fabricated h-BN/WSe2 memristors and used their measured conductance characteristics to calibrate a model of RRAM weight mapping and variability. Software-based inference with this model linked the measured device behavior to network performance. A variation-aware training (VAT) strategy that incorporates device-inspired noise during software training improved simulated postmapping accuracy by up to approximately 9 percentage points across the evaluated tasks. Operation-based calculations provided highly efficient estimates of arithmetic and RRAM array-core read energy, achieving approximately 11.28–11.30 fJ per synaptic operation. These results support the use of dyadic pupil features for classification of the studied conditions and demonstrate the benefit of accounting for measured device variability during software training.

Authors

Institutions

Publication Details

Journal
ACS Nano
Published
2026-11-17
DOI
https://doi.org/10.1021/acsnano.6c14473
Primary Topic
Advanced Memory and Neural Computing
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Noncontact Neuromorphic Emotion Inference via Gaze Interaction

Sungchul Mun, Yaeeun Han, Abdul Rehman, Bo‐In Park et al.
2 links
ACS Nano
Advanced Memory and Neural Computing
article

Noncontact Neuromorphic Emotion Inference via Gaze Interaction

Sungchul Mun, Yaeeun Han, Abdul Rehman, Bo‐In Park, Jonghyeon Ko, Siwoo Jeong, Sang‐Hoon Bae, Mincheol Whang, Minho Jin, Dong Won Lee, Ji‐Hoon Kang, Sangin Park, Hyeonu Jeong, Eunsoo Park, Zhihao Xu
article en

Abstract

Abstract Understanding social emotion through physiological signals remains highly challenging, as it requires bulky devices and computer systems, limiting its practicality in real-world applications. Here, we propose a neuromorphic approach that decodes affective states from pupillary responses by integrating physiological feature extraction, spiking neural computation, and memristive hardware simulation. Pupil dynamics provide a noncontact physiological signal for studying social-interaction and attentional demands. We derived individual and dyadic features from infrared pupil-diameter recordings from 108 participants and encoded them as spikes for classification by a spiking neural network (SNN). The framework was evaluated in four independent binary tasks defined by the experimental protocol. These tasks examined familiarity and an empathy-related contrast, as well as competition versus cooperation and low versus high attentional demands. With solid spike encoding and a spike-event loss, the SNN trained on combined individual and interactive features achieved performance comparable to the strongest conventional machine learning baselines with an accuracy of 95.4%. We also fabricated h-BN/WSe2 memristors and used their measured conductance characteristics to calibrate a model of RRAM weight mapping and variability. Software-based inference with this model linked the measured device behavior to network performance. A variation-aware training (VAT) strategy that incorporates device-inspired noise during software training improved simulated postmapping accuracy by up to approximately 9 percentage points across the evaluated tasks. Operation-based calculations provided highly efficient estimates of arithmetic and RRAM array-core read energy, achieving approximately 11.28–11.30 fJ per synaptic operation. These results support the use of dyadic pupil features for classification of the studied conditions and demonstrate the benefit of accounting for measured device variability during software training.

ACS Nano
Kunsan National University (KR), Seoul National University (KR), Sangmyung University (KR), Inha University (KR), Korea University (KR), Washington University in St. Louis (US), Kyungil University (KR), Korean Association Of Science and Technology Studies (KR), Insilicogen (South Korea) (KR), Hanyang University (KR), Korea Institute of Science and Technology (KR), Jeonbuk National University (KR)
National Research Foundation of Korea
Advanced Memory and Neural Computing
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon