ENMA: Integrating Human Affect into Reinforcement Learning for Legged Locomotion

As we introduce robots into increasingly more demanding social environments like office s and hospitals, a need for robots who not only avoid people, but also make them feel comfortable while interacting with them. Integrating affect into the gait of the robot is not sufficient as it does not iclude live reactions of the robot. Human in the loop reinforcement learning where humans annotate their reactions to robots behaviour fails as reinforcement learning needs hundreds or thousands steps to learn sufficiently, and human annotators simply do not have capacity to do so reliably and objectively. Thats where we introduce ENMA, a middleware sitting inbetween high level controller or teleoperator and a low level controller of the robot. It takes in the command of high level controller, robots state and positions and velocities of people around the robot and modulates the command of the high-level controller, based on a predicted affect metrics(valence/arousal) of the nearest person during training, before passing it to the low-level controller. These predicted affect metrics come from a lightweight 3D ResNet model called CAESAR, which we trained on videos of robots, animals and humans and pseudolabels generted by teacher pipeline consiting of Tribe V2 and Swift 4D transformer. ENMA itself is a policy using spiking neural networks, which have achieved similar benchmarks as classical ANNs, are more power efficient and better suited for processing of dynamic data, as they are more neuromorphic then their analog couterparts. We train ENMA in Isaac Lab where we simulate the quadrupedal robot Unitree Go2 interacting with a human avatar with a POV camera. We then let CAESAR annotate this feed online during training which helps us eliminate the need of human annotators in the reinforcement learning process. We validate our framework both in simulation and real-world deployment on a physical Unitree Go2 quadruped across four distinct human-robot interaction scenarios. Our results demonstrate that biologically grounded affective constraints can be successfully integrated into mid-level robot behavior to yield smooth, predictable, and socially appropriate navigation. Preprint, version 1 (November 2026). Video of the real-robot experiments: https://www.youtube.com/watch?v=ZvLrn8Owd1M

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-15
DOI
https://doi.org/10.5281/zenodo.24243976
Primary Topic
Reinforcement Learning in Robotics
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

ENMA: Integrating Human Affect into Reinforcement Learning for Legged Locomotion

Jun Miura, Marek Hromada
Zenodo (CERN European Organization for Nuclear Research)
Reinforcement Learning in Robotics
preprint

ENMA: Integrating Human Affect into Reinforcement Learning for Legged Locomotion

Jun Miura, Marek Hromada
preprint en

Abstract

As we introduce robots into increasingly more demanding social environments like office s and hospitals, a need for robots who not only avoid people, but also make them feel comfortable while interacting with them. Integrating affect into the gait of the robot is not sufficient as it does not iclude live reactions of the robot. Human in the loop reinforcement learning where humans annotate their reactions to robots behaviour fails as reinforcement learning needs hundreds or thousands steps to learn sufficiently, and human annotators simply do not have capacity to do so reliably and objectively. Thats where we introduce ENMA, a middleware sitting inbetween high level controller or teleoperator and a low level controller of the robot. It takes in the command of high level controller, robots state and positions and velocities of people around the robot and modulates the command of the high-level controller, based on a predicted affect metrics(valence/arousal) of the nearest person during training, before passing it to the low-level controller. These predicted affect metrics come from a lightweight 3D ResNet model called CAESAR, which we trained on videos of robots, animals and humans and pseudolabels generted by teacher pipeline consiting of Tribe V2 and Swift 4D transformer. ENMA itself is a policy using spiking neural networks, which have achieved similar benchmarks as classical ANNs, are more power efficient and better suited for processing of dynamic data, as they are more neuromorphic then their analog couterparts. We train ENMA in Isaac Lab where we simulate the quadrupedal robot Unitree Go2 interacting with a human avatar with a POV camera. We then let CAESAR annotate this feed online during training which helps us eliminate the need of human annotators in the reinforcement learning process. We validate our framework both in simulation and real-world deployment on a physical Unitree Go2 quadruped across four distinct human-robot interaction scenarios. Our results demonstrate that biologically grounded affective constraints can be successfully integrated into mid-level robot behavior to yield smooth, predictable, and socially appropriate navigation. Preprint, version 1 (November 2026). Video of the real-robot experiments: https://www.youtube.com/watch?v=ZvLrn8Owd1M

Zenodo (CERN European Organization for Nuclear Research)
Toyohashi University of Technology (JP)
Reinforcement Learning in Robotics
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon