Generative AI Nonverbal Communication: A Unified Framework for Bidirectional and Unidirectional Information Exchange Through Facial Expressions, Gestures, and Body Movements

Generative artificial intelligence (AI) has rapidly expanded the possibilities of human–AI communication through natural language, speech synthesis, image generation, and increasingly sophisticated embodied interaction. However, communication between humans and generative AI is still predominantly organized around verbal and text-based exchanges, while nonverbal expressions are frequently treated as supplementary outputs, isolated recognition tasks, or components of embodied agent animation. This paper introduces Generative AI Nonverbal Communication (GANC), a unified conceptual framework that positions nonverbal expression as an independent and generalizable information-exchange mechanism between humans and generative AI systems. The framework encompasses facial expressions, gestures, hand movements, body movements, posture, gaze, and other visually or physically observable communicative behaviors. Unlike approaches that primarily investigate how an artificial agent generates gestures or interprets human expressions, GANC explicitly incorporates four communication configurations: AI-to-human unidirectional communication, human-to-AI unidirectional communication, bidirectional human–AI communication, and hybrid communication combining nonverbal and verbal modalities. The framework further distinguishes expressive generation, perceptual interpretation, semantic representation, communicative intention, contextual adaptation, interaction coordination, and feedback-based correction as interconnected functional components. It proposes that nonverbal communication should not be restricted to accompanying spoken language but should also support information transmission independently of speech and text. This conceptual expansion creates opportunities for gesture-based commands, facially expressed intentions, movement-based responses, silent interaction, and continuous multimodal communication. The paper establishes a theoretical foundation, presents a unified architectural model, identifies research challenges, and proposes an evaluation framework for future empirical investigation. To the best of our knowledge, this paper introduces the first explicitly unified conceptual framework within the reviewed literature that organizes generative AI nonverbal communication around both communication directions, multiple nonverbal modalities, and independent as well as integrated communication configurations. The proposed framework provides a foundation for developing more expressive, adaptive, accessible, and socially responsive human–AI communication systems.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-17
DOI
https://doi.org/10.5281/zenodo.24298247
Primary Topic
Social Robot Interaction and HRI
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Generative AI Nonverbal Communication: A Unified Framework for Bidirectional and Unidirectional Information Exchange Through Facial Expressions, Gestures, and Body Movements

Jincheng Zhang
13 links
Zenodo (CERN European Organization for Nuclear Research)
Social Robot Interaction and HRI
preprint

Generative AI Nonverbal Communication: A Unified Framework for Bidirectional and Unidirectional Information Exchange Through Facial Expressions, Gestures, and Body Movements

Jincheng Zhang
preprint en

Abstract

Generative artificial intelligence (AI) has rapidly expanded the possibilities of human–AI communication through natural language, speech synthesis, image generation, and increasingly sophisticated embodied interaction. However, communication between humans and generative AI is still predominantly organized around verbal and text-based exchanges, while nonverbal expressions are frequently treated as supplementary outputs, isolated recognition tasks, or components of embodied agent animation. This paper introduces Generative AI Nonverbal Communication (GANC), a unified conceptual framework that positions nonverbal expression as an independent and generalizable information-exchange mechanism between humans and generative AI systems. The framework encompasses facial expressions, gestures, hand movements, body movements, posture, gaze, and other visually or physically observable communicative behaviors. Unlike approaches that primarily investigate how an artificial agent generates gestures or interprets human expressions, GANC explicitly incorporates four communication configurations: AI-to-human unidirectional communication, human-to-AI unidirectional communication, bidirectional human–AI communication, and hybrid communication combining nonverbal and verbal modalities. The framework further distinguishes expressive generation, perceptual interpretation, semantic representation, communicative intention, contextual adaptation, interaction coordination, and feedback-based correction as interconnected functional components. It proposes that nonverbal communication should not be restricted to accompanying spoken language but should also support information transmission independently of speech and text. This conceptual expansion creates opportunities for gesture-based commands, facially expressed intentions, movement-based responses, silent interaction, and continuous multimodal communication. The paper establishes a theoretical foundation, presents a unified architectural model, identifies research challenges, and proposes an evaluation framework for future empirical investigation. To the best of our knowledge, this paper introduces the first explicitly unified conceptual framework within the reviewed literature that organizes generative AI nonverbal communication around both communication directions, multiple nonverbal modalities, and independent as well as integrated communication configurations. The proposed framework provides a foundation for developing more expressive, adaptive, accessible, and socially responsive human–AI communication systems.

Zenodo (CERN European Organization for Nuclear Research)
Social Robot Interaction and HRI
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon