Enhancing Intangible Cultural Heritage Dance Formation Choreography in VR via Large Language Models

The recent popularity of consumer-grade virtual reality devices has enabled users to experience immersive Intangible Cultural Heritage (ICH) dances in virtual environments, where ICH is defined by UNESCO as traditions or living expressions inherited from our ancestors. As in real-world choreography, the dancers in a virtual environment should be instructed in how to perform their dance. However, this can be a time-consuming and challenging task with the current Virtual Reality (VR) controllers. To address these challenges, we introduce an innovative choreography system leveraging Large Language Models (LLMs) to automate dance creation and presentation in VR. To achieve this goal, we first establish a corpus dataset comprising almost five thousand prompt phrases to describe fundamental movements and formation changes within dance formation choreography. We then leverage an LLM to understand these prompts, thereby comprehending the commands issued by users during the interaction. Based on the understanding provided by the LLM, these prompts are converted into primitive commands that our system can execute. This allows the virtual character to perform corresponding actions to complete the dance formation choreography. We conduct three user studies with 94 subjects to validate the proposed approach. Through user studies, we find that the proposed approach can help users generate high-quality ICH dances regarding human likeness, appropriateness, and operability. For the demo, please refer to this link .

Authors

Institutions

Publication Details

Journal
ACM Transactions on Multimedia Computing Communications and Applications
Published
2026-11-12
DOI
https://doi.org/10.1145/3820370
Primary Topic
Human Motion and Animation
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Enhancing Intangible Cultural Heritage Dance Formation Choreography in VR via Large Language Models

Junjun Pan, Shuo Yan, Feng Zhou, Yu‐Kun Lai et al.
ACM Transactions on Multimedia Computing Communications and Applications
Human Motion and Animation
article

Enhancing Intangible Cultural Heritage Dance Formation Choreography in VR via Large Language Models

Junjun Pan, Shuo Yan, Feng Zhou, Yu‐Kun Lai, Paul L. Rosin, Wei Zhou, Junkai Rao, Yong Hu, J. P. Dai
article en

Abstract

The recent popularity of consumer-grade virtual reality devices has enabled users to experience immersive Intangible Cultural Heritage (ICH) dances in virtual environments, where ICH is defined by UNESCO as traditions or living expressions inherited from our ancestors. As in real-world choreography, the dancers in a virtual environment should be instructed in how to perform their dance. However, this can be a time-consuming and challenging task with the current Virtual Reality (VR) controllers. To address these challenges, we introduce an innovative choreography system leveraging Large Language Models (LLMs) to automate dance creation and presentation in VR. To achieve this goal, we first establish a corpus dataset comprising almost five thousand prompt phrases to describe fundamental movements and formation changes within dance formation choreography. We then leverage an LLM to understand these prompts, thereby comprehending the commands issued by users during the interaction. Based on the understanding provided by the LLM, these prompts are converted into primitive commands that our system can execute. This allows the virtual character to perform corresponding actions to complete the dance formation choreography. We conduct three user studies with 94 subjects to validate the proposed approach. Through user studies, we find that the proposed approach can help users generate high-quality ICH dances regarding human likeness, appropriateness, and operability. For the demo, please refer to this link .

ACM Transactions on Multimedia Computing Communications and Applications
North China University of Technology (CN), Peng Cheng Laboratory (CN), Cardiff University (GB), Beihang University (CN)
Human Motion and Animation
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon