Integrating Large Language Models with Digital Avatars: A Case Study of the Educational Domain

In recent years, the integration of Large Language Models (LLMs) with digital avatars has gained significant attention, particularly in educational and customer service domains. This paper presents a comprehensive case study of applying LLMs to enhance the intelligence and interaction capabilities of digital avatars. By fine-tuning LLMs with localized knowledge bases, the study aims to improve domain-specific responsiveness and real-time interaction in educational settings. Additionally, the integration of advanced Text-to-Speech (TTS) and Speech-to-Text (STT) technologies further enhances the ability to interact naturally with users. Theoretical analysis delves into the self-attention mechanisms of LLMs and their interaction with knowledge bases, while empirical results validate the improvements in user experience and response accuracy. By addressing the challenges of latency, response accuracy, and emotional expressiveness, this study offers valuable insights for the future development of multimodal digital avatars and their applications in education.

Authors

Institutions

Publication Details

Published
2024-12-13
DOI
https://doi.org/10.1145/3718391.3718411
Primary Topic
Speech and dialogue systems
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Integrating Large Language Models with Digital Avatars: A Case Study of the Educational Domain

Jingze Liu, Li‐Hua Gong, Yiming Wang, Jianing Sun et al.
Speech and dialogue systems
article

Integrating Large Language Models with Digital Avatars: A Case Study of the Educational Domain

Jingze Liu, Li‐Hua Gong, Yiming Wang, Jianing Sun, Yongle Du, Bin Zhou, Yimei Zhu
article en

Abstract

In recent years, the integration of Large Language Models (LLMs) with digital avatars has gained significant attention, particularly in educational and customer service domains. This paper presents a comprehensive case study of applying LLMs to enhance the intelligence and interaction capabilities of digital avatars. By fine-tuning LLMs with localized knowledge bases, the study aims to improve domain-specific responsiveness and real-time interaction in educational settings. Additionally, the integration of advanced Text-to-Speech (TTS) and Speech-to-Text (STT) technologies further enhances the ability to interact naturally with users. Theoretical analysis delves into the self-attention mechanisms of LLMs and their interaction with knowledge bases, while empirical results validate the improvements in user experience and response accuracy. By addressing the challenges of latency, response accuracy, and emotional expressiveness, this study offers valuable insights for the future development of multimodal digital avatars and their applications in education.

Chongqing University (CN)
Quality Education
Openalex Percentile: Top 72%
Speech and dialogue systems
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.