Voice Chat Robot with Integrated Floating Projection

This paper proposes a novel lip-synced animated voice chatbot system to enhance user interaction and participation. The system uses the Wii Balance Board as an input trigger and leverages Azure Bot Service for speech recognition and Language Studio for keyword extraction. For unmatched user queries, the system will hand them over to ChatGPT for processing and convert the responses into voice. A key contribution of this research is to modify the Wav2Lip model to enable continuous standby and real-time generation of lip-sync animations during audio detection, significantly improving system performance by eliminating duplicate face detections. The system has four different animation states (idle, listening, thinking, and answering), providing a smooth and responsive interactive experience.