Domain Knowledge Integration with Chain of Thought and Knowledge Prompting in Text to Motion Generation

In this paper, we propose a Zero-shot Chain-of-Thought Knowledge Prompting (ZCKP) technique to diversify text-to-motion generation by infusing domain expertise into pretrained models without requiring retraining. Current text-to-motion generation models rely heavily on limited datasets, which lack diversity. Generating specialized motions, such as those influenced by age, gender, or physical disabilities, typically requires retraining on custom datasets, an approach that is both computationally expensive and resource-intensive. ZCKP addresses this limitation by integrating instruction-driven CoT prompting into text-to-motion models. This method dynamically incorporates domain-specific knowledge, such as inverse kinematics and physical laws, enabling the generation of specialized, realistic, and physically plausible motion sequences in a zero-shot setting. To evaluate its effectiveness, the model was tested in age-based motion generation for toddlers and elderly individuals (healthy and disabled). Inter-class diversity scores demonstrate the model’s ability to produce varied motions, with Euclidean distances ranging from 56.23 to 78.15 and cosine value from 0.53 to 0.81. These scores indicate that motions across different age groups are distinct and diverse. Meanwhile, intra-class homogeneity scores confirm the generation of realistic and consistent motions within each group, with Euclidean distances ranging from 12.36 to 36.97 and cosine value from 0.12 to 0.32. These results highlight the model’s capacity to maintain coherence and naturalness while introducing diversity. By reducing the dependency on retraining and computational overhead, ZCKP offers a scalable and flexible solution for domain-specific text-driven motion generation.