Algorithm Application of Speech Recognition Technology Based on Multimodal Information in Human-Computer Interaction

Multimodal human-computer interaction (MMHCI) has been considered a key direction during the evolution of the human-computer interaction (HCI) domain. To speed up the technical advancement of HCI systems, this investigation initially combines attention mechanism (AM) with the Connectionist Temporal Classification (CTC) algorithm for developing a speech recognition (SR) framework. The framework is further enhanced by applying an optimized artificial swarming algorithm to the end-to-end SR framework, resulting in an SR model appropriate for the MMHCI system. The experimental outcomes indicate that the proposed improved swarm algorithm achieves better convergence, solving the solution set with less difference from the real solution set, the hypervolume metric converges at 0.861, and the inverse generation distance converges at 0.194. The SR model surpasses others in terms of evaluation parameters containing accuracy, precision, and recall, while achieving the lowest error rates in various aspects as the sentence level, word, and character, which are correspondingly $0.035,0.037$, and 0.036. This ensures better accuracy of recognition while maintaining a balanced on-access processing rate. As to a system of multimodal interaction, the relatively higher scores of mean opinion and objective grading for voice quality were received in the experimental group (EG), instead of the control group (CG). The speech processing technology developed in such investigation is quite essential for elevating interaction efficiency together with user experience, offering valuable insights and guidance for research in the areas of HCI and SR.