EchoCSL: Chinese sign language recognition based on acoustic sensing

The accuracy of acoustic sensing-based sign language recognition technology is affected by various factors, such as environmental noise and the presence of multiple moving objects. This paper introduces EchoCSL, an enhanced Chinese sign language recognition system utilizing acoustic sensing technology, aiming to improve the system's anti-interference capability and ensure accurate semantic expression of the output text. EchoCSL employs the modulation of a Zadoff-Chu (ZC) sequence as transmitted signals and extracts the channel impulse response (CIR) of the echo signal to enhance the system's resistance to interference. Additionally, text matching technology based on natural language is utilized to ensure the accuracy of the text output from the ConvAttnLSTMNet model and the Connectionist Temporal Classification (CTC) decoder. Experimental data show that the proposed EchoCSL method achieves an average accuracy of 99.8% for wordlevel sign language recognition in an indoor environment and 88.4% for sentence-level recognition in an outdoor environment. These results demonstrate that EchoCSL significantly outperforms existing sign language recognition methods based on Doppler frequency shift, highlighting its potential for improved accuracy and reliability in sign language recognition.