Speech-driven sentence type classification in Chokri using traditional and transfer learning methods

Sentence classification typically relies on textual information in high-resource languages. However, it is rather challenging to perform such tasks in under-resourced languages that often lack sufficient text or speech data to support such tasks. This paper examines the effectiveness of audio features derived from speech data (speech features) in the endangered tonal language Chokri for classifying sentence types using traditional machine learning and transfer learning algorithms. Results show that speech features can be effectively used for sentence classification tasks without relying on textual information. Two approaches are employed to achieve the sentence classification tasks. First, we extracted various individual and combined speech features, including Mel Frequency Cepstral Coefficients (MFCCs), Chroma, and other spectral features, as numerical data and applied the random forest algorithm for traditional machine learning. The results show that the average F1 score for MFCCs alone, MFCCs with other features, and MFCCs with Chroma is 86% each, with the highest average of 88% achieved by combining all features. These findings underscore the critical role of MFCCs in speech-based sentence classification and the varying contributions of other features. The second approach involves converting speech into spectrogram and MFCC images and applying transfer learning models like ResNet101 and InceptionResNet. This method improves classification accuracy by 10%, resulting in 97–98% accuracy. This study suggests further experimentation with various grammatical structures by demonstrating the effectiveness of speech features alone for sentence classification, thereby expanding the possibilities for language processing tasks in resource-limited scenarios.