Deaf and Mute Communication Transformation: A Unified Framework for Real-Time Scribing and Emotion Recognition

The current inventions and technologies have contributed significantly to the welfare of the human population, especially to the deaf and dumb community. One notable contribution to this community is the aids that can perform the task of converting Speech to Text and vice versa. Though the presence of the conversion systems they do have leaps in accuracy due to the presence of the dependencies in long-form and noisy networks. To address these concerns a machine learning-based pipeline to address the problems of not only speech to text and vice versa but also performs the process in a secure environment. This is achieved with the assistance of the Vision and Language Transformer (ViLT), which conducts the training in the federated setup. The quantitative analysis shows that the proposed pipeline is more efficient than the State of the art in terms of accuracy and privacy.