FL-former: Chinese automatic speech recognition architecture under the federated large model

In the context of the rapid development of automatic speech recognition (ASR) technology, improving model accuracy while protecting data privacy has become an important research topic. This paper proposes a new federated large model framework to improve the accuracy of ASR while effectively solving industry privacy issues. It leverages each customer's unique data sources for individual training, shares parameters for trained models, and enhances the ability to extract features from heterogeneous sources. In addition, the framework also incorporates a large language model to fully utilize its powerful semantic understanding capabilities to further improve the accuracy of speech recognition. The proposed model architecture achieves a 27.5% performance improvement compared to a single speech recognition model.