Image text recognition and post-processing of language model based on DBNet and TrOCR

With the rapid development of deep learning technology, natural Scene Text Recognition (STR) has become a key research topic in the field of computer vision. While traditional two-stage OCR systems (text detection and text recognition) perform well in standard text scenarios, they still face robustness and accuracy challenges in real-world applications such as complex backgrounds, font variations, irregular layouts, and long text dependencies. To overcome these problems, this paper proposes an end-to-end text recognition framework that combines multi-modal feature enhancement with dynamic language correction to significantly improve recognition performance. The framework first uses differentiable binarization network (DBNet) for text detection, and uses adaptive thresholds to deal with the text regions with low contrast and blurred borders. Next, a Pathformer module is added to act as a bridge between text detection and recognition, and the combination of convolutional neural network and Transformer architecture is used to strengthen the spatial path features and global semantic information of the text region. Then, the TrOCR model based on Transformer decodes the enhanced features to generate preliminary recognition results. Finally, BERT language model is used to correct the context of the low confidence characters, which can effectively correct the confusion and syntax errors of the near words. The experimental results show that the proposed system achieves higher recognition accuracy compared with the baseline model (DBNet+TrOCR) on ICDAR2015, MSRA-TD500 and other data sets. Especially in the processing of curved text and long text, the system shows obvious advantages.