Brazilian Identity Document Recognition Using Computer Vision Techniques

The verification of identity documents is crucial for institutions such as banks, legal entities, and academia. However, manual processing is costly, which makes automation highly beneficial for both efficiency and accuracy. In this context, we propose a pipeline for classifying and extracting information from different Brazilian identity documents, combining computer vision and machine learning techniques. The Brazilian Identity Document Dataset was used to train CNN models for document classification, and U-Net models were employed to perform fields segmentation. We used Tesseract OCR in conjunction with pre- and post-processing techniques to extract textual information. In our experiments, VGG-16 and ResNet-101 achieved the best overall performance, while U-Net also delivered excellent results in field segmentation, reaching 97% Dice score and 94% IoU. Moreover, OCR errors were significantly reduced through post-processing — down to 1% for CPF numbers in the CPF<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> Front class. These findings emphasize the crucial role of post-processing in improving document recognition accuracy.