Harnessing Named Entity Recognition to Extract Key Information from Unstructured Documents

The extraction of information from unstructured documents is performed in a wide range of business processes including Know Your Customer (KYC). Such business processes require significant human resources and the main blocking factor to automate such processes is that these documents are unstructured. In other words, these documents do not follow a pre-defined layout, and the layout varies from client to client. It is beyond the control of the business process owners to predict the layout of the documents to be received for the business process and the sought data from such documents are not located in the same area. Commercially available document processing engines such as AI Builder require collections of all possible layouts during training. Otherwise, the performance parameters (recall, precision and F1 score) do not meet the benchmark for automation. In this study, we propose a Named Entity Recognition (NER) based solution to seamlessly extract key information from unstructured documents irrespective of their layouts. The unstructured documents used are annual account documents of UK-based companies and the extracted key information are Company Name, Company Number and Accounting year. The documents for the training and testing dataset were downloaded from the Companies House web portal leveraging Power Automate Desktop. The developed NER-based architecture was successfully trained leveraging the created dataset and the model achieved an impressive precision of 91 %, a recall of 97% and F1 score of 94%. The output from the trained model was compared with the ones from a trained AI Builder model. In contrast to the trained AI Builder model, the trained NER model performed better, and it must be noted that the developed architecture utilized solely open-source frameworks and libraries.