Creating Resources and Evaluating the Impact of OCR Quality on Information Retrieval: A Case Study in the Geoscientific Domain

The evaluation paradigm in Information Retrieval (IR) requires a test collection with documents, queries, and relevance judgments. Creating such collections demands significant human effort, mainly to provide relevance judgments. As a result, there are still many domains and languages that, to this day, lack a proper evaluation testbed. To bridge this gap, we developed REGIS (Retrieval Evaluation for Geoscientific Information Systems), a test collection for the geoscientific domain in Portuguese. The documents in REGIS are in PDF. Optical Character Recognition (OCR) is typically used to extract the textual contents of scanned texts. The output of OCR can be noisy, especially when the quality of the scanned image is poor, which in turn can impact downstream tasks such as Information Retrieval. This work evaluates the impact of OCR extraction and correction on IR. Our results have shown significant differences in IR metrics for the different digitization methods.