A ROBUST FRAMEWORK FOR DATA EXTRACTION FROMUNSTRUCTURED SOURCES USING LARGE LANGUAGEMODELS

The exponential growth of unstructured Portable Document Format (PDF) documents in industries, such as material certificates and invoices, necessitates robust methods for structured data extraction. While Large Language Models (LLMs) offer promising solutions, challenges such as dependency on input data quality, prompt sensitivity, hallucinations, and scalability (handling diverse layouts or unstructured data) limit their reliable application in real-world scenarios. This work addresses these limitations by developing a hybrid framework that combines expert-guided evaluation, iterative prompt engineering, and LLM adaptation strategies for accurate and scalable information extraction from complex industrial documents. This thesis conducts a hybrid comparison of data extraction approaches with different LLMs, analyzes the impact of expert-driven prompt engineering, and performs a multi-faceted evaluation of different LLM adaptation methods, including zero-shot, few-shot, and Low-Rank Adaptation (LoRA) fine-tuning, on domain-specific PDF data. Key contributions include: (1) a rigorous evaluation of text extraction tools for varied document types; (2) empirical evidence of the superiority of expert-guided few-shot prompting over zero-shot and fine-tuned approaches; and (3) a framework that bridges the gap between general-purpose LLMs and industrial document processing. Results demonstrate that Tesseract and pdfplumber are promising tools for layout-preserved text extraction across various PDF types. Furthermore, few-shot prompting significantly enhances performance, with open-weight models like Mistral 7B achieving notable accuracy (87.11% data retrieval), bridging a significant gap compared to commercial models like GPT-4 and Deepseek when paired with optimized prompts. In contrast, fine-tuning on small datasets yields inconsistent improvements, underscoring the limitations of this approach for complex structural tasks where spatial context also conveys data sequence. This thesis provides actionable insights for deploying LLM-based extraction in real-world applications while balancing accuracy and adaptability.