Combining Retrieval-Augmented Generation and Fine-tuning of Large Language Models to Enhance Port Industry Question-Answering Systems

In this research, we develop a new hybrid architecture that combines Retrieval-Augmented Generation (RAG) and LLMs (Large Language Models) in order to address the specific gaps in the domain question answering systems for the maritime port industry. Our approach mitigates the generic LLMs’ limitations concerning domain-specific queries through a combination of knowledge retrieval specific to the industry and adaptive modelling with implemented parameters. The overarching evaluation protocol designed for investigating the approach was both quantitative and qualitative by using expert judgement which showed marked improvement in justifiable gains across multi-dimensional stand-alone approaches regarding factual correctness, accuracy of use of maritime terms, and compliance with relevant policies. The hybrid system achieved 23% improvement in nDCG@5 scores alongside exceeding 90% accuracy in terminology used in maritime context, maintaining sub-second response times under typical operational loads. The domain experts we consulted in the study were particularly impressed by the balance the system struck between factual precision and contextual understanding of complex operational scenarios. Such improvement enables decision-makers for critical operational environments to greatly trust the system within their active contexts. This research demonstrates a practical methodology for balancing the adaptation of a domain to the computational algorithms of a system in specialised professional application domains that require high factual precision but allow for context interpretation.