Enhancing Security and Applicability of Local LLM-Based Document Retrieval Systems in Smart Grid Isolated Environments

The deployment of large language models (LLMs) in closed-network industrial environments remains constrained by privacy and connectivity limitations. This study presents a retrieval-augmented question-answering system designed to operate entirely offline, integrating local vector embeddings, ontology-based semantic enrichment, and quantized LLMs, while ensuring compliance with industrial security standards like IEC 62351. The system was implemented using OpenChat-3.5 models with two quantization variants (Q5 and Q8), and evaluated through comparative experiments focused on response accuracy, generation speed, and secure document handling. Empirical results show that both quantized models delivered comparable answer quality, with the Q5 variant achieving approximately 1.5 times faster token generation under limited hardware. The ontology-enhanced retriever further improved semantic relevance by incorporating structured domain knowledge into the retrieval stage. Throughout the experiments, the system demonstrated effective performance across speed, accuracy, and information containment—core requirements for AI deployment in security-sensitive domains. These findings underscore the practical viability of offline LLM systems for privacy-compliant document search, while also highlighting architectural considerations essential for extending their utility to environments such as smart grids or defense-critical infrastructures.