A Local-First AI/ML Intelligence Layer for Retrieval-Augmented Generation in a Spring Boot Application

Large language models (LLMs) exhibit strong generative capability but suffer from hallucination, stale knowledge, and an inability to cite sources. Retrieval-Augmented Generation (RAG) mitigates these weaknesses by grounding responses in externally retrieved documents. This paper presents the design and implementation of an AI/ML intelligence layer that integrates RAG into an existing Spring Boot / Kotlin application with minimal new infrastructure. The layer reuses the application's embedded SQLite database as a vector store, serialises embedding vectors as little-endian BLOBs, and performs brute-force cosine-similarity search in process. A local, Docker-native Ollama runtime provides both the embedding model (nomic-embed-text) and the generative model (llama3.2), eliminating external API keys. We describe the layered architecture, the ingestion and query pipelines, the data model, tenant-scoped security, and deployment topology, and we discuss the scalability and quality trade-offs of the local-first approach. The result is a reference pattern for adding private, citeable, document-grounded chat to conventional enterprise web stacks without adopting a heavyweight vector database or third-party LLM subscription.