Mari: Automating Responses to Thai Legal Debt Queries Using Generative AI and Retrieval-Augmented Generation

Over the years, there have been lots of reports that point out the problems of Thailand’s high personal debt levels. With limited public knowledge of legal debt rights, these problems result in right violation between the creditors and debtors. Mari is designed to address such issues. It is an AI-powered system that offers automated legal consultations pertaining to the Thai Debt Collection Act of 2015. Main focus is on aiding creditors, debtors, or any individual, to understand their rights which contribute a boon of addressing information gaps, and enhancing accessibility to legal knowledge without requiring costly consultations. Mari is a website that uses a large language model (LLM) with integrated Retrieval-Augmented Generation (RAG) to ensure the precise responses for the required scope and semantics of the concerned legal statute. In order to achieve the effectiveness of these responses, we considered several LLM models, namely OpenThaiGPT, Typhoon, and SambaNova under the evaluation based on a set of human-generated multiple-choice and open-end questions. The comparison score showed that SambaNova’s LLM, with English prompt engineering, was the best for this case. Lastly the semantic difference, representing the level of hallucination effect, of the final model is measured by the BERT score with satisfaction of 0.726.