Top 2 at ALQAC 2024: Large Language Models (LLMs) for Legal Question Answering

This paper details the methods proposed by the UIT-DarkCow team during their participation in the 4th Automated Legal Question Answering Competition (ALQAC 2024). Specifically, the team focused on two main tasks: legal document retrieval and legal question answering (LQA). For the legal document retrieval task, the goal was to return articles related to a given question. An article is considered “relevant” if it contains information that helps answer the question. To achieve this, the team applied a Vietnamese sentence parsing method, combined with two prominent retrieval methods: BM25, a traditional and proven method, and sentence transformer, a recently emerged and highly regarded method in natural language processing. This combination significantly improved the accuracy of finding relevant documents for the questions. In the LQA task, we faced three types of questions: true/false questions, multiple-choice questions, and free-text questions. To address these questions, the team used the most advanced technique currently available: prompt engineering with large language models. This technique allows the model to understand and accurately answer legal questions. The results of the ALQAC 2024 competition demonstrated the effectiveness of the methods applied by the team. UIT-DarkCow achieved third in the legal document retrieval task and second in the LQA task.