On the Analysis of a BERT-based Domain-Specific Question Answering Models for Indian Legal System

The Indian legal system is too large and complex which includes the largest Constitution in the world, thousands of laws, central laws, thousands of acts and many bilateral, regional, and multilateral treaties and agreements with different countries. This makes it difficult to understand and memorize the contents for legal professionals as well as the general public. Some individuals and government officials take advantage of this fact by doing unlawful acts and avoid punishments due to the unawareness in public. So, we are developing a closed domain extractive model for question answering on the Indian legal system. The motivation behind developing a Question Answering System (QAS) on Indian laws is to enhance the accessibility of legal information for both legal professionals and the general public. PDFs of legal acts were collected from the Government website from which Question Answering dataset was created. Dataset contains automatic system generated questions using T5 question generated model and then manual cleaning such as removing wrong questions, minor changes were done. Models used for QA are DistilBERT, RoBERTa, BERT Large. BERT Large outperforms the other models with cosine similarity of 0.944, BLEU score of 0.781, Exact match(EM) with 0.72. Question answering model provides short and precise answers of the questions with context from which the answer was extracted to avoid giving wrong information.