Evaluating Accuracy in Legal Research: A Comparative Analysis of Traditional and Artificial Intelligence-Driven Methods

Legal professionals are challenged with performing accurate legal research to meet professional standards. The use of artificial intelligence (AI) presents a potential for enhanced efficiency. However, knowledge and process gaps exist, questioning the accuracy of results generated by AI. The uncertainty of accurate results poses a risk for poor outcomes for all stakeholders. This thesis aimed to quantitatively measure the accuracy of AI legal research in comparison to traditional legal research. Accuracy and relevance were measured by analyzing case precedent results using three legal research platforms, Westlaw, Nexis Uni, and Chat GPT. The three platforms represented traditional and AI-driven research, free access and paid subscriptions. Queries (n=25) were designed by an attorney, experienced in legal research. The queries were designed in both Boolean and Natural Language Processing (NLP) formats and represented federal and state cases, as well as an array of the most common legal practice specialties. The most accurate combination of legal research was Westlaw with Boolean queries. Eight percent of the AI-driven Chat GPT results yielded hallucinated non-existent cases. Results align with findings and concerns published in current peer-review literature, and the results validated the anecdotal assumption among some specialists in legal bibliography that Boolean queries consistently out-perform NLP. Implications for practice include the continuation of Boolean language use and education, the need for education surrounding the optimal use of AI, and the need for a validation system.