The Lawyer That Never Thinks: Consistency and Fairness as Keys to Reliable AI

Large Language Models (LLMs) are increasingly used in high-stakes domains like law and research, yet their inconsistencies and response instability raise concerns about trustworthiness.This study evaluates six leading LLMs-GPT-3.5, GPT-4, Claude, Gemini, Mistral, and LLaMA 2-on rationality, stability, and ethical fairness through reasoning tests, legal challenges, and bias-sensitive scenarios.Results reveal significant inconsistencies, highlighting trade-offs between model scale, architecture, and logical coherence.These findings underscore the risks of deploying LLMs in legal and policy settings, emphasizing the need for AI systems that prioritize transparency, fairness, and ethical robustness.

Authors

Institutions

Publication Details

Published
2025-01-01
DOI
https://doi.org/10.18653/v1/2025.acl-long.491
Primary Topic
Law, AI, and Intellectual Property
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

The Lawyer That Never Thinks: Consistency and Fairness as Keys to Reliable AI

A. S. M. Maksud Kamal, Chengjun Wu, Dana Alsagheer, Mohammad Kamal et al.
Law, AI, and Intellectual Property
article

The Lawyer That Never Thinks: Consistency and Fairness as Keys to Reliable AI

A. S. M. Maksud Kamal, Chengjun Wu, Dana Alsagheer, Mohammad Kamal, Weidong Shi
article en

Abstract

Large Language Models (LLMs) are increasingly used in high-stakes domains like law and research, yet their inconsistencies and response instability raise concerns about trustworthiness.This study evaluates six leading LLMs-GPT-3.5, GPT-4, Claude, Gemini, Mistral, and LLaMA 2-on rationality, stability, and ethical fairness through reasoning tests, legal challenges, and bias-sensitive scenarios.Results reveal significant inconsistencies, highlighting trade-offs between model scale, architecture, and logical coherence.These findings underscore the risks of deploying LLMs in legal and policy settings, emphasizing the need for AI systems that prioritize transparency, fairness, and ethical robustness.

University of Houston (US)
Openalex Percentile: Top 90%
Law, AI, and Intellectual Property
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

The Lawyer That Never Thinks: Consistency and Fairness as Keys to Reliable AI — A. S. M. Maksud Kamal, Chengjun Wu, et al. · (2025) | TGRS Research Map | TGRS