Lawyering Up: An Evaluation of Large Language Models’ Potential for Transforming Business Law Practice

Recent developments in Large Language Models (LLMs) and advanced conversational agents (CAs) suggest a rapidly evolving capability to handle complex legal tasks. Building on our previous study—which established a benchmark of 60% accuracy in addressing German business law tasks—this paper presents new findings indicating that current-generation LLMs such as Gemini 2.5 Pro can now successfully solve more than 70% of all cases, and more than 80% in specific domains. Moreover, a longitudinal analysis reveals an average annual improvement rate of approximately 13.68%. To arrive at these findings, we administered a corpus of diverse legal prompts to prominent CAs, including different versions of ChatGPT, Claude, Gemini, and emerging model families such as DeepSeek. Performance and reliability were examined, revealing persistent challenges such as domain-specific knowledge gaps. Despite these constraints, the study investigates whether LLM-based tools are beginning to integrate into legal workflows in a manner that is consistent with the theory of disruptive innovation. From automating routine legal processes to supporting complex contract reviews, these systems exhibit the capacity to boost efficiency and reshape the role of legal professionals. At the same time, their limitations underscore the necessity of expert oversight—particularly for novices relying on AI-generated legal advice. By illuminating both the strengths and shortcomings of today’s advanced CAs, this study provides guidance for legal practitioners, policymakers, and researchers aiming to apply LLM technology in business law and beyond.