Japanese Legal Judgment Prediction Using ModernBERT and Generative AI-based Information Extraction

Abstract Legal Judgment Prediction (LJP) has emerged as a promising research topic in the legal domain, aiming to assist decision-making processes by predicting judicial outcomes. In the COLIEE 2025 shared task, a new pilot subtask named LJPJT 2025 was introduced, focusing on Japanese civil tort cases. This subtask consists of two binary classification tasks: Tort Prediction (TP) and Rationale Extraction (RE). In the first part of this paper, we propose a simple fine-tuned ModernBERT-based system for LJPJT 2025, which achieved competitive results, including the highest F1 score among all participants in the RE task and a top-ranked performance in the TP task. In addition to these two classification tasks, we further present an exploratory information extraction task that aims to automatically extract textual segments corresponding to the plaintiff’s or defendant’s claims and the alleged tort instances from raw judgment document. This task is designed to automate the annotation process that has so far relied entirely on manual efforts in previous studies. In contrast to the strong results obtained in the TP and RE tasks, our baseline experiments on this new information extraction task revealed considerably lower performance, highlighting the substantial challenges that remain for achieving end-to-end judgment prediction directly from raw judgment document.