Task Framing Outweighs Model Scale in Contract Clause Extraction: Pre-Registered Experiments with Frontier LLMs and Small Fine-Tuned Specialists on CUAD

We benchmark four frontier language models (Claude Opus 4.8, Claude Opus 4.6, GPT-5.2, GPT-4o) and a series of small fine-tuned specialists on CUAD contract clause extraction, scored with the official AUPR metric through one identical pipeline, with every raw prediction file published. Three findings emerge. First, the frontier gap is category-shaped rather than uniform: a fine-tuned 14B specialist that trails the best frontier model by 17 points overall still beats GPT-4o in 25 of 40 clause categories and beats all four frontier models simultaneously in six. Second, task framing dominates model scale. Reframing the specialist from answer generation to extractive span prediction, with the base model, data, and budget held fixed, moves test AUPR from 0.291 to 0.389, while growing the extractive backbone 28x (0.5B to 14B) moves dev AUPR only from 0.284 to 0.303. Frontier scaling shows the same pattern: GPT-4o and GPT-5.2, a full model generation apart, score within 0.002 of each other. Third, we pre-registered a public prediction that the extractive 14B would clear 0.50 test AUPR; it missed, and an analysis of the miss localizes the remaining gap to the deliberately lean training recipe rather than architecture or scale. All code, predictions, and the trained adapter are public at https://github.com/ashish24142/nightwing

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-07-24
DOI
https://doi.org/10.5281/zenodo.21530248
Primary Topic
Artificial Intelligence in Law
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Task Framing Outweighs Model Scale in Contract Clause Extraction: Pre-Registered Experiments with Frontier LLMs and Small Fine-Tuned Specialists on CUAD

ASHISH SINGH
Zenodo (CERN European Organization for Nuclear Research)
Artificial Intelligence in Law
preprint

Task Framing Outweighs Model Scale in Contract Clause Extraction: Pre-Registered Experiments with Frontier LLMs and Small Fine-Tuned Specialists on CUAD

ASHISH SINGH
preprint en

Abstract

We benchmark four frontier language models (Claude Opus 4.8, Claude Opus 4.6, GPT-5.2, GPT-4o) and a series of small fine-tuned specialists on CUAD contract clause extraction, scored with the official AUPR metric through one identical pipeline, with every raw prediction file published. Three findings emerge. First, the frontier gap is category-shaped rather than uniform: a fine-tuned 14B specialist that trails the best frontier model by 17 points overall still beats GPT-4o in 25 of 40 clause categories and beats all four frontier models simultaneously in six. Second, task framing dominates model scale. Reframing the specialist from answer generation to extractive span prediction, with the base model, data, and budget held fixed, moves test AUPR from 0.291 to 0.389, while growing the extractive backbone 28x (0.5B to 14B) moves dev AUPR only from 0.284 to 0.303. Frontier scaling shows the same pattern: GPT-4o and GPT-5.2, a full model generation apart, score within 0.002 of each other. Third, we pre-registered a public prediction that the extractive 14B would clear 0.50 test AUPR; it missed, and an analysis of the miss localizes the remaining gap to the deliberately lean training recipe rather than architecture or scale. All code, predictions, and the trained adapter are public at https://github.com/ashish24142/nightwing

Zenodo (CERN European Organization for Nuclear Research)
Artificial Intelligence in Law
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.