Clause-Level Contract Risk Classification: Comparing Rules, TF-IDF, Sentence Embeddings, and LEGAL-BERT on CUAD

Small businesses often sign contracts without legal review because professional review is slow and expensive. We study clause-level risk classification with labels derived from the expert-annotated CUAD dataset. Using a split by contract, so that clauses from one contract never appear in both training and test data, we compare four methods on a shared test set of 1,650 clauses from 77 contracts over three seeds: a hand-written rule baseline, sentence embeddings with logistic regression, TF-IDF with logistic regression, and fine-tuned LEGAL-BERT. LEGALBERT reaches macro F1 0.867, against 0.796 for TF-IDF, 0.712 for the embedding classifier, and 0.240 for the rules. A paired contract-level bootstrap puts the macro F1 gain over TF-IDF at 0.073 . On the HIGH class, LEGAL-BERT has higher precision than TF-IDF but lower recall. Nearly half of the HIGH clauses that LEGAL-BERT misses (31 of 63 in one seed) are Uncapped Liability clauses predicted as LOW, a pattern that appears to reflect wording overlap with the Cap On Liability category,which our mapping labels LOW. We state the limits: labels are a category-based proxy for risk, the study covers English CUAD contracts only, and we report no evaluation on Indian contracts. The system is a screening aid and not legal advice.