Learning from Examiners: An Open Embedding Model and a Benchmark for US Patent Prior-Art Retrieval

Prior-art search asks which earlier documents anticipate a set of patent claims or render them obvious, a narrower notion than topical similarity. We study how far a general-purpose text embedding model moves toward that notion when it is fine-tuned on the references patent examiners actually cite, using only data published by the United States Patent and Trademark Office (USPTO). We make three contributions. First, we build a family-level retrieval benchmark from USPTO office actions and Patent Trial and Appeal Board (PTAB) trials: 11,968 queries with graded relevance, full text on both sides, priority-date filtering, audited separation from the training data, and pre-computed vectors that let the evaluation run without a GPU. Second, we release Patent Mechanics Anticipate-0.6B, a 0.6B-parameter embedding model fine-tuned from harrier-oss-v1-0.6b with a contrastive loss on 8,192-token documents that include the description, on cleaned examiner labels and model-mined hard negatives, and averaged over three seeds. Third, we report what each choice buys, for prior-art search and invalidity search separately. On a held-out test half scored once, with the comparisons fixed in advance, the model reaches Recall@10 of 0.538 on a 250,367-family collection, against 0.451 for its base model and 0.326 for BM25 (+8.7 percentage points over the base, 95% CI +7.9 to +9.6); search-time passage scoring, classification agreement and lexical fusion raise this to 0.562. The gain transfers to PTAB invalidity queries that were never used in training. Model, benchmark, vectors, run files and evaluation code are released under permissive licences. Model: https://huggingface.co/patentmechanics/anticipate-0.6bBenchmark: https://huggingface.co/datasets/patentmechanics/prior-art-bench-v1Code: https://github.com/patentmechanics/prior-art-bench

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-20
DOI
https://doi.org/10.5281/zenodo.24394099
Primary Topic
Information Retrieval and Search Behavior
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Learning from Examiners: An Open Embedding Model and a Benchmark for US Patent Prior-Art Retrieval

Rahul Venkatraj
2 links
Zenodo (CERN European Organization for Nuclear Research)
Information Retrieval and Search Behavior
preprint

Learning from Examiners: An Open Embedding Model and a Benchmark for US Patent Prior-Art Retrieval

Rahul Venkatraj
preprint en

Abstract

Prior-art search asks which earlier documents anticipate a set of patent claims or render them obvious, a narrower notion than topical similarity. We study how far a general-purpose text embedding model moves toward that notion when it is fine-tuned on the references patent examiners actually cite, using only data published by the United States Patent and Trademark Office (USPTO). We make three contributions. First, we build a family-level retrieval benchmark from USPTO office actions and Patent Trial and Appeal Board (PTAB) trials: 11,968 queries with graded relevance, full text on both sides, priority-date filtering, audited separation from the training data, and pre-computed vectors that let the evaluation run without a GPU. Second, we release Patent Mechanics Anticipate-0.6B, a 0.6B-parameter embedding model fine-tuned from harrier-oss-v1-0.6b with a contrastive loss on 8,192-token documents that include the description, on cleaned examiner labels and model-mined hard negatives, and averaged over three seeds. Third, we report what each choice buys, for prior-art search and invalidity search separately. On a held-out test half scored once, with the comparisons fixed in advance, the model reaches Recall@10 of 0.538 on a 250,367-family collection, against 0.451 for its base model and 0.326 for BM25 (+8.7 percentage points over the base, 95% CI +7.9 to +9.6); search-time passage scoring, classification agreement and lexical fusion raise this to 0.562. The gain transfers to PTAB invalidity queries that were never used in training. Model, benchmark, vectors, run files and evaluation code are released under permissive licences. Model: https://huggingface.co/patentmechanics/anticipate-0.6bBenchmark: https://huggingface.co/datasets/patentmechanics/prior-art-bench-v1Code: https://github.com/patentmechanics/prior-art-bench

Zenodo (CERN European Organization for Nuclear Research)
Information Retrieval and Search Behavior
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon