Did the Agent Read It? ReadProof: Access-Aware Reading, Channel-Located Evidence and Verified Quotes for Agentic Search over Social Media

Agentic search systems now approach the ceiling of open-web text benchmarks, yet on social media they often fail before any reasoning starts: the link to a public post returns a verification page, a login wall or an expired share token, and when the post is served, the answer often sits in speech, burned-in text, carousel images or comments that a text pipeline never reads. Benchmarks that supply a fixed corpus, pre-extracted media or the video itself cannot observe these failures. We present ReadProof, an open-source harness and benchmark that make the read itself measurable. The harness enforces its access rules in code, each pinned by a test: a declared identity, no cookies, no captcha solving or signature generation, and no change of identity or address after a refusal. It evaluates robots.txt and Content-Signal on every request under declared access profiles, returns a typed wall instead of page text, reads media through a platform's official player, stores evidence as content-addressed records located by channel, and accepts a citation only when its quote is an exact substring of a stored record. ReadProof-Bench measures access across the link forms people actually paste (145 public posts on nine platforms, 522 link forms) and, with a planted-canary panel of the author's own posts carrying one code per channel, which channels a system actually reads. A declared reader that follows these rules served 81.8% of link forms, against 66.5% for a headless browser, 64.9% for a reader service and 25.4% for a plain fetch, counting machine-translated pages of the post as served; no reader won on every platform. Obeying robots.txt strictly cost 26.7 points of coverage, which we call the honesty penalty, and left the declared reader below the headless browser. With the backbone we used, no system answered from a wall: given a wall page, the model declined or said what the page was. On the canaries ReadProof returned the most codes (9 of 28 items, against at most 5), comment codes were read only on X, and five commercial assistants found three codes in 27 answers, two of them on the platform of their own company. No system invented a code.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-21
DOI
https://doi.org/10.5281/zenodo.24422487
Primary Topic
Web Data Mining and Analysis
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Did the Agent Read It? ReadProof: Access-Aware Reading, Channel-Located Evidence and Verified Quotes for Agentic Search over Social Media

Loong Nian Chew
Zenodo (CERN European Organization for Nuclear Research)
Web Data Mining and Analysis
preprint

Did the Agent Read It? ReadProof: Access-Aware Reading, Channel-Located Evidence and Verified Quotes for Agentic Search over Social Media

Loong Nian Chew
preprint en

Abstract

Agentic search systems now approach the ceiling of open-web text benchmarks, yet on social media they often fail before any reasoning starts: the link to a public post returns a verification page, a login wall or an expired share token, and when the post is served, the answer often sits in speech, burned-in text, carousel images or comments that a text pipeline never reads. Benchmarks that supply a fixed corpus, pre-extracted media or the video itself cannot observe these failures. We present ReadProof, an open-source harness and benchmark that make the read itself measurable. The harness enforces its access rules in code, each pinned by a test: a declared identity, no cookies, no captcha solving or signature generation, and no change of identity or address after a refusal. It evaluates robots.txt and Content-Signal on every request under declared access profiles, returns a typed wall instead of page text, reads media through a platform's official player, stores evidence as content-addressed records located by channel, and accepts a citation only when its quote is an exact substring of a stored record. ReadProof-Bench measures access across the link forms people actually paste (145 public posts on nine platforms, 522 link forms) and, with a planted-canary panel of the author's own posts carrying one code per channel, which channels a system actually reads. A declared reader that follows these rules served 81.8% of link forms, against 66.5% for a headless browser, 64.9% for a reader service and 25.4% for a plain fetch, counting machine-translated pages of the post as served; no reader won on every platform. Obeying robots.txt strictly cost 26.7 points of coverage, which we call the honesty penalty, and left the declared reader below the headless browser. With the backbone we used, no system answered from a wall: given a wall page, the model declined or said what the page was. On the canaries ReadProof returned the most codes (9 of 28 items, against at most 5), comment codes were read only on X, and five commercial assistants found three codes in 27 answers, two of them on the platform of their own company. No system invented a code.

Zenodo (CERN European Organization for Nuclear Research)
Web Data Mining and Analysis
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon
Did the Agent Read It? ReadProof: Access-Aware Reading, Channel-Located Evidence and Verified Quotes for Agentic Search over Social Media — Loong Nian Chew · Zenodo (CERN European Organization for Nuclear Research) (2026) | TGRS Research Map | TGRS