AI Agent Readiness Across 100,000 Websites
This study evaluates web readiness for autonomous AI agents across the top 100,000 domains in the Tranco global ranking using 120 deterministic technical indicators across six operational layers. While direct requests retrieve readable HTML on nearly half of all websites (48.66%), dedicated machine interfaces such as llms.txt (7.16%), OpenAPI schemas (0.23%), and Model Context Protocol servers (0.05%) remain rare. Domains satisfy an average of 21.33% of evaluated criteria when counting verified capabilities, reflecting substantial operational uncertainty caused by live-web network drops, timeouts, and bot defenses. These empirical observations establish a baseline for web agent infrastructure, indicating that autonomous systems must continue relying on browser rendering, DOM heuristics, and fallback scraping until structured machine contracts achieve widespread adoption.
Authors
- Vedang Ratan Vatsa (ORCID: https://orcid.org/0000-0003-1269-0486)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-10
- DOI
- https://doi.org/10.5281/zenodo.24121070
- Primary Topic
- Web Data Mining and Analysis
- Type
- article
- Field-Weighted Citation Impact
- 0.00