Seconds Against a Day: Certificate Transparency, Wildcard Certificates and the Discovery of New Hostnames by Scanners and AI Crawlers in a Six-Arm Honeypot

Every publicly trusted TLS certificate is recorded in the Certificate Transparency (CT) logs, which makes the logs a real-time feed of new hostnames. The usual advice for keeping a hostname out of the feed is a wildcard certificate, but that advice has not been tested with the served certificate held constant. We created 18 empty hostnames on three established domains in a two-by-three design: nine were named in their own certificate, nine were covered only by a wildcard, all 18 served the same wildcard certificate, and each listed 500 URLs in one of three places. Within 24 hours all 9 logged hostnames were requested by third parties (95% CI 70.1 to 100.0%) against 1 of 9 unlogged (2.0 to 43.5%; Fisher’s exact p = 0.00041). The first request reached a logged host 1.2 to 4.4 s after the log’s timestamp. The unlogged host was reached after 23.9 hours by one client; DNS analytics show the domains under constant subdomain guessing, so a wildcard certificate does not hide a dictionary label. ClaudeBot reached every logged host within 161 to 283 s and fetched all 500 URLs on each, finding an unannounced /sitemap.xml on its own. GPTBot arrived within 1,361 to 5,498 s and fetched all 500 only where the root page listed them as HTML links. Neither reached an unlogged host. An estimated 92.0% of first-hour scanner requests were for unlisted paths, mostly environment, credential and configuration files; one scanner sent 41.0% of all traffic. One client sent 14 AI-crawler names from a single cloud network, so we counted crawlers only from their operators’ networks. The pseudonymised request-level dataset, the honeypot source, the analysis plan and a script that recomputes every number are released under CC BY 4.0. Pre-print, not peer reviewed. Dataset and analysis script: 10.5281/zenodo.24203362. AI assistance: a large language model (Claude, Anthropic) wrote the honeypot and analysis code and drafted the manuscript under the author’s direction; the paper’s AI-assistance statement gives the details. The vendor of that model operates one of the crawlers measured.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-13
DOI
https://doi.org/10.5281/zenodo.24203363
Primary Topic
Web Application Security Vulnerabilities
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Seconds Against a Day: Certificate Transparency, Wildcard Certificates and the Discovery of New Hostnames by Scanners and AI Crawlers in a Six-Arm Honeypot

Guerin Lee Green
Zenodo (CERN European Organization for Nuclear Research)
Web Application Security Vulnerabilities
preprint

Seconds Against a Day: Certificate Transparency, Wildcard Certificates and the Discovery of New Hostnames by Scanners and AI Crawlers in a Six-Arm Honeypot

Guerin Lee Green
preprint en

Abstract

Every publicly trusted TLS certificate is recorded in the Certificate Transparency (CT) logs, which makes the logs a real-time feed of new hostnames. The usual advice for keeping a hostname out of the feed is a wildcard certificate, but that advice has not been tested with the served certificate held constant. We created 18 empty hostnames on three established domains in a two-by-three design: nine were named in their own certificate, nine were covered only by a wildcard, all 18 served the same wildcard certificate, and each listed 500 URLs in one of three places. Within 24 hours all 9 logged hostnames were requested by third parties (95% CI 70.1 to 100.0%) against 1 of 9 unlogged (2.0 to 43.5%; Fisher’s exact p = 0.00041). The first request reached a logged host 1.2 to 4.4 s after the log’s timestamp. The unlogged host was reached after 23.9 hours by one client; DNS analytics show the domains under constant subdomain guessing, so a wildcard certificate does not hide a dictionary label. ClaudeBot reached every logged host within 161 to 283 s and fetched all 500 URLs on each, finding an unannounced /sitemap.xml on its own. GPTBot arrived within 1,361 to 5,498 s and fetched all 500 only where the root page listed them as HTML links. Neither reached an unlogged host. An estimated 92.0% of first-hour scanner requests were for unlisted paths, mostly environment, credential and configuration files; one scanner sent 41.0% of all traffic. One client sent 14 AI-crawler names from a single cloud network, so we counted crawlers only from their operators’ networks. The pseudonymised request-level dataset, the honeypot source, the analysis plan and a script that recomputes every number are released under CC BY 4.0. Pre-print, not peer reviewed. Dataset and analysis script: 10.5281/zenodo.24203362. AI assistance: a large language model (Claude, Anthropic) wrote the honeypot and analysis code and drafted the manuscript under the author’s direction; the paper’s AI-assistance statement gives the details. The vendor of that model operates one of the crawlers measured.

Zenodo (CERN European Organization for Nuclear Research)
Web Application Security Vulnerabilities
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon