SignSense-Agent: A Condition-Aware AI Agent and a Reproducible Robustness Benchmark for Traffic-Sign OCR Under Adverse Weather, Glare and Adversarial Stickers

Traffic-sign recognition and sign-text OCR that are near-perfect on clean images can fail abruptly in fog, glare, darkness or when a sign carries stickers, and the failure is often silent: the system is wrong and confident. This paper makes two contributions toward measuring and reducing that risk. First, we release a reproducible robustness toolkit that catalogs 15 public traffic-sign and driving-condition datasets, converts them to one manifest format, audits data quality, applies 14 physically motivated corruptions at five severities, runs five attack families (random stickers, a universal printable patch, RP2-style stickers with expectation over transformation, FGSM and PGD), and reports degradation slope, breakdown severity, mean corruption error, calibration and silent-failure rate. Second, we propose SignSense-Agent, a deterministic perceive–assess–plan–act–verify–learn loop placed after a probabilistic region-ofinterest OCR pipeline. It estimates the image condition, selects one of nine OCR strategies, cross-checks classifier and OCR outputs, fuses evidence over a track and abstains or raises an alarm rather than commit a doubtful reading. An offline validation agent ranks weak conditions and proposes policy changes for human approval. On U.S. signs rendered from publicdomain MUTCD artwork, a CNN with 99.8% clean accuracy keeps only 75.7% mean accuracy across corruptions; fog breaks it at severity 2 (57%) and headlight glare at severity 1 (68%). In controlled experiments on synthetic signs, corruption-augmented PGD training cuts mean corruption error to 61.1 (reference 100) and raises PGD accuracy at ϵ = 8/255 from 30% to 80%; temporal log-fusion raises final-decision accuracy from 63% to 82% under transient glare; and the agent halves silent OCR failures (1.5%→0.7%) while raising clean exact match from 79% to 96%, but does not yet raise correct reads under adverse conditions. We report these limits explicitly and define the real-world validation protocol (CURE-TSR, LISA, BDD100KMUTCD) that the toolkit is built to run.

Authors

Publication Details

Journal
Preprints.org
Published
2026-11-18
DOI
https://doi.org/10.20944/preprints202611.1778.v1
Primary Topic
Adversarial Robustness in Machine Learning
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

SignSense-Agent: A Condition-Aware AI Agent and a Reproducible Robustness Benchmark for Traffic-Sign OCR Under Adverse Weather, Glare and Adversarial Stickers

Subramaniam Ganesan, Ajay A. Waghmare
Preprints.org
Adversarial Robustness in Machine Learning
preprint

SignSense-Agent: A Condition-Aware AI Agent and a Reproducible Robustness Benchmark for Traffic-Sign OCR Under Adverse Weather, Glare and Adversarial Stickers

Subramaniam Ganesan, Ajay A. Waghmare
preprint en

Abstract

Traffic-sign recognition and sign-text OCR that are near-perfect on clean images can fail abruptly in fog, glare, darkness or when a sign carries stickers, and the failure is often silent: the system is wrong and confident. This paper makes two contributions toward measuring and reducing that risk. First, we release a reproducible robustness toolkit that catalogs 15 public traffic-sign and driving-condition datasets, converts them to one manifest format, audits data quality, applies 14 physically motivated corruptions at five severities, runs five attack families (random stickers, a universal printable patch, RP2-style stickers with expectation over transformation, FGSM and PGD), and reports degradation slope, breakdown severity, mean corruption error, calibration and silent-failure rate. Second, we propose SignSense-Agent, a deterministic perceive–assess–plan–act–verify–learn loop placed after a probabilistic region-ofinterest OCR pipeline. It estimates the image condition, selects one of nine OCR strategies, cross-checks classifier and OCR outputs, fuses evidence over a track and abstains or raises an alarm rather than commit a doubtful reading. An offline validation agent ranks weak conditions and proposes policy changes for human approval. On U.S. signs rendered from publicdomain MUTCD artwork, a CNN with 99.8% clean accuracy keeps only 75.7% mean accuracy across corruptions; fog breaks it at severity 2 (57%) and headlight glare at severity 1 (68%). In controlled experiments on synthetic signs, corruption-augmented PGD training cuts mean corruption error to 61.1 (reference 100) and raises PGD accuracy at ϵ = 8/255 from 30% to 80%; temporal log-fusion raises final-decision accuracy from 63% to 82% under transient glare; and the agent halves silent OCR failures (1.5%→0.7%) while raising clean exact match from 79% to 96%, but does not yet raise correct reads under adverse conditions. We report these limits explicitly and define the real-world validation protocol (CURE-TSR, LISA, BDD100KMUTCD) that the toolkit is built to run.

Preprints.org
Adversarial Robustness in Machine Learning
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon