Coverage Failure as a Diagnostic Signal in Neural Model Probing

We report a methodological observation from a multi-machine dense-grid activation probing study of nineteen small to mid-sized language models (124M to 7.6B parameters). Every experiment satisfied a nominal threshold of one hundred million observation slots, yet observed fill rate varied from 58.75 percent to 97.05 percent across experiments. The observed gap is strongly associated with protocol choices, in particular the prompt corpus size and the use of per-channel quantile binning. We cannot yet rule out interaction with the network under study, the prompt set, hook discovery completeness, or quantile edge quality; the comparisons we report are observational rather than controlled. We argue that total slot count alone is an invalid completion criterion for activation probing, that coverage percentage should be reported as a first-class result, and that under-coverage should be treated as a diagnostic signal about the probe protocol before being attributed to the model.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-05-29
DOI
https://doi.org/10.5281/zenodo.20448668
Primary Topic
Neurobiology of Language and Bilingualism
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Coverage Failure as a Diagnostic Signal in Neural Model Probing

Ho Yiing Chen
Zenodo (CERN European Organization for Nuclear Research)
Neurobiology of Language and Bilingualism
preprint

Coverage Failure as a Diagnostic Signal in Neural Model Probing

Ho Yiing Chen
preprint en

Abstract

We report a methodological observation from a multi-machine dense-grid activation probing study of nineteen small to mid-sized language models (124M to 7.6B parameters). Every experiment satisfied a nominal threshold of one hundred million observation slots, yet observed fill rate varied from 58.75 percent to 97.05 percent across experiments. The observed gap is strongly associated with protocol choices, in particular the prompt corpus size and the use of per-channel quantile binning. We cannot yet rule out interaction with the network under study, the prompt set, hook discovery completeness, or quantile edge quality; the comparisons we report are observational rather than controlled. We argue that total slot count alone is an invalid completion criterion for activation probing, that coverage percentage should be reported as a first-class result, and that under-coverage should be treated as a diagnostic signal about the probe protocol before being attributed to the model.

Zenodo (CERN European Organization for Nuclear Research)
Neurobiology of Language and Bilingualism
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.