A constant output scores above two trained systems under raw Pearson correlation

Note: the quantity N* is the mean resultant length of the unit-normalized references, a standard object in directional statistics (Fisher, 1993); the contribution here is its use as a pre-flight diagnostic, not the identity. On a spiking sensorimotor benchmark, a constant output — the same trajectoryfor every input — scores 0.944 under raw Pearson correlation, above the twotrained systems (0.865 and 0.873; both below the null at p < 0.001 over tenseeds). The result replicates on a second task (k = 4, a new shape and a newtarget, retrained): a constant output scores 0.937, above both systems (0.746and 0.865). The ordering holds across thirteen configurations — reservoir size200–2000, three readouts (STDP, ridge, FORCE RLS), `k = 3` and `k = 5`, 10 and20 seeds — plus the three `k = 2` pairs and the separate `k = 4` task; andacross four of five metric families; only centering, which removes the sharedcomponent, reverses it. The best constant output depends only on the referenceset. For Pearson, the optimum is ```v* = sum_i (t_i - mean(t_i)) / ||t_i - mean(t_i)||N* = (1/k) * sum_i corr(v*, t_i) = ||v*|| / k``` so the check needs no training and no model. A system scoring below `N*` isindistinguishable from one that ignores its input. The verdict is**metric-dependent**: under centered Pearson the same outputs give a null scoreof 0.068 and both trained systems clear it (0.101 and 0.360), so the failurebelongs to the metric, not to the systems. A target-specific distributionalmodel for rank-1 (one mean per row type, shared covariance) predicts observedrank-1 within four points; under leave-one-seed-out it gives Brier 0.184 vs0.200 for STDP (modest) and 0.000 vs 0.333 for ridge (every seed shares thesame per-target ordering).Shape information is partially retained (100% in the reservoir, 84% at thereadout, 67%–87% at the motor output) but is not routed to the correct target.We relate the result to null-model cheating in LLM-judge benchmarks, to hubnessin high-dimensional retrieval, and to proxy-optimization accounts of rewardhacking, and argue that the single-reference similarity case is the one wherethe boundary is exact and pre-computable.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-13
DOI
https://doi.org/10.5281/zenodo.24194408
Primary Topic
Neural Networks and Reservoir Computing
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

A constant output scores above two trained systems under raw Pearson correlation

haimeng Pan
Zenodo (CERN European Organization for Nuclear Research)
Neural Networks and Reservoir Computing
article

A constant output scores above two trained systems under raw Pearson correlation

haimeng Pan
article en

Abstract

Note: the quantity N* is the mean resultant length of the unit-normalized references, a standard object in directional statistics (Fisher, 1993); the contribution here is its use as a pre-flight diagnostic, not the identity. On a spiking sensorimotor benchmark, a constant output — the same trajectoryfor every input — scores 0.944 under raw Pearson correlation, above the twotrained systems (0.865 and 0.873; both below the null at p < 0.001 over tenseeds). The result replicates on a second task (k = 4, a new shape and a newtarget, retrained): a constant output scores 0.937, above both systems (0.746and 0.865). The ordering holds across thirteen configurations — reservoir size200–2000, three readouts (STDP, ridge, FORCE RLS), `k = 3` and `k = 5`, 10 and20 seeds — plus the three `k = 2` pairs and the separate `k = 4` task; andacross four of five metric families; only centering, which removes the sharedcomponent, reverses it. The best constant output depends only on the referenceset. For Pearson, the optimum is ```v* = sum_i (t_i - mean(t_i)) / ||t_i - mean(t_i)||N* = (1/k) * sum_i corr(v*, t_i) = ||v*|| / k``` so the check needs no training and no model. A system scoring below `N*` isindistinguishable from one that ignores its input. The verdict is**metric-dependent**: under centered Pearson the same outputs give a null scoreof 0.068 and both trained systems clear it (0.101 and 0.360), so the failurebelongs to the metric, not to the systems. A target-specific distributionalmodel for rank-1 (one mean per row type, shared covariance) predicts observedrank-1 within four points; under leave-one-seed-out it gives Brier 0.184 vs0.200 for STDP (modest) and 0.000 vs 0.333 for ridge (every seed shares thesame per-target ordering).Shape information is partially retained (100% in the reservoir, 84% at thereadout, 67%–87% at the motor output) but is not routed to the correct target.We relate the result to null-model cheating in LLM-judge benchmarks, to hubnessin high-dimensional retrieval, and to proxy-optimization accounts of rewardhacking, and argue that the single-reference similarity case is the one wherethe boundary is exact and pre-computable.

Zenodo (CERN European Organization for Nuclear Research)
Neural Networks and Reservoir Computing
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon