A constant output scores above two trained systems under raw Pearson correlation
Note: the quantity N* is the mean resultant length of the unit-normalized references, a standard object in directional statistics (Fisher, 1993); the contribution here is its use as a pre-flight diagnostic, not the identity. On a spiking sensorimotor benchmark, a constant output — the same trajectoryfor every input — scores 0.944 under raw Pearson correlation, above the twotrained systems (0.865 and 0.873; both below the null at p < 0.001 over tenseeds). The result replicates on a second task (k = 4, a new shape and a newtarget, retrained): a constant output scores 0.937, above both systems (0.746and 0.865). The ordering holds across thirteen configurations — reservoir size200–2000, three readouts (STDP, ridge, FORCE RLS), `k = 3` and `k = 5`, 10 and20 seeds — plus the three `k = 2` pairs and the separate `k = 4` task; andacross four of five metric families; only centering, which removes the sharedcomponent, reverses it. The best constant output depends only on the referenceset. For Pearson, the optimum is ```v* = sum_i (t_i - mean(t_i)) / ||t_i - mean(t_i)||N* = (1/k) * sum_i corr(v*, t_i) = ||v*|| / k``` so the check needs no training and no model. A system scoring below `N*` isindistinguishable from one that ignores its input. The verdict is**metric-dependent**: under centered Pearson the same outputs give a null scoreof 0.068 and both trained systems clear it (0.101 and 0.360), so the failurebelongs to the metric, not to the systems. A target-specific distributionalmodel for rank-1 (one mean per row type, shared covariance) predicts observedrank-1 within four points; under leave-one-seed-out it gives Brier 0.184 vs0.200 for STDP (modest) and 0.000 vs 0.333 for ridge (every seed shares thesame per-target ordering).Shape information is partially retained (100% in the reservoir, 84% at thereadout, 67%–87% at the motor output) but is not routed to the correct target.We relate the result to null-model cheating in LLM-judge benchmarks, to hubnessin high-dimensional retrieval, and to proxy-optimization accounts of rewardhacking, and argue that the single-reference similarity case is the one wherethe boundary is exact and pre-computable.
Authors
- haimeng Pan
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-13
- DOI
- https://doi.org/10.5281/zenodo.24194408
- Primary Topic
- Neural Networks and Reservoir Computing
- Type
- article
- Field-Weighted Citation Impact
- 0.00