Connectome-constrained models of fly motion vision predict genetic silencing no better than their wiring diagram: a preregistered test

Connectome-constrained network models are judged mainly by how well they reproduce unperturbed neural activity, yet their value as mechanistic models rests on predicting what happens when parts of the circuit are removed. We tested whether the 50 task-trained, connectome-constrained models of the Drosophila visual system of Lappalainen et al. (2024) predict published results of genetic silencing. We removed the synaptic output of the silenced cell types in each model and compared predicted changes in motion responses with outcomes extracted from the literature by two blinded AI coders, under a protocol, mapping and analysis fixed before any outcome was coded. Of 192 experiment-level results from 23 papers, the coders placed 124 in the same effect category. The model ensemble agreed with published outcomes better than chance (weighted kappa 0.49 over 120 full-text results; permutation p < 0.001) but not better than a baseline computed from synapse counts alone (kappa 0.56). The preregistered paired agreement difference favoured the wiring baseline across all modelled readouts (-0.039; 97.5% CI -0.052 to -0.021; 17 papers) and did not favour the models for T4/T5 recordings (-0.067; -0.083 to 0.000; 25 results, 4 papers). Training raised agreement from chance (untrained network kappa 0.04) but not beyond anatomy. The models most often predicted no change where silencing reduced responses. Neither a model's fit to measured direction tuning nor its task performance predicted its silencing agreement (Spearman rho 0.16 and -0.02). In five models, including the best-performing one, the lamina neuron L1 never crossed its synaptic threshold at any light level, so silencing it had no effect. Reproducing activity did not ensure reproducing the effects of intervention; perturbation benchmarks with anatomical baselines should be part of how such models are validated. Paper 11 by Thomas Ryan. Preprint; not peer reviewed. The complete PDF includes the manuscript and supplement. The accompanying package contains code, predictions, outcome coding, protocol history, audits and reproducibility records. The protocol was frozen and hashed locally before outcome coding; this is an audit record, not third-party preregistration. AI assistance is disclosed in the manuscript; Thomas Ryan is the sole author.