Class-conditional conformal coverage of a cellfinder classifier fails to transfer from serial two-photon training data to public light-sheet datasets
Whole-brain cell-detection pipelines such as cellfinder classify each candidate cell with a network that is often trained on another laboratory's data. Conformal prediction could flag unreliable candidates, but its coverage guarantee assumes that new data are exchangeable with the calibration data. We asked whether class-conditional conformal sets, calibrated on serial two-photon training data to contain the true label for 90% of cells and 90% of non-cells, keep that coverage on two public light-sheet datasets. Six pre-registered studies found that they did not. On CellSeg3D the classifier itself failed. On DELiVR, a classifier trained on per-cube standardised inputs was 72.9% accurate on 2,988 test candidates, yet its sets covered 52.5% of non-cells and 34.8% of cells. The failure held under two label rules, in each of four test patch groups and, in a smaller earlier test, after intensity matching. Class-conditional coverage is unaffected by changes in class proportions, so this failure means that scores shifted within each class. Controlled perturbations of a held-out source brain showed that blur, downsampling, noise and object size cause such shifts in the standardised model; noise cut cell coverage by 27.5 percentage points while accuracy fell by 1.5. No tested remedy restored reliable coverage on held-out target groups. A label-shift correction cannot repair within-class shift, and covariate-shift weighting failed. With 160 target labels from two other patch groups, recalibrated sets covered both classes at 85% or more in 56.5% of resampled label sets, against 86.0% expected without shift. Coverage should be checked on labelled target data from each acquisition group.
Authors
- S. M. Abdulla Al Mamun
Institutions
- University of Manchester (GB)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-19
- DOI
- https://doi.org/10.5281/zenodo.24369430
- Primary Topic
- Cell Image Analysis Techniques
- Type
- preprint