Energy-Based Open-Set Recognition for Video Surveillance: A Shared-Checkpoint Comparison of Four Detection Layers

Convolutional classifiers operate under the closed-set assumption: every input is taken to belong to a category seen during training. Deployed surveillance systems violate that assumption routinely, and a classifier confronted with an unfamiliar object assigns it to the nearest known category with high confidence, giving no indication that the classification is unwarranted. This paper evaluates an energy-based open-set detection layer built on a ResNet-50 backbone against three baselines, namely a conventional closed-set classifier, a softmax confidence threshold and OpenMax. The contribution is a comparison protocol in which the four detectors share a single trained checkpoint, a single calibration procedure restricted to the validation split, and a single decision rule, so that any measured difference is attributable to the detection layer alone. Twenty-four evaluation cells were produced from three training runs, spanning two regimes of unknown samples and three random seeds. With unknown samples drawn from held-out CIFAR-10 categories, OpenMax reached an AUROC of 0.845 against 0.815 for the energy detector, a significant difference under a paired test; the energy detector nevertheless reduced the false positive rate at 95 percent true positive rate by 0.062 relative to softmax thresholding while remaining indistinguishable from it on AUROC. With unknown samples drawn from TinyImageNet, all methods improved and no difference between the energy detector and OpenMax reached significance, the energy detector showing a dispersion across seeds four to five times larger. The largest effect measured was neither of these: changing the origin of the unknown samples raised the AUROC of the energy detector by 0.109 and lowered its false positive rate by 0.305, more than three times the span between the strongest and weakest detectors within either regime. Both regimes are built from public object-recognition corpora rather than from surveillance footage, so the results characterise the detection layers themselves; their transfer to operational surveillance imagery remains to be established. Keywords— energy-based detection: open-set recognition: out-of-distribution detection: reproducibility: ResNet-50: video surveillance