Representation vs. prior: do in-context foundation models remove feature engineering in few-shot dark-matter searches? A study on native CMS NanoAOD
Compressed top-squark signals, in which the stop decays to a nearly degenerate neutralino dark-matter candidate, leave soft, background-like events and are simulated at only a few mass points. Machine-learning searches therefore face two coupled problems: few labelled signal events, and a reliance on hand-engineered physics variables. We separate the two by comparing pretrained in-context foundation models used without any training (NVIDIA Kumo Tabular and Kumo Relational, and the separately trained TabICLv2 from the same TabICL family) with trained models (XGBoost, DeepSets, a Set Transformer, a parameterised neural network), on three representations of the same CMS Open Data NanoAOD events: 32 engineered variables, a naive flattening of the leading reconstructed objects, and the native relational structure. With 10 to 1000 signal events and five seeds per setting, Kumo Tabular on the naive flattening beats XGBoost on the engineered variables in 90 of 100 paired settings (sensitivity ratio 1.13, 95% CI 1.10–1.17) and reaches 96% of Kumo Tabular on the engineered variables; TabICLv2 shows the same pattern at a smaller margin. Reading the native tables with Kumo Relational beats trained models on the same raw objects but trails flat in-context learning by about 18%, a gap that survives matched model size, matched ensembling and explicit jet–object links. The soft-lepton table carries the signal. At unseen mass points every in-context tabular model transfers better than XGBoost and a parameterised network.
Authors
- Sabuhi Aghayev
Institutions
- Vilnius Gediminas Technical University (LT)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-21
- DOI
- https://doi.org/10.5281/zenodo.24418433
- Primary Topic
- Particle physics theoretical and experimental studies
- Type
- preprint