Representation vs. prior: do in-context foundation models remove feature engineering in few-shot dark-matter searches? A study on native CMS NanoAOD

Compressed top-squark signals, in which the stop decays to a nearly degenerate neutralino dark-matter candidate, leave soft, background-like events and are simulated at only a few mass points. Machine-learning searches therefore face two coupled problems: few labelled signal events, and a reliance on hand-engineered physics variables. We separate the two by comparing pretrained in-context foundation models used without any training (NVIDIA Kumo Tabular and Kumo Relational, and the separately trained TabICLv2 from the same TabICL family) with trained models (XGBoost, DeepSets, a Set Transformer, a parameterised neural network), on three representations of the same CMS Open Data NanoAOD events: 32 engineered variables, a naive flattening of the leading reconstructed objects, and the native relational structure. With 10 to 1000 signal events and five seeds per setting, Kumo Tabular on the naive flattening beats XGBoost on the engineered variables in 90 of 100 paired settings (sensitivity ratio 1.13, 95% CI 1.10–1.17) and reaches 96% of Kumo Tabular on the engineered variables; TabICLv2 shows the same pattern at a smaller margin. Reading the native tables with Kumo Relational beats trained models on the same raw objects but trails flat in-context learning by about 18%, a gap that survives matched model size, matched ensembling and explicit jet–object links. The soft-lepton table carries the signal. At unseen mass points every in-context tabular model transfers better than XGBoost and a parameterised network.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-21
DOI
https://doi.org/10.5281/zenodo.24418433
Primary Topic
Particle physics theoretical and experimental studies
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Representation vs. prior: do in-context foundation models remove feature engineering in few-shot dark-matter searches? A study on native CMS NanoAOD

Sabuhi Aghayev
Zenodo (CERN European Organization for Nuclear Research)
Particle physics theoretical and experimental studies
preprint

Representation vs. prior: do in-context foundation models remove feature engineering in few-shot dark-matter searches? A study on native CMS NanoAOD

Sabuhi Aghayev
preprint en

Abstract

Compressed top-squark signals, in which the stop decays to a nearly degenerate neutralino dark-matter candidate, leave soft, background-like events and are simulated at only a few mass points. Machine-learning searches therefore face two coupled problems: few labelled signal events, and a reliance on hand-engineered physics variables. We separate the two by comparing pretrained in-context foundation models used without any training (NVIDIA Kumo Tabular and Kumo Relational, and the separately trained TabICLv2 from the same TabICL family) with trained models (XGBoost, DeepSets, a Set Transformer, a parameterised neural network), on three representations of the same CMS Open Data NanoAOD events: 32 engineered variables, a naive flattening of the leading reconstructed objects, and the native relational structure. With 10 to 1000 signal events and five seeds per setting, Kumo Tabular on the naive flattening beats XGBoost on the engineered variables in 90 of 100 paired settings (sensitivity ratio 1.13, 95% CI 1.10–1.17) and reaches 96% of Kumo Tabular on the engineered variables; TabICLv2 shows the same pattern at a smaller margin. Reading the native tables with Kumo Relational beats trained models on the same raw objects but trails flat in-context learning by about 18%, a gap that survives matched model size, matched ensembling and explicit jet–object links. The soft-lepton table carries the signal. At unseen mass points every in-context tabular model transfers better than XGBoost and a parameterised network.

Zenodo (CERN European Organization for Nuclear Research)
Vilnius Gediminas Technical University (LT)
Particle physics theoretical and experimental studies
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon