Utvärdering av testtidsinlärning för konceptuellt resonerande i stora språkmodeller

Large Language Models (LLMs) excel at pattern recognition but still struggle with abstract reasoning, that is, inferring high-level rules from few examples. The Abstraction and Reasoning Corpus (ARC) benchmark exposes this weakness by forcing models to deduce grid transformations from only a handful of demonstrations. Many high-performing ARC solvers use test-time training (TTT): they add Low-Rank Adaptation (LoRA) layers and update them on the labelled grids seen during evaluation while the backbone stays frozen. We investigate how TTT carries over to ConceptARC, a collection of ARC-inspired tasks divided into sixteen interpretable concept groups that enables a direct comparison between individual (per-concept) and aggregated (entire ConceptARC) reasoning. Without TTT, an ARC-pre-trained solver attains 9.33% aggregated accuracy and 9.66% mean per-concept accuracy with a 3 billion-parameter Llama-3.2 model, and 12.1% aggregated and 12.0% per-concept accuracy with an 8 billion-parameter NeMo-Minitron model. A single TTT epoch lifts these scores to 40.8% aggregated and 11.6% per-concept for Llama, and to 50.4% aggregated and 25.6% per-concept for NeMo. These results show that a data-efficient TTT step can more than quadruple LLM performance on aggregated conceptual reasoning while at least doubling performance on individual concept groups.