RL + Transformer = A General-Purpose Problem Solver

What if artificial intelligence could not only solve problems for which it was trained but also teach itself to tackle novel tasks?In this paper, we finetune Llama 3.1 using reinforcement learning on the grid-world game Frozen Lake and investigate its ability to solve maps it has never encountered-a phenomenon recently termed In-Context Reinforcement Learning (ICRL).Without additional training, the transformer demonstrates the capacity to adapt to both in-distribution and outof-distribution environment parameterizations.Moreover, it remains effective when trained on data that blends optimal and suboptimal behavior, combines strategies from its context (behavior-stitching), and dynamically adapts to non-stationary environments.These proof-ofconcept findings suggest that in-context learning via reinforcement-tuned transformers may form the basis of a promising general-purpose problem-solver.