Faster Is Not Stronger: A Case Study of Search Semantics, Simulator Fidelity and Throughput in a Fixed-Team gen9ubers Pokémon Agent

A faster simulator lets a game-playing agent search more positions within a turn. Does that extra search improve its choices? We studied a fixed-team agent for the Pokémon Showdown gen9ubers ladder, where each player chooses a move without seeing the opponent's choice or full team configuration. A Rust battle engine ran about 220 times faster per simulated turn than the reference simulator. After three rounds of fixes on the same 3,000 ladder positions, it matched the reference on 99.82% of deterministic one-turn outcomes. Agreement on a separate 80-position check before the final fixes was 98.1%. In a benchmark with 384 games per arm, the engine's native best-response search won 6–7 percentage points fewer games than the reference search despite completing roughly 1,400 times as many iterations. The two algorithms count different work as an iteration. A second benchmark ported the reference search, including its learned model of human move choice, to the faster engine. At the original node cap, the estimated win-rate difference was +1.4 percentage points (95% CI −2.1 to +4.9). With the full time budget it was +3.8 points (95% CI +0.5 to +7.1 before adjustment and −0.2 to +7.9 after adjustment for three comparisons). This benchmark completed 77% of its planned games. Across 2,463 fully logged ladder games, the introduction of human move models coincided with a higher estimated rating. Four policy updates with documented offline screening gains showed no detectable ladder gain. The benchmarks changed several features together, including how often the search could override the policy. They leave open which change accounts for the difference in playing strength.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-19
DOI
https://doi.org/10.5281/zenodo.24374068
Primary Topic
Artificial Intelligence in Games
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Faster Is Not Stronger: A Case Study of Search Semantics, Simulator Fidelity and Throughput in a Fixed-Team gen9ubers Pokémon Agent

S M Abdulla Al Mamun
Zenodo (CERN European Organization for Nuclear Research)
Artificial Intelligence in Games
preprint

Faster Is Not Stronger: A Case Study of Search Semantics, Simulator Fidelity and Throughput in a Fixed-Team gen9ubers Pokémon Agent

S M Abdulla Al Mamun
preprint en

Abstract

A faster simulator lets a game-playing agent search more positions within a turn. Does that extra search improve its choices? We studied a fixed-team agent for the Pokémon Showdown gen9ubers ladder, where each player chooses a move without seeing the opponent's choice or full team configuration. A Rust battle engine ran about 220 times faster per simulated turn than the reference simulator. After three rounds of fixes on the same 3,000 ladder positions, it matched the reference on 99.82% of deterministic one-turn outcomes. Agreement on a separate 80-position check before the final fixes was 98.1%. In a benchmark with 384 games per arm, the engine's native best-response search won 6–7 percentage points fewer games than the reference search despite completing roughly 1,400 times as many iterations. The two algorithms count different work as an iteration. A second benchmark ported the reference search, including its learned model of human move choice, to the faster engine. At the original node cap, the estimated win-rate difference was +1.4 percentage points (95% CI −2.1 to +4.9). With the full time budget it was +3.8 points (95% CI +0.5 to +7.1 before adjustment and −0.2 to +7.9 after adjustment for three comparisons). This benchmark completed 77% of its planned games. Across 2,463 fully logged ladder games, the introduction of human move models coincided with a higher estimated rating. Four policy updates with documented offline screening gains showed no detectable ladder gain. The benchmarks changed several features together, including how often the search could override the policy. They leave open which change accounts for the difference in playing strength.

Zenodo (CERN European Organization for Nuclear Research)
University of Manchester (GB)
Artificial Intelligence in Games
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon