From Human Hands to Host Advantage? Demonstration-Seeded vs. Vanilla DQN in ViZDoom Deathmatches

Training autonomous first-person shooter (FPS) agents from raw pixels is sample-inefficient, and small-scale multiplayer evaluations are often confounded by network roles. This paper presents a controlled case study of a deep Q-network (DQN) agent whose replay buffer is seeded with human demonstrations in ViZDoom. Agent A receives 629 human transitions; Agent B is trained with vanilla DQN. Both agents are evaluated against a random-action baseline and against each other in a custom 1v1 deathmatch across five repeated batches, with host and joiner roles swapped. After recomputing outcomes from perepisode frag counts, Agent A achieved a 68.0% win rate against the random baseline (95% CI 54.2-79.2; McNemar p = 0.006), whereas Agent B achieved 52.0% (38.5-65.2; p = 0.22) and did not significantly outperform the baseline. Contrary to the original expectation, demonstration seeding did not accelerate convergence: the vanilla agent first reached positive test scores at epoch 3, versus epoch 5 for the seeded agent, and the final plateau was nearly identical (last-10 test means 72.6 vs. 72.7). In head-to-head play, hosts won 60.0% of duels versus 19.0% for joiners, with 21.0% ties (McNemar p < 0.001), but the effect was heterogeneous: the B-host condition showed a large advantage, while the A-host condition did not. The results highlight action-space encoding, per-episode logging, and counterbalanced host/joiner designs as critical for reproducible FPS benchmarks.

Authors

Institutions

Publication Details

Journal
International Journal for Research in Applied Science and Engineering Technology
Published
2026-11-16
DOI
https://doi.org/10.22214/ijraset.2026.85703
Primary Topic
Reinforcement Learning in Robotics
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

From Human Hands to Host Advantage? Demonstration-Seeded vs. Vanilla DQN in ViZDoom Deathmatches

Taufik Hidayat, Louis Bratawidjaja, Calvanny Lambue Rezananda
International Journal for Research in Applied Science and Engineering Technology
Reinforcement Learning in Robotics
article

From Human Hands to Host Advantage? Demonstration-Seeded vs. Vanilla DQN in ViZDoom Deathmatches

Taufik Hidayat, Louis Bratawidjaja, Calvanny Lambue Rezananda
article en

Abstract

Training autonomous first-person shooter (FPS) agents from raw pixels is sample-inefficient, and small-scale multiplayer evaluations are often confounded by network roles. This paper presents a controlled case study of a deep Q-network (DQN) agent whose replay buffer is seeded with human demonstrations in ViZDoom. Agent A receives 629 human transitions; Agent B is trained with vanilla DQN. Both agents are evaluated against a random-action baseline and against each other in a custom 1v1 deathmatch across five repeated batches, with host and joiner roles swapped. After recomputing outcomes from perepisode frag counts, Agent A achieved a 68.0% win rate against the random baseline (95% CI 54.2-79.2; McNemar p = 0.006), whereas Agent B achieved 52.0% (38.5-65.2; p = 0.22) and did not significantly outperform the baseline. Contrary to the original expectation, demonstration seeding did not accelerate convergence: the vanilla agent first reached positive test scores at epoch 3, versus epoch 5 for the seeded agent, and the final plateau was nearly identical (last-10 test means 72.6 vs. 72.7). In head-to-head play, hosts won 60.0% of duels versus 19.0% for joiners, with 21.0% ties (McNemar p < 0.001), but the effect was heterogeneous: the B-host condition showed a large advantage, while the A-host condition did not. The results highlight action-space encoding, per-episode logging, and counterbalanced host/joiner designs as critical for reproducible FPS benchmarks.

International Journal for Research in Applied Science and Engineering TechnologyVol. 14(11)
Nanjing University of Information Science and Technology (CN)
Reinforcement Learning in Robotics
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon