Fast ResNet-DQN trajectory planning algorithm for dynamic grid environments

Under a fully observable 40 × 40 occupancy-grid model, dynamic obstacles are represented as time-varying occupied cells that are refreshed before each decision. The agent then acts in a four-neighbour discrete action space. This paper proposes a lightweight residual deep Q-network (ResNet-DQN) for fast trajectory planning in this setting. Two-dimensional normalized coordinates and a six-channel compact map representation are used as state inputs. The training protocol combines planner-labelled behavioural pretraining and decaying training-only guidance with Double DQN targets, validity-gated action selection, reward shaping, Huber loss, gradient clipping and hybrid soft-hard target-network updates. Under a matched five-seed protocol using seeds 11, 37, 71, 101 and 102, ResNet-DQN achieved a mean success rate of 94.8% and a complete decision latency of 0.912 ms on an NVIDIA GeForce RTX 4090 D. The present model is limited to four-direction discrete motion and does not represent continuous robot kinematics or predict obstacle trajectories.

Authors

Institutions

Publication Details

Journal
Discover Applied Sciences
Published
2026-11-17
DOI
https://doi.org/10.1007/s42452-026-09903-4
Primary Topic
Reinforcement Learning in Robotics
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Fast ResNet-DQN trajectory planning algorithm for dynamic grid environments

Fengkai Gan
3 links
Discover Applied Sciences
Reinforcement Learning in Robotics
article

Fast ResNet-DQN trajectory planning algorithm for dynamic grid environments

Fengkai Gan
article en

Abstract

Under a fully observable 40 × 40 occupancy-grid model, dynamic obstacles are represented as time-varying occupied cells that are refreshed before each decision. The agent then acts in a four-neighbour discrete action space. This paper proposes a lightweight residual deep Q-network (ResNet-DQN) for fast trajectory planning in this setting. Two-dimensional normalized coordinates and a six-channel compact map representation are used as state inputs. The training protocol combines planner-labelled behavioural pretraining and decaying training-only guidance with Double DQN targets, validity-gated action selection, reward shaping, Huber loss, gradient clipping and hybrid soft-hard target-network updates. Under a matched five-seed protocol using seeds 11, 37, 71, 101 and 102, ResNet-DQN achieved a mean success rate of 94.8% and a complete decision latency of 0.912 ms on an NVIDIA GeForce RTX 4090 D. The present model is limited to four-direction discrete motion and does not represent continuous robot kinematics or predict obstacle trajectories.

Discover Applied Sciences
South China University of Technology (CN)
Reinforcement Learning in Robotics
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon