Kalman-Based Temporal Difference Neural Network (KBTDNN): A Controlled Reassessment of Belief Input, Kalman Gain and Training Loop

In this controlled empirical reassessment and ablation of KBTDNN, our original Kalman-based temporal difference neural network, a light-dark task with position-dependent sensor accuracy is introduced to analyze the effect of covariance under conditions in which uncertainty becomes state- and trajectory-dependent, unlike in the original tasks. Policy input, Kalman gain, and the training loop are varied separately under pre-specified hypotheses H1–H3. The re-implementation reaches 0.80 and 0.75 of the Bayes-optimal tiger and maze ceilings. The shortfalls come from the belief reward’s single-Gaussian posterior (+1.34 to +1.40) and the training loop (+0.075 to +0.078 per step), respectively, not the filter state. The covariance is a clock on the tiger (H1 refuted), not confirmed in either direction on the maze and cart-pole. The pre-specified light-dark instance left every pre-specified learner’s interquartile mean at the never-stop floor (H1 refuted as scored; the test uninformative). On a post hoc noisy-action instance, the policy using the belief mean and covariance outperformed the mean-only policy on ten seeds (26.75 versus 6.05 to 7.72) and nine of ten fresh ones: an exploratory existence result. After a cart-pole sensor-noise switch, online innovation-adaptive and offline-pre-trained gains reduced estimation error without a supported return improvement (H2 refuted as scored), and fine-tuned gains diverged.

Authors

Institutions

Publication Details

Journal
Applied Sciences
Published
2026-11-19
DOI
https://doi.org/10.3390/app162211493
Primary Topic
Reinforcement Learning in Robotics
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Kalman-Based Temporal Difference Neural Network (KBTDNN): A Controlled Reassessment of Belief Input, Kalman Gain and Training Loop

Alp Sardag, Abdullah Sardağ
Applied Sciences
Reinforcement Learning in Robotics
article

Kalman-Based Temporal Difference Neural Network (KBTDNN): A Controlled Reassessment of Belief Input, Kalman Gain and Training Loop

Alp Sardag, Abdullah Sardağ
article en

Abstract

In this controlled empirical reassessment and ablation of KBTDNN, our original Kalman-based temporal difference neural network, a light-dark task with position-dependent sensor accuracy is introduced to analyze the effect of covariance under conditions in which uncertainty becomes state- and trajectory-dependent, unlike in the original tasks. Policy input, Kalman gain, and the training loop are varied separately under pre-specified hypotheses H1–H3. The re-implementation reaches 0.80 and 0.75 of the Bayes-optimal tiger and maze ceilings. The shortfalls come from the belief reward’s single-Gaussian posterior (+1.34 to +1.40) and the training loop (+0.075 to +0.078 per step), respectively, not the filter state. The covariance is a clock on the tiger (H1 refuted), not confirmed in either direction on the maze and cart-pole. The pre-specified light-dark instance left every pre-specified learner’s interquartile mean at the never-stop floor (H1 refuted as scored; the test uninformative). On a post hoc noisy-action instance, the policy using the belief mean and covariance outperformed the mean-only policy on ten seeds (26.75 versus 6.05 to 7.72) and nine of ten fresh ones: an exploratory existence result. After a cart-pole sensor-noise switch, online innovation-adaptive and offline-pre-trained gains reduced estimation error without a supported return improvement (H2 refuted as scored), and fine-tuned gains diverged.

Applied SciencesVol. 16(22)
Istanbul Topkapi University (TR)
Reinforcement Learning in Robotics
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon