Kalman-Based Temporal Difference Neural Network (KBTDNN): A Controlled Reassessment of Belief Input, Kalman Gain and Training Loop
In this controlled empirical reassessment and ablation of KBTDNN, our original Kalman-based temporal difference neural network, a light-dark task with position-dependent sensor accuracy is introduced to analyze the effect of covariance under conditions in which uncertainty becomes state- and trajectory-dependent, unlike in the original tasks. Policy input, Kalman gain, and the training loop are varied separately under pre-specified hypotheses H1–H3. The re-implementation reaches 0.80 and 0.75 of the Bayes-optimal tiger and maze ceilings. The shortfalls come from the belief reward’s single-Gaussian posterior (+1.34 to +1.40) and the training loop (+0.075 to +0.078 per step), respectively, not the filter state. The covariance is a clock on the tiger (H1 refuted), not confirmed in either direction on the maze and cart-pole. The pre-specified light-dark instance left every pre-specified learner’s interquartile mean at the never-stop floor (H1 refuted as scored; the test uninformative). On a post hoc noisy-action instance, the policy using the belief mean and covariance outperformed the mean-only policy on ten seeds (26.75 versus 6.05 to 7.72) and nine of ten fresh ones: an exploratory existence result. After a cart-pole sensor-noise switch, online innovation-adaptive and offline-pre-trained gains reduced estimation error without a supported return improvement (H2 refuted as scored), and fine-tuned gains diverged.
Authors
- Alp Sardag (ORCID: https://orcid.org/0009-0005-2850-0837)
- Abdullah Sardağ
Institutions
- Istanbul Topkapi University (TR)
Publication Details
- Journal
- Applied Sciences
- Published
- 2026-11-19
- DOI
- https://doi.org/10.3390/app162211493
- Primary Topic
- Reinforcement Learning in Robotics
- Type
- article
- Field-Weighted Citation Impact
- 0.00