REINFORCEMENT LEARNING AND KNOWLEDGE SHARING IN THE DECISION-MAKING PROCESS
Modern autonomous systems face increasingly complex decision-making challenges across various domains, from telecommunications networks to robotic swarms. These systems must operate in dynamic environments where the state space is large and complex, multiple agents need to coordinate under unreliable communications, and learning must occur across distributed, heterogeneous systems while preserving privacy. Traditional approaches to decision-making often fail to address these fundamental challenges, limiting the deployment and effectiveness of autonomous systems in real-world applications. This dissertation investigates four critical challenges in modern decision-making systems. The first challenge concerns the complexity of state spaces in single-agent decision-making. As systems become more sophisticated, the state space grows exponentially, making it difficult for reinforcement learning agents to learn effective policies. This is further complicated by the variability of operating conditions—what works well in one scenario may fail in another. Traditional approaches either attempt to handle the entire state space at once, leading to poor convergence and performance, or rely on hand-crafted feature engineering that doesn’t generalize well to new scenarios. The second challenge concerns long-horizon tasks that must be decomposed into intermediate goals and low-level actions. Large language models can act from natural-language observations without task-specific training, but most language-based planners operate in an open loop, do not learn from environmental feedback across episodes, and cannot tell, from a long failed trajectory, whether a wrong goal or a wrong action caused the failure. The third challenge emerges in multi-agent systems where coordination is essential but complicated by unreliable communications. In real-world deployments, communication links between agents can fail or become unstable due to various factors such as physical obstacles, network congestion, or hardware failures. Most existing multi-agent reinforcement learning approaches assume perfect communication during both training and execution, making them impractical for real-world applications. When communications fail, agents can make conflicting decisions that lead to poor performance or even dangerous situations. The fourth challenge arises in distributed learning scenarios where knowledge must be shared across heterogeneous systems while maintaining privacy. Traditional federated learning approaches assume homogeneous models and identically distributed data across clients, which rarely hold in practice. Real-world systems often have different model architectures optimized for their specific tasks and computational constraints, and data distributions vary significantly across different locations or use cases. These differences, combined with privacy requirements that prohibit direct data sharing, make it difficult to effectively share knowledge across distributed systems. To address these challenges, this dissertation presents five complementary frameworks. Cascade Reinforcement Learning (CaRL) and CaRL+ manage complex state spaces. Retrieval-Augmented Hierarchical in-context reinforcement Learning (RAHL) decomposes language-based tasks and improves multi-episode decisions through retrieved modular reflections. Aerial-DeepSearch+ supports continuous and satellite-assisted multi-UAV search under delayed information. Hierarchical and Transformer Multi-Agent Actor-Critic (HMAAC and TMAAC) provide robust coordination under unreliable communication. Fed2KD+ enables privacy-preserving knowledge sharing across heterogeneous systems. These solutions are validated through comprehensive evaluations in simulated environments, interactive language environments, hardware experiments, and real-world deployments. By addressing these fundamental challenges in decision-making and knowledge sharing, this dissertation contributes to the broader goal of making autonomous systems more practical and reliable in real-world applications. The proposed solutions demonstrate that effective autonomous decision-making is possible even in complex environments with unreliable communications and privacy constraints, paving the way for wider deployment of autonomous systems across various domains.
Authors
- Chuanneng Sun (ORCID: https://orcid.org/0000-0001-7524-9044)
Publication Details
- Journal
- Rutgers University Community Repository (Rutgers University)
- Published
- 2026-11-17
- DOI
- https://doi.org/10.7282/t3-0a43-kc19
- Primary Topic
- Reinforcement Learning in Robotics
- Type
- article
- Field-Weighted Citation Impact
- 0.00