REINFORCEMENT LEARNING AND KNOWLEDGE SHARING IN THE DECISION-MAKING PROCESS

Modern autonomous systems face increasingly complex decision-making challenges across various domains, from telecommunications networks to robotic swarms. These systems must operate in dynamic environments where the state space is large and complex, multiple agents need to coordinate under unreliable communications, and learning must occur across distributed, heterogeneous systems while preserving privacy. Traditional approaches to decision-making often fail to address these fundamental challenges, limiting the deployment and effectiveness of autonomous systems in real-world applications. This dissertation investigates four critical challenges in modern decision-making systems. The first challenge concerns the complexity of state spaces in single-agent decision-making. As systems become more sophisticated, the state space grows exponentially, making it difficult for reinforcement learning agents to learn effective policies. This is further complicated by the variability of operating conditions—what works well in one scenario may fail in another. Traditional approaches either attempt to handle the entire state space at once, leading to poor convergence and performance, or rely on hand-crafted feature engineering that doesn’t generalize well to new scenarios. The second challenge concerns long-horizon tasks that must be decomposed into intermediate goals and low-level actions. Large language models can act from natural-language observations without task-specific training, but most language-based planners operate in an open loop, do not learn from environmental feedback across episodes, and cannot tell, from a long failed trajectory, whether a wrong goal or a wrong action caused the failure. The third challenge emerges in multi-agent systems where coordination is essential but complicated by unreliable communications. In real-world deployments, communication links between agents can fail or become unstable due to various factors such as physical obstacles, network congestion, or hardware failures. Most existing multi-agent reinforcement learning approaches assume perfect communication during both training and execution, making them impractical for real-world applications. When communications fail, agents can make conflicting decisions that lead to poor performance or even dangerous situations. The fourth challenge arises in distributed learning scenarios where knowledge must be shared across heterogeneous systems while maintaining privacy. Traditional federated learning approaches assume homogeneous models and identically distributed data across clients, which rarely hold in practice. Real-world systems often have different model architectures optimized for their specific tasks and computational constraints, and data distributions vary significantly across different locations or use cases. These differences, combined with privacy requirements that prohibit direct data sharing, make it difficult to effectively share knowledge across distributed systems. To address these challenges, this dissertation presents five complementary frameworks. Cascade Reinforcement Learning (CaRL) and CaRL+ manage complex state spaces. Retrieval-Augmented Hierarchical in-context reinforcement Learning (RAHL) decomposes language-based tasks and improves multi-episode decisions through retrieved modular reflections. Aerial-DeepSearch+ supports continuous and satellite-assisted multi-UAV search under delayed information. Hierarchical and Transformer Multi-Agent Actor-Critic (HMAAC and TMAAC) provide robust coordination under unreliable communication. Fed2KD+ enables privacy-preserving knowledge sharing across heterogeneous systems. These solutions are validated through comprehensive evaluations in simulated environments, interactive language environments, hardware experiments, and real-world deployments. By addressing these fundamental challenges in decision-making and knowledge sharing, this dissertation contributes to the broader goal of making autonomous systems more practical and reliable in real-world applications. The proposed solutions demonstrate that effective autonomous decision-making is possible even in complex environments with unreliable communications and privacy constraints, paving the way for wider deployment of autonomous systems across various domains.

Authors

Publication Details

Journal
Rutgers University Community Repository (Rutgers University)
Published
2026-11-17
DOI
https://doi.org/10.7282/t3-0a43-kc19
Primary Topic
Reinforcement Learning in Robotics
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

REINFORCEMENT LEARNING AND KNOWLEDGE SHARING IN THE DECISION-MAKING PROCESS

Chuanneng Sun
Rutgers University Community Repository (Rutgers University)
Reinforcement Learning in Robotics
article

REINFORCEMENT LEARNING AND KNOWLEDGE SHARING IN THE DECISION-MAKING PROCESS

Chuanneng Sun
article en

Abstract

Modern autonomous systems face increasingly complex decision-making challenges across various domains, from telecommunications networks to robotic swarms. These systems must operate in dynamic environments where the state space is large and complex, multiple agents need to coordinate under unreliable communications, and learning must occur across distributed, heterogeneous systems while preserving privacy. Traditional approaches to decision-making often fail to address these fundamental challenges, limiting the deployment and effectiveness of autonomous systems in real-world applications. This dissertation investigates four critical challenges in modern decision-making systems. The first challenge concerns the complexity of state spaces in single-agent decision-making. As systems become more sophisticated, the state space grows exponentially, making it difficult for reinforcement learning agents to learn effective policies. This is further complicated by the variability of operating conditions—what works well in one scenario may fail in another. Traditional approaches either attempt to handle the entire state space at once, leading to poor convergence and performance, or rely on hand-crafted feature engineering that doesn’t generalize well to new scenarios. The second challenge concerns long-horizon tasks that must be decomposed into intermediate goals and low-level actions. Large language models can act from natural-language observations without task-specific training, but most language-based planners operate in an open loop, do not learn from environmental feedback across episodes, and cannot tell, from a long failed trajectory, whether a wrong goal or a wrong action caused the failure. The third challenge emerges in multi-agent systems where coordination is essential but complicated by unreliable communications. In real-world deployments, communication links between agents can fail or become unstable due to various factors such as physical obstacles, network congestion, or hardware failures. Most existing multi-agent reinforcement learning approaches assume perfect communication during both training and execution, making them impractical for real-world applications. When communications fail, agents can make conflicting decisions that lead to poor performance or even dangerous situations. The fourth challenge arises in distributed learning scenarios where knowledge must be shared across heterogeneous systems while maintaining privacy. Traditional federated learning approaches assume homogeneous models and identically distributed data across clients, which rarely hold in practice. Real-world systems often have different model architectures optimized for their specific tasks and computational constraints, and data distributions vary significantly across different locations or use cases. These differences, combined with privacy requirements that prohibit direct data sharing, make it difficult to effectively share knowledge across distributed systems. To address these challenges, this dissertation presents five complementary frameworks. Cascade Reinforcement Learning (CaRL) and CaRL+ manage complex state spaces. Retrieval-Augmented Hierarchical in-context reinforcement Learning (RAHL) decomposes language-based tasks and improves multi-episode decisions through retrieved modular reflections. Aerial-DeepSearch+ supports continuous and satellite-assisted multi-UAV search under delayed information. Hierarchical and Transformer Multi-Agent Actor-Critic (HMAAC and TMAAC) provide robust coordination under unreliable communication. Fed2KD+ enables privacy-preserving knowledge sharing across heterogeneous systems. These solutions are validated through comprehensive evaluations in simulated environments, interactive language environments, hardware experiments, and real-world deployments. By addressing these fundamental challenges in decision-making and knowledge sharing, this dissertation contributes to the broader goal of making autonomous systems more practical and reliable in real-world applications. The proposed solutions demonstrate that effective autonomous decision-making is possible even in complex environments with unreliable communications and privacy constraints, paving the way for wider deployment of autonomous systems across various domains.

Rutgers University Community Repository (Rutgers University)
Reinforcement Learning in Robotics
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon