Hierarchical Two-Agent Reinforcement Learning for Long-Horizon Energy Optimization and Short-Horizon Navigation in Autonomous Electric Boats
This paper presents a hierarchical two-agent deep reinforcement learning architecture for an electric racing boat, addressing the joint challenges of long-horizon energy management and short-horizon autonomous navigation. The system integrates two independently trained agents: an energy optimization agent based on an importance-weighted actor–learner architecture, selected for its ability to handle long-horizon decision-making and distributed training, and a navigation agent trained with Proximal Policy Optimization, chosen for its training stability. The navigation policy tracks a target speed issued by the energy agent for each track segment. This target speed is the only coupling between the two levels. A curriculum learning strategy is used for both tasks. In a race simulator calibrated on real telemetry, the energy agent reaches the full 34-lap configuration, with a training success rate of 0.51 on average and about 0.72 at the end of training, while the navigation agent reaches its 0.80 evaluation success criterion under a ±5 km/h tolerance. A Unity3D-based environment with hydrodynamic modeling and autopilot software-in-the-loop emulation provides a framework for validation prior to on-water deployment; the closed-loop evaluation of the coupled agents remains future work.
Authors
- Zamirddine Mari (ORCID: https://orcid.org/0009-0009-3277-7010)
- Mohamad Motasem Nawaf (ORCID: https://orcid.org/0000-0003-2156-8589)
- Jérôme Pasquet (ORCID: https://orcid.org/0000-0002-7993-4724)
- Quentin Vassard
Institutions
- Centre National de la Recherche Scientifique (FR)
- École Nationale Supérieure de Techniques Avancées (FR)
- Aix-Marseille Université (FR)
- Laboratoire d'Informatique, de Robotique et de Microélectronique de Montpellier (FR)
- Laboratoire d’Informatique et Systèmes (FR)
- École Nationale Supérieure de Techniques Avancées Paris (FR)
- Université de Montpellier Paul-Valéry (FR)
- Direction Générale de l'Armement (FR)
Publication Details
- Journal
- Sensors
- Published
- 2026-11-13
- DOI
- https://doi.org/10.3390/s26227167
- Primary Topic
- Reinforcement Learning in Robotics
- Type
- article
- Field-Weighted Citation Impact
- 0.00