Hierarchical Two-Agent Reinforcement Learning for Long-Horizon Energy Optimization and Short-Horizon Navigation in Autonomous Electric Boats

This paper presents a hierarchical two-agent deep reinforcement learning architecture for an electric racing boat, addressing the joint challenges of long-horizon energy management and short-horizon autonomous navigation. The system integrates two independently trained agents: an energy optimization agent based on an importance-weighted actor–learner architecture, selected for its ability to handle long-horizon decision-making and distributed training, and a navigation agent trained with Proximal Policy Optimization, chosen for its training stability. The navigation policy tracks a target speed issued by the energy agent for each track segment. This target speed is the only coupling between the two levels. A curriculum learning strategy is used for both tasks. In a race simulator calibrated on real telemetry, the energy agent reaches the full 34-lap configuration, with a training success rate of 0.51 on average and about 0.72 at the end of training, while the navigation agent reaches its 0.80 evaluation success criterion under a ±5 km/h tolerance. A Unity3D-based environment with hydrodynamic modeling and autopilot software-in-the-loop emulation provides a framework for validation prior to on-water deployment; the closed-loop evaluation of the coupled agents remains future work.

Authors

Institutions

Publication Details

Journal
Sensors
Published
2026-11-13
DOI
https://doi.org/10.3390/s26227167
Primary Topic
Reinforcement Learning in Robotics
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Hierarchical Two-Agent Reinforcement Learning for Long-Horizon Energy Optimization and Short-Horizon Navigation in Autonomous Electric Boats

Zamirddine Mari, Mohamad Motasem Nawaf, Jérôme Pasquet, Quentin Vassard
5 links
Sensors
Reinforcement Learning in Robotics
article

Hierarchical Two-Agent Reinforcement Learning for Long-Horizon Energy Optimization and Short-Horizon Navigation in Autonomous Electric Boats

Zamirddine Mari, Mohamad Motasem Nawaf, Jérôme Pasquet, Quentin Vassard
article en

Abstract

This paper presents a hierarchical two-agent deep reinforcement learning architecture for an electric racing boat, addressing the joint challenges of long-horizon energy management and short-horizon autonomous navigation. The system integrates two independently trained agents: an energy optimization agent based on an importance-weighted actor–learner architecture, selected for its ability to handle long-horizon decision-making and distributed training, and a navigation agent trained with Proximal Policy Optimization, chosen for its training stability. The navigation policy tracks a target speed issued by the energy agent for each track segment. This target speed is the only coupling between the two levels. A curriculum learning strategy is used for both tasks. In a race simulator calibrated on real telemetry, the energy agent reaches the full 34-lap configuration, with a training success rate of 0.51 on average and about 0.72 at the end of training, while the navigation agent reaches its 0.80 evaluation success criterion under a ±5 km/h tolerance. A Unity3D-based environment with hydrodynamic modeling and autopilot software-in-the-loop emulation provides a framework for validation prior to on-water deployment; the closed-loop evaluation of the coupled agents remains future work.

SensorsVol. 26(22)
Centre National de la Recherche Scientifique (FR), École Nationale Supérieure de Techniques Avancées (FR), Aix-Marseille Université (FR), Laboratoire d'Informatique, de Robotique et de Microélectronique de Montpellier (FR), Laboratoire d’Informatique et Systèmes (FR), École Nationale Supérieure de Techniques Avancées Paris (FR), Université de Montpellier Paul-Valéry (FR), Direction Générale de l'Armement (FR)
Affordable and clean energy
Reinforcement Learning in Robotics
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon