How far from energy-optimal are learned quadrotor navigation policies? Flight-calibrated energy modelling, optimal-control bounds, and wind- and budget-aware learning

Abstract Energy-aware deep reinforcement learning for multirotor navigation is usually evaluated by comparing one learned policy with another, which establishes an ordering but leaves the quantity a designer needs unmeasured: how much energy is still being left on the table. We supply the missing reference. A blade-element/momentum energy model with a Thevenin battery, embedded in six-degree-of-freedom dynamics with a geometric inner loop, is calibrated on the cruise regime of 188 instrumented DJI Matrice 100 delivery flights at two payloads; it then predicts cruise power at a held-out payload to 2.3% and the power of two flight regimes that played no part in the calibration – climb and descent – to 6.9% and 1.9%, a pooled held-out error of 3.5% against the 2.1% of the empirical regression it is tested on. The optimal mission energy is bracketed between a rigorous lower bound, the minimum energy per metre of the vehicle times the straight-line distance, and an achievable reference that is planned, given an exactly optimal speed profile by dynamic programming, and then flown in the same simulator as the learned policies; across 200 held-out urban-delivery scenarios the reference solves 99% and uses 1.31 times the lower bound. The ratio of a policy’s energy to that reference is therefore a conservative optimality gap: a rigorous lower bound on how far the policy is from optimal. Over four seeds per configuration, soft actor-critic with an energy term in the reward attains a gap of 7.3% (95% CI 4.0–12.3) against 14.8% (8.6–25.0) without it, with a large effect size (Hedges’ g = − 0.94) that four seeds cannot certify after multiplicity correction. Decomposing the excess by solving the minimum-energy speed profile along the path each policy actually flew attributes roughly half of it to speed selection rather than routing, and the speed term is the one the energy signal moves, falling from 11.1% to 4.9%; routing is the part a map would remove, speed is not. Energy awareness is not free: the energy reward costs about nine points of success rate at this budget. We also report what a single seed would have concluded – a 2.8% gap against 26.8% – as a caution for a literature that commonly reports one or two. The bounds and the decomposition are cheap and reusable by any study reporting the energy of a learned flight policy.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-17
DOI
https://doi.org/10.21203/rs.3.rs-11749401/v1
Primary Topic
Reinforcement Learning in Robotics
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

How far from energy-optimal are learned quadrotor navigation policies? Flight-calibrated energy modelling, optimal-control bounds, and wind- and budget-aware learning

Iqrar Ali², SAYEED OMAR, Ma Guoli
2 links
Research Square
Reinforcement Learning in Robotics
preprint

How far from energy-optimal are learned quadrotor navigation policies? Flight-calibrated energy modelling, optimal-control bounds, and wind- and budget-aware learning

Iqrar Ali², SAYEED OMAR, Ma Guoli
preprint en

Abstract

Abstract Energy-aware deep reinforcement learning for multirotor navigation is usually evaluated by comparing one learned policy with another, which establishes an ordering but leaves the quantity a designer needs unmeasured: how much energy is still being left on the table. We supply the missing reference. A blade-element/momentum energy model with a Thevenin battery, embedded in six-degree-of-freedom dynamics with a geometric inner loop, is calibrated on the cruise regime of 188 instrumented DJI Matrice 100 delivery flights at two payloads; it then predicts cruise power at a held-out payload to 2.3% and the power of two flight regimes that played no part in the calibration – climb and descent – to 6.9% and 1.9%, a pooled held-out error of 3.5% against the 2.1% of the empirical regression it is tested on. The optimal mission energy is bracketed between a rigorous lower bound, the minimum energy per metre of the vehicle times the straight-line distance, and an achievable reference that is planned, given an exactly optimal speed profile by dynamic programming, and then flown in the same simulator as the learned policies; across 200 held-out urban-delivery scenarios the reference solves 99% and uses 1.31 times the lower bound. The ratio of a policy’s energy to that reference is therefore a conservative optimality gap: a rigorous lower bound on how far the policy is from optimal. Over four seeds per configuration, soft actor-critic with an energy term in the reward attains a gap of 7.3% (95% CI 4.0–12.3) against 14.8% (8.6–25.0) without it, with a large effect size (Hedges’ g = − 0.94) that four seeds cannot certify after multiplicity correction. Decomposing the excess by solving the minimum-energy speed profile along the path each policy actually flew attributes roughly half of it to speed selection rather than routing, and the speed term is the one the energy signal moves, falling from 11.1% to 4.9%; routing is the part a map would remove, speed is not. Energy awareness is not free: the energy reward costs about nine points of success rate at this budget. We also report what a single seed would have concluded – a 2.8% gap against 26.8% – as a caution for a literature that commonly reports one or two. The bounds and the decomposition are cheap and reusable by any study reporting the energy of a learned flight policy.

Research Square
TianjinSino-German University of Applied Sciences (CN)
Affordable and clean energy
Reinforcement Learning in Robotics
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon