Deceptive sequential decision-making via regularised policy optimisation

Autonomous systems are increasingly expected to operate in the presence of adversaries, though adversaries may infer sensitive information simply by observing a system. Therefore, we present a deceptive sequential decision-making framework that not only conceals sensitive information, but actively misleads adversaries about it. We model autonomous systems as Markov decision processes, with adversaries using inverse reinforcement learning to recover reward functions. To counter them, we present three regularisation strategies for policy synthesis problems that actively deceive an adversary about a system’s reward. “Diversionary deception” leads an adversary to draw any false conclusion about a system’s reward function. “Targeted deception” leads an adversary to draw a specific false conclusion about a system’s reward function. “Equivocal deception” leads an adversary to infer that the real reward and a false reward both explain a system’s behaviour. We show how each form of deception can be implemented in policy optimisation problems and analytically bound the loss in total accumulated reward induced by deception. Next, we empirically evaluate these developments. We show that diversionary, targeted, and equivocal deception all steer the adversary to false beliefs while still attaining a total accumulated reward that is approximately 85% to 99% of its optimal, non-deceptive value.

Authors

Institutions

Publication Details

Journal
Journal of the Operational Research Society
Published
2026-11-17
DOI
https://doi.org/10.1080/01605682.2026.2757159
Citations
5
Primary Topic
Reinforcement Learning in Robotics
Type
article
Field-Weighted Citation Impact
1.06

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Deceptive sequential decision-making via regularised policy optimisation

Alexander Benvenuti, Yerin Kim, Matthew Hale, Nathaniel D. Bastian et al.
5 citations
Journal of the Operational Research Society
Reinforcement Learning in Robotics
1.06
article

Deceptive sequential decision-making via regularised policy optimisation

Alexander Benvenuti, Yerin Kim, Matthew Hale, Nathaniel D. Bastian, Abhishek N. Kulkarni, Ufuk Topcu, Mustafa O. Karabag, Bo Chen
article en
5 citations

Abstract

Autonomous systems are increasingly expected to operate in the presence of adversaries, though adversaries may infer sensitive information simply by observing a system. Therefore, we present a deceptive sequential decision-making framework that not only conceals sensitive information, but actively misleads adversaries about it. We model autonomous systems as Markov decision processes, with adversaries using inverse reinforcement learning to recover reward functions. To counter them, we present three regularisation strategies for policy synthesis problems that actively deceive an adversary about a system’s reward. “Diversionary deception” leads an adversary to draw any false conclusion about a system’s reward function. “Targeted deception” leads an adversary to draw a specific false conclusion about a system’s reward function. “Equivocal deception” leads an adversary to infer that the real reward and a false reward both explain a system’s behaviour. We show how each form of deception can be implemented in policy optimisation problems and analytically bound the loss in total accumulated reward induced by deception. Next, we empirically evaluate these developments. We show that diversionary, targeted, and equivocal deception all steer the adversary to false beliefs while still attaining a total accumulated reward that is approximately 85% to 99% of its optimal, non-deceptive value.

Journal of the Operational Research Society
Georgia Institute of Technology (US), United States Military Academy (US), The University of Texas at Austin (US)
National Science Foundation, Defense Advanced Research Projects Agency, Advanced Research Projects Agency, Office of Naval Research, National Science Foundation Graduate Research Fellowship Program
Openalex Percentile: Top 22%
Reinforcement Learning in Robotics
1.06
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon