Training Observable Control Policies to Expose Agent State Through Actions

Physical or operational constraints often impose communication limitations on autonomous agents. Such limitations complicate monitoring or multi-agent coordination. Even when strong communications are absent, some information may still be available. The remainder of the relevant agent state may be reconstructed via estimation. The actions taken by an agent are a potential source of information—as the agent interacts with the environment, these actions may be observed even in the absence of explicit communication. We investigate using actions to estimate the state of an agent, using reinforcement learning to develop policies that make the estimation problem more tractable. Policy observability is encouraged through the training reward and is analyzed using simulations of the trained agent. In an aircraft tracking problem, a policy with enhanced observability is found that has minimal impact on nominal task performance.