Shared Control with Black Box Agents Using Oracle Queries

Shared control problems deal with an agent learning to act in collaboration with other agents or systems. When learning a shared control policy, often a short communication between the agents can significantly reduce running times and improve the system's accuracy. We extend the shared control problem to include the ability to directly query a cooperating agent. We consider two types of potential responses to a query, namely oracles: one that assumes that the oracle has a perfect knowledge of the shared system and can provide the learner with the best action it should take, even when that action might be myopically wrong (a teacher), and one with a bounded knowledge limited to its own part of the system (an expert). Given this additional information channel, this work presents several heuristics for choosing when to query: reinforcement learning-based, utility-based, and entropy-based. These heuristics are aimed at reducing the overall learning cost of a system. Empirical results on two domains show the benefits of querying to learn a better control policy and demonstrate the tradeoffs between the proposed heuristics.