Achieving Preferable Agreement by Utilizing offline Negotiation dialogue on Decision Transformer

In recent years, “automatic negotiation agents”, which utilize AI to solve negotiation problems, have attracted much attention. A negotiation problem is a problem that leads to an agreement to be reached that the decision-maker is satisfied with among possible alternatives. An automatic negotiation agent is expected to maximize decision-making utility in combinatorial optimization and sequential decision-making problems involving large-scale negotiation problems. So far, the best automatic negotiation agents employ deep reinforcement learning (DRL). However, the DRL approach requires interaction with the negotiation environment. Therefore, to obtain a negotiation strategy, preparing the counterparty to negotiate in advance is necessary, which could be more practical. Therefore, this study proposes a method to acquire negotiation strategies using only previously acquired negotiation history data without requiring learning through interaction with the negotiation environment. In order to learn the sequential relationships among data in history, we introduce a Decision Transformer that can explicitly handle causal relationships by time series. The performance of the proposed method was evaluated and compared with that of the Behavior Cloning (BC) method, which learns only from the best history, using a negotiation environment in which two agents exchange proposals. Computer experiments confirm that the average utility obtained by the proposed method is higher than that obtained by the BC method. We also analyze the bargaining strategies acquired using the proposed method to understand their performance.