How to Decide Like Human? A Commonsense-Aware Hierarchical Framework for Knowledge Graph Reasoning

Reasoning over knowledge graphs has attracted considerable attention from researchers and is being widely applied to contribute question answering systems, recommender systems, and other information retrieval systems. However, existing reasoning methods tend to suffer from poor interpretability which is not consistent with human commonsense. The trustworthiness and reliability of the knowledge discover outcomes thus decreased as a result. Inspired by the process of human decision-making, we propose a commonsense-aware hierarchical framework called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">HDLH</i>, which incorporates commonsense knowledge into hierarchical knowledge graph reasoning process with deep reinforcement learning. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">HDLH</i> implements hierarchical reasoning process through exploration and exploitation sequentially by applying multi-agent reinforcement learning. Multiple agents in <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">HDLH</i> simulate the multi-level decision-making ability of humans, and reason hierarchically and reasonably to maintain its efficiency and interpretability. Moreover, commonsense knowledge is incorporated by means of the reward-shaping function, ultimately guiding the agent to reason more consistently with human perceptions and reduce the huge search space. We evaluated <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">HDLH</i> with various tasks on five real-world datasets. The experimental results reveal that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">HDLH</i> achieves better performance compared with state-of-the-art baseline models.