Assessing Human Viewpoints in Theory of Mind for Large Language Models in Open-Ended Questioning

Theory of Mind (ToM) reasoning involves understanding that others have unique mental states-like beliefs, thoughts, intentions, viewpoints, and emotions-different from one's own, and incorporating this into one's reasoning. While some research suggests that LLMs possess reasoning abilities, other studies challenge this assertion, often focusing on structured responses and overlooking the complexities of open-ended interactions. As LLMs are increasingly employed in different sectors, their ability to accurately interpret human mental states in reasoning becomes critical. For example, in psychological services, if LLMs generate reasoning responses without understanding human mental states, their answers may lack logical soundness and potentially exacerbate client distress. Therefore, understanding LLMs' ToM capabilities is crucial to ensure they deliver effective and appropriate responses in real-world scenarios. In this research, I investigate the effectiveness of incorporating questioners' viewpoints in the questions-whether posed in a rational or intuitive manner-on the generation of reasoning answers by LLMs and how these generated answers align with human-written responses. The results demonstrate that incorporating these viewpoints into the prompt instructions enhances the reasoning performance of LLMs, although the responses still fall short of being truly human-like. This research contributes to the information retrieval and generative AI community by raising awareness about the limitations of LLMs in reasoning and their alignment with human responses in this domain.