Bridging AI and Business: Are Large Language Models Good Management Consultants?

We present an evaluation of ten state-of-the-art large language models (LLMs) taking the role of management consultants. The assessment is performed in a simulated interview setting where the LLMs are tasked with solving case studies – a popular testing procedure during job interviews at management consulting firms. We compose a dataset of eight annotated case studies with a total of 37 business questions and corresponding reference answers. We record the LLMs’ answers and evaluate their performance in terms of outcome and process validity. Our findings indicate that eight out of the ten evaluated LLMs would pass these case interviews, exceeding the minimum level of performance expected from human candidates. Further, the LLMs seem to take measures to ensure high-quality decisions. These findings highlight the potential of LLMs to serve as strategic advisors to managers in charge of making reliable decisions.