Hype or Revolution: How GPT4, Claude3 Opus, and GPT3.5 Exaggerate Their Influence on HRM Occupations Exposed in Practice

This study investigates the actual impact of ChatGPT on human resource management (HRM) occupations and compares it with assessments from large language models (LLMs). Using data from the Occupational Information Network (O*NET), we developed a novel methodology combining real-world evidence from Google search results with evaluations from GPT-4, GPT-3.5, and Claude 3 Opus. Our findings reveal significant variations in the degree of exposure among HRM occupations. Training and Development Managers were the most exposed, with 86.52% of their tasks potentially automated by ChatGPT, while Labor Relations Specialists were the least exposed at 11.08%. Notably, LLM assessments consistently overestimated the exposure levels by an average of 30% compared to real-world evidence. This discrepancy highlights the importance of critically evaluating LLM capabilities in task automation. Our research contributes to understanding AI's impact on HRM practices and provides valuable insights for professionals adapting to the AI era. It also underscores the need for caution when relying solely on LLM assessments for workforce planning and development.