Comparing GenAI platforms on cybersecurity management task performances

Purpose The persistent shortage of cybersecurity professionals, coupled with the consistent increase and complexity of cyberattacks, requires a novel examination of the processes and tasks performed by those professionals to cope with the workloads. Research shows that artificial intelligence (AI) tools often target technical rather than managerial tasks, highlighting the need for continued human involvement in cybersecurity management. This study aims to examine how using Generative AI (GenAI) for cybersecurity managerial tasks has the potential to assist in reducing human errors and perform repetitive tasks, thus, reducing the cybersecurity managerial loads, allowing them to focus on more strategic aspects of their work. Design/methodology/approach This experimental research study used five GenAI platforms: ChatGPT, CoPilot, Gemini, MetaAI and Claude. Each GenAI platform generated a real-life scenario and guidelines for cybersecurity managers associated with a managerial cybersecurity task and then cross-evaluated the scenarios and guidelines following predetermined metric measures of (1) relevancy, (2) accuracy and reliability, (3) completeness as well as (4) clarity. Scores were generated by each of the five GenAI platforms based on the four metric measures and ranged from 1 = very low to 10 = very high. Then the scores were averaged across all measures and all five GenAI platforms for an overall metrics score ranging from 1 to 10. Analysis of variance was conducted to test for mean differences. Findings The experimental results indicated that there was a statistically significant mean differences in the scores received between all scenarios (F = 7.841, df = 4, p < 0.001). Specifically, the scenario generated by Claude achieved the highest overall average score (9.3), followed by Gemini (9.0), MetaAI (8.9), ChatGPT (8.7) and CoPilot (8.5). In general, the scenario generated by Claude consistently performed well across all metrics by all five GenAI platforms. Practical implications The rapid integration of GenAI capabilities into everyday activity may suggest that cybersecurity managers must currently be trained to use AI tools in their daily operations to alleviate their workloads. Having said that, some ethical issues and risks of using GenAI for cybersecurity managerial tasks must be further studied. Social implications The cybersecurity workforce shortage was reported to exceed 4 million workers worldwide in 2024 and is estimated to exceed 5 million by the end of 2025. Thus, it is significant to further understand the role of AI in improving the efficiency of managerial cybersecurity tasks. Originality/value The value of this research lies in three facets. First, the demonstration of using GenAI to perform managerial cybersecurity tasks. Second, the novel methodology, in which the GenAI platforms assess the outputs by self- and cross-evaluating them. Finally, the development of novel metrics to assess managerial tasks can be of great value for researchers and industry.