Generating an Instruction Dataset to Build Cyber Intelligent Large Language Models

The rapid evolution of cybersecurity necessitates the commitment of professionals to continuously update their knowledge and skills. This challenges cybersecurity educators and trainers to update teaching materials to keep pace with vulnerabilities, technologies, and adversaries. Advancements in Artificial Intelligence (AI), especially within Large Language Models (LLMs) present opportunities to help build curriculum and training materials. However, many publicly available LLMs like BERT, GPT, and LLaMA lack deep knowledge in specific domains and require fine-tuning using specialized datasets. Due to its dynamic nature, the cybersecurity discipline faces a shortage of rich datasets, both general and specialized, necessary for effective fine-tuning. This paper presents the process and lessons learned from generating a cybersecurity instruction dataset using an LLM and cybersecurity learning modules created by humans. The aim is to create and publish for public use, a high-quality dataset that can be used to fine-tune pretrained LLMs (like BERT, LLaMA, and others) for NLP tasks in the cybersecurity domain with the larger goal of developing AI-generated curriculum.