An Analysis of Pre-trained Models to Identify Cybersecurity Incidents Entities in the Healthcare Industry

Healthcare institutions have always been a critical sector in any community. Cybersecurity issues, such as attacks or incidents, may impact their operations and cause damage that could eventually lead to patient death. Named Entity Recognition and Classification (NERC) can support these institutions in analyzing incidents, highlighting the incident’s type, attack type, and location, just to name a few examples. This work evaluated pre-trained machine learning models to comprehend how they help in this identification. For this purpose, we analyzed two fine-tuned BERT models used in a corpus with incidents related to Healthcare institutions in the U.S. We evaluated the entity recognition using both the Strict and Partial approaches. Experiment results indicated a higher precision (above 0.776) but with low Recall, with less than 0.267. This may indicate a good performance for entity recognition. However, the models missed many entities.