Protecting medical privacy with pretrained language models: named entity recognition-based de-identification in Korean unstructured electronic medical records

This study introduces a lightweight de-identification software tailored to the Korean healthcare environment, where heterogeneous document formats and limited computing resources hinder clinical data integration. The proposed system protects sensitive patient information while preserving essential clinical content—such as diagnoses, surgical schedules, and prescriptions—for research purposes. We defined de-identification categories specific to the Korean context and implemented a preprocessing pipeline optimized for discharge summaries containing mixed Korean, English, and special characters. To address the lack of high-quality Korean Named Entity Recognition (NER) datasets, we applied Korean-specific data augmentation and fine-tuned the Korean Language Understanding Evaluation (KLUE) Bidirectional Encoder Representations from Transformers (BERT) model to enhance generalizability. Model performance was compared against lightweight Korean large language models (LLMs), including Llama3-Open-Ko-8B and EEVE-Korean-Instruct-10.8B. The KLUE BERT model, trained on an augmented dataset, achieved an F1 score of 91.42% on the internal validation set and maintained 94.30% on real discharge summary data. Notably, it outperformed LLMs in recognizing more than 200 categories of sensitive entities, demonstrating superior performance. This compact solution offers a scalable and privacy-preserving approach for anonymizing electronic medical records (EMRs) in clinical settings.