Developing a named entity framework for thyroid cancer staging and risk level classification using large language models
We developed a named entity (NE) framework for information extraction from semi-structured clinical notes retrieved from The Cancer Genome Atlas-Thyroid Cancer (TCGA-THCA) database and examined Large Language Models (LLMs) strategies to classify the 8<sup>th</sup> edition of American Joint Committee on Cancer (AJCC) staging and American Thyroid Association (ATA) risk category for patients with well-differentiated thyroid cancer. The NE framework consisted of annotation guidelines development, ground truth labelling, prompting approaches, and evaluation codes. Four LLMs (Mistral-7B-Instruct, Llama-3.1-8B-Instruct, Gemma-2-9B-Instruct, and Qwen2.5-7B-Instruct) were offline utilised for information extraction, comparing with expert-curated ground truth. Our framework was developed using 50 TCGA-THCA pathology notes. 289 TCGA-THCA notes and 35 pseudo-clinical cases were used for validation. Taking an ensemble-like majority-vote strategy achieved satisfactory performance for AJCC and ATA in both development and validation sets. Our framework and ensemble classifier optimised efficiency and accuracy of classifying stage and risk category in thyroid cancer patients.
Authors
- Matrix Man Him Fung (ORCID: https://orcid.org/0000-0002-2812-0616)
- Yan Luk (ORCID: https://orcid.org/0000-0002-0164-5260)
- Zoie Shui-Yee Wong (ORCID: https://orcid.org/0000-0003-4499-9779)
- Joshua W. K. Ho (ORCID: https://orcid.org/0000-0003-2331-7011)
- I. Tai (ORCID: https://orcid.org/0000-0002-0256-5705)
- Kathy Leung (ORCID: https://orcid.org/0000-0003-4777-388X)
- Carlos King Ho Wong (ORCID: https://orcid.org/0000-0002-6895-6071)
- Ivan Chi Ho Au (ORCID: https://orcid.org/0000-0001-5904-8322)
- Eric Ho Man Tang (ORCID: https://orcid.org/0000-0003-4196-8686)
- Chun‐Ka Wong (ORCID: https://orcid.org/0000-0001-5205-9440)
- Zhili Wei
- Brian Hung‐Hin Lang (ORCID: https://orcid.org/0000-0002-9362-0086)
- Joseph T. Wu (ORCID: https://orcid.org/0000-0002-3155-5987)
- Jason W.H. Wong (ORCID: https://orcid.org/0000-0003-2953-7728)
- Xiaodong Liu (ORCID: https://orcid.org/0000-0003-1843-8660)
- Tingting Wu (ORCID: https://orcid.org/0000-0003-3437-8899)
- Victor Lee (ORCID: https://orcid.org/0000-0002-6283-978X)
- Winnie Cheng (ORCID: https://orcid.org/0000-0003-0709-0419)
Institutions
- The University of Sydney (AU)
- Chinese University of Hong Kong (HK)
- UNSW Sydney (AU)
- London School of Hygiene & Tropical Medicine (GB)
- St. Luke's International University (JP)
- Hong Kong Jockey Club (HK)
- University of Hong Kong - Shenzhen Hospital (CN)
- University of Hong Kong (HK)
Publication Details
- Journal
- npj Digital Medicine
- Published
- 2025-03-01
- DOI
- https://doi.org/10.1038/s41746-025-01528-y
- Citations
- 7
- Primary Topic
- Biomedical Text Mining and Ontologies
- Type
- article
- Field-Weighted Citation Impact
- 5.64
Funders
- Innovation and Technology Commission