MAT: Marker-Lattice Transformer for Entity, Relation and Attribute Extraction From Chinese Clinical Text

Electronic Medical Records (EMRs), extensively recognized as a significant repository of clinical experience and medical knowledge, often jeopardize their utility by being typically penned in free text, resulting in under-structured information. This lack of structural organization poses a major hurdle in fully capitalizing on the medical data embedded in these EMRs. Medical information extraction (IE) can fill this gap by converting the clinic text to structured data. In this paper, we propose <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</b>arker L<bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A</b>ttice <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">T</b>ransformer (MAT), a strong framework for medical IE. This framework is composed of three separate models, each designed for a specific task: medical entity recognition (MER), medical relation extraction (MRE), and medical attribute extraction (MAE). All the models are deeply based on <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">markers</i> embedded in the input text, with which the models compute representations from bottom to top layers. This allows the representations to encode deep semantic information, leading to better outputs. In addition, we enhance the models by <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">lattice</i>-style incorporation of medical dictionary information, further pre-training on large-scale EMRs, and auxiliary inputs of medical departments and EMR sections. We evaluated MAT using the HwaMei-500 dataset, the most comprehensive and current collection of Chinese electronic medical records. The framework demonstrated exceptional performance, achieving <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$F_{1}$</tex-math></inline-formula> scores of 93.0%, 71.5%, and 88.9% for MER, MRE, and MAE tasks respectively. These scores outperform the baseline by large margins and also surpass the results of recent state-of-the-art relation extraction models. As a medical IE framework, we provide a practical example for other information extraction efforts, particularly those focusing on Chinese electronic medical record data. In the spirit of collaborative research, we have publicly released our code and all relevant resources at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/WangSheng21s/MAT</uri>.