Annotation Guideline for Free-Text Activity Functioning Information: Communication & Cognition

Creation of this guideline was driven by a use case involving a collaborative agreement with the United States (U.S.) Social Security Administration (SSA) to leverage clinical natural language processing to improve their disability determination process. The research purpose was to develop tools that automate the detection of relevant functioning information in electronic health records (EHR) to support SSA adjudicators’ case review. This guideline aims to standardize information corresponding to the domain of Communication & Cognition (ComCog) documented in free-text EHRs and provide a process for annotating this information. These efforts resulted in the development of annotated datasets that have been used to train computational models to automatically identify ComCog information. Contents of the guideline include an (1) introduction to the guideline and its use, a description of the ICF as the conceptual framework, and an explanation of how ComCog is conceptualized; (2) data description; (3) annotation tool description; (4) annotation schema; (5) general annotation considerations; (6) specific considerations for ComCog; (7) organization of ICF concepts for annotation with examples of free text annotation; (8) examples of how to annotate attributes; (9) challenges and considerations for annotating at the activity level; (10) acknowledgements; (11) references; and (12) three appendices including an example of a synthetic clinical note that has been annotated for ComCog information. This guideline was developed by clinical rehabilitation domain experts with backgrounds in occupational therapy, physical therapy, and primary care medicine, and individuals with public health and data science expertise. Additional input was sought from domain experts in psychiatry and speech-language pathology. This guideline represents an initial effort at conceptualizing and standardizing ComCog concepts as documented in clinical notes. We anticipate that future iterations would incorporate additional perspectives to strengthen the guideline for annotation of Communication & Cognition information at the activity level in medical records.