Defining the Clinical Phenotypic Ground Truth: Expert Interrater Agreement in Hyperkinetic Movement Disorders

BACKGROUND: Reliable clinical phenotyping is crucial for diagnosis, treatment, and research in hyperkinetic movement disorders, yet its interrater reliability remains uncertain. OBJECTIVES: To quantify interrater agreement in clinical phenotyping of hyperkinetic movement disorders among experts in a large, phenotypically diverse cohort. METHODS: Within the NEMO project, 161 patients with hyperkinetic movement disorders underwent standardized video assessments. Each patient was independently classified by three international movement disorders experts in a structured, stepwise procedure. Experts first assessed videos alone, then received clinical history and examination, and finally diagnostic test results. At each step, they assigned primary and secondary phenotypes and rated their diagnostic confidence. Interrater agreement was assessed using Fleiss' and Cohen's kappa. RESULTS: Across all phenotypes, interrater agreement on the primary phenotype increased with added information (κ = 0.457 to κ = 0.607). Agreement with the NEMO reference classification similarly improved (κ = 0.478 to κ = 0.656). Among patients with a single dominant phenotype, agreement reached κ = 0.78. In 93% of cases, at least two out of three experts reached consensus. Agreement was substantially lower in patients with mixed or complex presentations, such as myoclonus-dystonia and functional movement disorders. Diagnostic certainty rose throughout, though secondary phenotype agreement remained poor. Experts rarely revised initial impressions, suggesting robust pattern recognition but also potential confirmation bias. CONCLUSIONS: Movement disorders specialists can reliably identify hyperkinetic phenotypes, even without in-person assessment. While complex, mixed and functional presentations remain diagnostically challenging, our findings reinforce the value of expert phenotyping in clinical care, training, and research, and provide a benchmark for future studies and computational approaches.

Authors

Institutions

Publication Details

Journal
Movement Disorders Clinical Practice
Published
2026-10-05
DOI
https://doi.org/10.1002/mdc3.70808
Primary Topic
Neurological disorders and treatments
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Defining the Clinical Phenotypic Ground Truth: Expert Interrater Agreement in Hyperkinetic Movement Disorders

Rick Brandsma, Inge Tuitert, Christos Ganos, Roberto Erro et al.
Movement Disorders Clinical Practice
Neurological disorders and treatments
article

Defining the Clinical Phenotypic Ground Truth: Expert Interrater Agreement in Hyperkinetic Movement Disorders

Rick Brandsma, Inge Tuitert, Christos Ganos, Roberto Erro, Kathryn J. Peall, Agnita J.W. Boon, Michèle Tinazzi, Cristian Falup‐Pecurariu, Petra Schwingenschuh, Gertrúd Tamás, Marina A.J. Tijssen, Rick C. G. Helmich, Hans Koelman, A.M. Madelein van der Stouwe, Eavan M. McGovern, Hendriekje Eggink, Emmanuel Roze, Rodi Zutt, Alexander Maximilian Münchau, Alberto J. Espay, Abhimanyu Mahajan, Francesca Morgante, Andres Deik, Pattamon Panyakaew, María Stamelou, Christine Klein, Selma Aybek, Tove Henriksen, Anna Sadnicka, Lisette Henriëtta Koens, Susanne A. Schneider, Matěj Škorvánek, Tereza Serranová, Sterre van der Veen, Emilia Mabel Gatto, Belén Pérez Dueñas, Marenka Smit, Jeroen J. de Vries, Wim Vandenberghe, Stephen Frucht, Anna Valadas, Rogier Luiken, Olof Vermeulen, Vincent Odekerken
article en

Abstract

BACKGROUND: Reliable clinical phenotyping is crucial for diagnosis, treatment, and research in hyperkinetic movement disorders, yet its interrater reliability remains uncertain. OBJECTIVES: To quantify interrater agreement in clinical phenotyping of hyperkinetic movement disorders among experts in a large, phenotypically diverse cohort. METHODS: Within the NEMO project, 161 patients with hyperkinetic movement disorders underwent standardized video assessments. Each patient was independently classified by three international movement disorders experts in a structured, stepwise procedure. Experts first assessed videos alone, then received clinical history and examination, and finally diagnostic test results. At each step, they assigned primary and secondary phenotypes and rated their diagnostic confidence. Interrater agreement was assessed using Fleiss' and Cohen's kappa. RESULTS: Across all phenotypes, interrater agreement on the primary phenotype increased with added information (κ = 0.457 to κ = 0.607). Agreement with the NEMO reference classification similarly improved (κ = 0.478 to κ = 0.656). Among patients with a single dominant phenotype, agreement reached κ = 0.78. In 93% of cases, at least two out of three experts reached consensus. Agreement was substantially lower in patients with mixed or complex presentations, such as myoclonus-dystonia and functional movement disorders. Diagnostic certainty rose throughout, though secondary phenotype agreement remained poor. Experts rarely revised initial impressions, suggesting robust pattern recognition but also potential confirmation bias. CONCLUSIONS: Movement disorders specialists can reliably identify hyperkinetic phenotypes, even without in-person assessment. While complex, mixed and functional presentations remain diagnostically challenging, our findings reinforce the value of expert phenotyping in clinical care, training, and research, and provide a benchmark for future studies and computational approaches.

Movement Disorders Clinical Practice
Semmelweis University (HU), University of Verona (IT), Universitat Autònoma de Barcelona (ES), Centre National de la Recherche Scientifique (FR), Thai Red Cross Society (TH), University of Salerno (IT), University Medical Center Groningen (NL), Royal College of Surgeons in Ireland (IE), Radboud University Nijmegen (NL), University of Fribourg (CH), Inserm (FR), Chulalongkorn University (TH), St George's, University of London (GB), University of Groningen (NL), Johannes Gutenberg University Mainz (DE), Medical University of Graz (AT), University of Pavol Jozef Šafárik (SK), Charles University (CZ), Toronto Western Hospital (CA), Universitair Ziekenhuis Leuven (BE), University Hospital of Bern (CH), Beaumont Hospital (IE), Bispebjerg Hospital (DK), King Chulalongkorn Memorial Hospital (TH), Radboud University Medical Center (NL), University Medical Center Utrecht (NL), Transylvania University of Brașov (RO), Sorbonne Université (FR), University Medical Center of the Johannes Gutenberg University Mainz (DE), Vall d'Hebron Institut de Recerca (ES), Univerzitná Nemocnica Louisa Pasteura (SK), Martini Ziekenhuis (NL), Oxford Centre for Computational Neuroscience (GB), Hygeia Hospital (GR), Institut de Psychiatrie et Neurosciences de Paris (FR), Hospital de Niños de la Santísima Trinidad (AR), Haga Hospital (NL), Hospital de Santa Maria (PT), National Hospital for Neurology and Neurosurgery (GB), Amsterdam University Medical Centers (NL), General University Hospital in Prague (CZ), Krembil Brain Institute, Ommelander Ziekenhuis Groningen (NL), University College London (GB), New York University (US), University of Cincinnati (US), Cardiff University (GB), University of Amsterdam (NL), Erasmus University Rotterdam (NL), University of Lübeck (DE), KU Leuven (BE)
Openalex Percentile: Top 12%
Neurological disorders and treatments
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.