Percentage gain versus normalized gain for identifying training targets in medical record documentation education: a pre–post study

Abstract Background Record audits are widely used to evaluate documentation training, and improvement is often compared across documentation domains using the percentage change in scores to judge which components worked and where to focus next. When post-training scores approach the instrument ceiling, this metric may mainly reflect baseline room for improvement. We compared percentage score gain with defect burden reduction, a normalized gain, and examined whether the defect profile changed after one training cycle. Methods We analysed blinded audit data from a self-paced micro-lecture program (53 lectures; 1327 min) for 49 physicians at a secondary-level hospital in China (December 2023 to July 2024). Five discharge records per physician were randomly sampled one month before and after training, de-identified, pooled, and scored on a 20-dimension instrument by four raters blinded to time point and author. Dimensions were ranked by each metric. Agreement between rankings, and between baseline and residual (post-training) defect burden, was assessed with Spearman correlation, with 95% uncertainty intervals (UIs) from 10 000 parametric simulations. Results The composite score increased from 89.14 ± 2.73 to 96.82 ± 1.38 ( P < 0.001), and post-training dimension scores converged within 96.04–97.97. Percentage score gain correlated with baseline score at r = − 0.960 (95% UI − 0.98 to − 0.90), versus r = − 0.649 (− 0.79 to − 0.35) for defect burden reduction. The two rankings were correlated ( r s = 0.707, 95% UI 0.52 to 0.90) but not interchangeable: only 2 of the top 5 dimensions overlapped, and 16 of 20 dimensions had rank intervals spanning more than 10 positions. Baseline and residual defect burden showed little agreement ( r s = 0.144, 95% UI − 0.33 to 0.43); informed consent moved from the lightest baseline burden to the second heaviest residual burden. Conclusions Near the instrument ceiling, percentage score gain largely reflects baseline room for improvement and should not be used alone to compare training effects across documentation domains. Normalized gain depends less on baseline, but dimension ranks are imprecise and are better reported with uncertainty intervals and in tiers. Targets for the next training cycle should be set from the residual defect burden re-measured after each cycle.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-12
DOI
https://doi.org/10.21203/rs.3.rs-11233839/v1
Primary Topic
Medical Coding and Health Information
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Percentage gain versus normalized gain for identifying training targets in medical record documentation education: a pre–post study

Zhihui Yu, Guoyong Deng, Chao Wang, Ziyi Chen et al.
Research Square
Medical Coding and Health Information
preprint

Percentage gain versus normalized gain for identifying training targets in medical record documentation education: a pre–post study

Zhihui Yu, Guoyong Deng, Chao Wang, Ziyi Chen, Qingxia Yang, Yidong Liu
preprint en

Abstract

Abstract Background Record audits are widely used to evaluate documentation training, and improvement is often compared across documentation domains using the percentage change in scores to judge which components worked and where to focus next. When post-training scores approach the instrument ceiling, this metric may mainly reflect baseline room for improvement. We compared percentage score gain with defect burden reduction, a normalized gain, and examined whether the defect profile changed after one training cycle. Methods We analysed blinded audit data from a self-paced micro-lecture program (53 lectures; 1327 min) for 49 physicians at a secondary-level hospital in China (December 2023 to July 2024). Five discharge records per physician were randomly sampled one month before and after training, de-identified, pooled, and scored on a 20-dimension instrument by four raters blinded to time point and author. Dimensions were ranked by each metric. Agreement between rankings, and between baseline and residual (post-training) defect burden, was assessed with Spearman correlation, with 95% uncertainty intervals (UIs) from 10 000 parametric simulations. Results The composite score increased from 89.14 ± 2.73 to 96.82 ± 1.38 ( P < 0.001), and post-training dimension scores converged within 96.04–97.97. Percentage score gain correlated with baseline score at r = − 0.960 (95% UI − 0.98 to − 0.90), versus r = − 0.649 (− 0.79 to − 0.35) for defect burden reduction. The two rankings were correlated ( r s = 0.707, 95% UI 0.52 to 0.90) but not interchangeable: only 2 of the top 5 dimensions overlapped, and 16 of 20 dimensions had rank intervals spanning more than 10 positions. Baseline and residual defect burden showed little agreement ( r s = 0.144, 95% UI − 0.33 to 0.43); informed consent moved from the lightest baseline burden to the second heaviest residual burden. Conclusions Near the instrument ceiling, percentage score gain largely reflects baseline room for improvement and should not be used alone to compare training effects across documentation domains. Normalized gain depends less on baseline, but dimension ranks are imprecise and are better reported with uncertainty intervals and in tiers. Targets for the next training cycle should be set from the residual defect burden re-measured after each cycle.

Research Square
Third People's Hospital of Chengdu (CN)
Good health and well-being, Quality education
Medical Coding and Health Information
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon