Real-Time Medical Record Quality Control System (MRQCS): Development and Retrospective Longitudinal Observational Study

BACKGROUND: Medical record quality is a key focus of health care quality management. Traditional manual medical record quality control (QC) suffers from low efficiency, inability to achieve full-sample coverage, and poor consistency. Existing intelligent QC systems have significant deficiencies in real-time intervention and quantitative assessment of application effects. OBJECTIVE: This study aimed to describe the design and development of the medical record quality control system (MRQCS) and to retrospectively evaluate its associations with medical record defect detection, medical record quality, and interreviewer consistency using 7-year longitudinal final-stage QC records (2019-2025). METHODS: MRQCS implements a 2-level QC process for clinicians: nonblocking real-time prompting during documentation and blocking validation upon medical record completion. For QC reviewers, it provides AI-assisted defect detection and a 1-click write-back function. QC records from January 2019 to December 2025 were analyzed as 84 monthly discharge cohorts (43,324 sampled admissions), using interrupted time series (ITS) segmented regression with breakpoints aligned to the first discharge cohorts affected by each intervention (December 2020, July 2023, and May 2025). A stable‑rule‑set analysis was performed for genuine quality improvement. Linear mixed‑effects modeling with intraclass correlation coefficients (ICC) was applied to assess reviewer consistency, alongside multiple sensitivity analyses. RESULTS: By 2025, the MRQCS QC rule library expanded from 41 rules initially to 597 rules. The sampled-case defect rate increased sharply after the January 2021 launch (trend change +0.06, 95% CI 0.03-0.08, P<.001), and rrates based on the 41 stable rules present in every year rose from 16.7% (880/5256) in 2021 to 50.9% (3662/7190) in 2023, indicating that most of the increase reflected improved system sensitivity. The high-risk case rate declined significantly after July 2023 (trend change -0.09, 95% CI -0.15 to -0.02, P=.007), and the residual defect rate showed an immediate decline at the May 2025 breakpoint (level change -0.44, 95% CI -0.85 to -0.03, P=.03). The AI-assisted write-back rate exceeded 97% after August 2023 and declined to 84%-87% after the May 2025 discharge blocking intervention (level change -2.14, 95% CI -3.08 to -1.21, P<.001), suggesting that clinicians corrected an increasing proportion of defects. Reviewer-level defect rates were far more homogeneous in the AI-assisted period (coefficient of variation [CV] 26.8% vs 115.7%), with reviewer identity accounting for a smaller share of the total variance (ICC 0.317 vs 0.782), indicating greater interreviewer consistency. Consistency of the high-risk defect rate showed the same pattern (CV 31.3% vs 135.3%; ICC 0.053 vs 0.541). CONCLUSIONS: Through the collaborative model of 2-level real-time QC and AI-assisted final-stage QC, MRQCS was associated with improved defect detection, reduction of documentation defects, and substantially greater interreviewer consistency; the discharge-time blocking intervention further reduced the residual defect rate. These findings provide a practical reference for the construction of medical record quality management systems.