Disagreement Between Outpatient Complications in the USNWR ENT Ratings and Clinical Adjudication

OBJECTIVE: In 2024, the US News and World Report (USNWR)'s Best Hospitals began reporting complications following elective outpatient otolaryngology procedures in its ears, nose, and throat (ENT) ranking. However, this methodology has not been clinically adjudicated in a peer-reviewed domain. We aimed to conduct manual health record adjudication to assess whether the USNWR methodology accurately attributes complications following elective outpatient otolaryngology procedures. STUDY DESIGN: Case-cohort. SETTING: A case-cohort of n = 80 elective, outpatient otolaryngology procedure/complication pairings from January 2019 to October 2023 across four academic medical centers using the 2024 USNWR otolaryngology procedure subgroups and Solventum Ambulatory Potentially Preventable Complications (AM-PPC) software. METHODS: We used the AM-PPC-defined complication types as our standard of comparison for our clinical adjudication. Using the electronic health record, we adjudicated: (1) Was there agreement (yes/no) on complication type between our clinical adjudication and the Solventum AM-PPC software? (2) Was the complication clinically related to, or a sequela of, the index otolaryngology procedure (yes/no)? RESULTS: The clinical adjudication agreed on complication type in 59/80 (74%) of patients, and 63/80 (79%) were judged to be clinically related to the index otolaryngology procedure. In total, 50/80 (63%) complications were adjudicated to be in agreement on both the complication type and clinical relatedness to the index procedure, while 9/80 (11%) complications did not agree with the complication type nor were clinically related to the index procedure. CONCLUSION: Outpatient complications reported in the USNWR ENT rating lack the specificity to exclude patients whose complications are sequelae of chronic underlying conditions or related to concurrent, non-otolaryngology procedures.