Reevaluating the Balance Between Overtreatment and Misdiagnosis

We thank the commentators for their interest in our article. In our study,1 we examined the empirical question of whether currently available screening tests are sufficiently accurate to replace or complement the gold standard. Methods to answer such diagnostic questions are well established2 and rely on comparing the accuracy of available tests to the gold standard (ie, urine culture) in an unselected large sample of children being tested for UTIs. We conducted exactly such a study in 3 large children’s hospitals, including consecutive children being tested for a UTI. Our results align well with findings from other studies. Indeed, nearly 100 studies have demonstrated that tests for pyuria, on average, fail to detect ∼20% of children who are later shown to have a positive culture.3 Most of these studies predate the publication of the 2011 Guidelines, which, for the first time, mandated the presence of pyuria for the diagnosis of UTI.4Dr Alverson and colleagues raised concerns about the inclusion of children without fever (ie, documented temperature of >38 °C). We included children without fever in an effort to evaluate available screening tests in the population in which they are used, rather than in a population that might artificially enhance their performance. We do, however, include data pertaining to children with fever in the lower part of each table in the manuscript. These data demonstrate that among the 3377 children with fever, ∼20% of children with a positive culture would have been missed by currently available tests for pyuria, which is essentially the same as when we include all children being evaluated for UTI regardless of fever. Thus, we disagree with the assertion that “the bulk of the reported false negatives” stems from the inclusion of children without fever.Dr Alverson and colleagues raised concerns regarding our use of sensitivity and specificity values instead of negative and positive predictive values (NPVs and PPVs). All accuracy metrics are equally valid because they derive from the same 2x2 table, for which we provided raw numbers. We chose to present sensitivity values because, for relatively rare conditions, NPVs can obscure a test’s inadequacies. For example, for a condition with a prevalence of 10%, even a completely uninformative test would yield a seemingly high NPV of 90%.Available data shows the sensitivity of current tests is suboptimal. The commentators propose two solutions: combining tests or using lower cutoffs. To examine the first suggestion, we combined the two most widely available tests—leukocyte esterase test “or” ≥5 WBC/hpf per digital imaging. Among febrile children, this combination had a sensitivity of 0.87 (95% CI, 0.81,0.94) and a specificity and 0.85 (0.82, 0.87). The sensitivity of the combination is only slightly higher than the leukocyte esterase test alone (ie, in a population of 1000 febrile children, only 3 extra UTIs would be detected). Importantly, the combined test has a substantially lower specificity than the leukocyte esterase test alone (ie, 90 extra children without UTIs would receive unnecessary antibiotics). Similarly, while lowering test cutoffs will increase sensitivity, this will be accompanied by a decrease in specificity. The goal of the AAP UTI guidelines in requiring pyuria for UTI diagnosis was to decrease overtreatment of asymptomatic bacteriuria and contaminated samples. As the preceding analysis indicates, these suggestions would decrease the specificity of available screening tests, thus increasing overtreatment. In fact, lowering specificity will likely result in a greater degree of overtreatment than removing the recommendation that pyuria must be present.Two commentators suggest that, even though catheterization was used to collect all samples, contamination still explains the bulk of our findings. Recall we defined a positive urine culture as growth of a single uropathogen with at least 50 000 CFU/ml. It is theoretically possible that contamination could result in cultures with high colony counts of a single pathogen. If a substantial proportion of children enrolled had this form of contamination, we could have underestimated the sensitivity of the screening tests being evaluated. Inclusion of children with other forms of contamination (mixed or low colony count cultures) would not have affected the reported sensitivity values. How often does this form of contamination occur? To our knowledge, only 2 studies have examined this question. In these 2 studies,5,6 Pryles collected concurrent samples using catheterization and suprapubic aspiration (SPA) from 92 children. Regardless of collection method, no child had growth of a pure colony of a single pathogen on culture. One boy with severe phimosis had growth of two organisms on culture from the first few drops of urine collected. Even if this boy is considered to have a positive culture, the rate of contamination would be 0.5% (assuming that of 28% of infants being evaluated for UTI are males). Dr Wald cites a study that evaluated the frequency of contamination defined by mixed growth or low colony counts, which is not relevant here as these samples did not meet our criteria for a positive culture.7 Two commentators cite a study by Eliacik8; unlike the studies by Pryles, however, collection of catheterized and SPA samples was separated by 3 or 4 days. This major source of bias likely explains why this study found that 71% of all catheterized samples were “contaminated”. Finally, contamination cannot not explain why other tests for pyuria (eg, Neutrophil Gelatinase Associated Lipocalin, which is not widely available) perform so much better than the tests evaluated in our study. In summary, contamination does not explain why current tests for pyuria perform poorly.Two commentators cite a study that examined the prognosis of 36 untreated children with positive cultures who did not have pyuria.9 Some of these 36 children had 2 pathogens on culture or had a urine specimen collected using methods other than catheterization, neither of which would have qualified as a positive culture in our study. In addition, prognosis was determined using a retrospective chart review of notes, which is less accurate than prospective methods. For these two reasons, the data from this study is limited and unlikely answers the question of prognosis.Two commentators point out that tests for pyuria perform well in subgroups of children with bacteremia or pyelonephritis.10,11 We acknowledge that screening tests often perform better in severely affected subgroups. In clinical practice, however, we often do not know if a child is bacteremic or has pyelonephritis as blood cultures and renal scans are not commonplace for routine UTI evaluation. Furthermore, the majority of children with UTI will not have bacteremia or pyelonephritis. Screening tests for UTI must be appropriate for all children being evaluated for UTI, not just those that are the most severely affected.In her commentary, Dr Wald describes a familiar diagnostic workflow. But the workflow portrayed corresponds to the state of affairs before the 2011 Guidelines took effect. In that workflow, the clinician would integrate clinical information with information about pyuria to make case-by-case decisions regarding empirical treatment. A few days later culture results would provide additional information that would be used to further individualize care. In the current workflow (algorithm used at Children’s Hospital of Pittsburgh available by request), except for immunocompromised children or those in the ICU, hospitalists can only order one test: urinalysis with reflex to culture. Thus, if the UA does not meet the predefined threshold for pyuria, a urine culture is not performed, eliminating the clinician’s ability to integrate urine culture results into their medical decision making. We would also like to highlight that the cutoff of 10 WBC per high powered filed, which is used at several institutions, has limited pediatric evidence. Based on our data, a single cutoff used across institutions with different methods of measuring pyuria is inaccurate.Whenever changes in clinical practice are made, it is important to explore unintended consequences. At a public health level, we estimate that requiring pyuria leads to close to 14 000 missed cases of UTIs annually.12The current approach overlooks advances in our understanding of UTI biology. Recent studies have shown the degree of pyuria differs according to the infecting organism.13 Enterococci, for example, can actively inhibit host response.14 It is also well known that current tests for pyuria can be absent early during an infection. These two populations likely represent some of the missed UTIs we reported.While the sensitivity of trace leukocyte esterase did seem high in our data set, we cannot endorse it because (1) this finding has not been replicated in other studies, (2) its accuracy might be lower in outpatient settings where automation is not generally available and most UTIs are diagnosed, and (3) its performance in unselected samples was much lower (see tables in supplement).We agree with the commentators that pyuria is a useful marker for UTIs, and we acknowledge the constant struggle in medicine to balance overtreatment with missed diagnoses. However, we maintain that missing 1 in 5 true UTIs is likely more harmful than overtreating asymptomatic bacteriuria and contamination, which has previously been reported at rates much lower than 20%. Finally, we acknowledge the need for more prognostic data to complement the available diagnostic data.