A novel approach to measuring emergency physician efficiency

In health care literature, data envelopment analysis (DEA) is a commonly used method for assessing efficiency.1 The aim of a DEA model is to determine how efficiently a decision-making unit (hospital, medical unit, physician) can turn inputs (staff time, equipment, facilities) into outputs (quality of care, throughput, cost reduction). DEA models then provide an objective measurement of that input to output conversion as an efficiency score. A variable returns to scale (VRS) DEA model allows efficiencies to be estimated when a there is no consistent proportional change with increases or decreases in input and output measures which can occur in healthcare. In recent years the application of DEA in hospitals has increased in popularity and is being used in a variety of ways, from helping to perform benchmarking analysis on specific services2 to measuring the overall efficiency of hospitals.3 There are many reasons DEA is apropos for use in health care. DEA can account for the many unique features of the health care system in ways that parametric efficiency scores cannot. The nonparametric nature of DEA allows for the investigation of how multiple inputs and outputs impact productivity, which addresses the multitude of complex factors affecting efficiency in health care.4 The information provided by a DEA analysis can also indicate relative efficiencies of the decision-making units rather than the average efficiencies.5 Through analyzing relative efficiencies of the decision-making units operations leaders can determine which specific inputs need optimization. In addition to evaluating relative efficiencies of larger decision-making units in health care, DEA can also be used to assess an individual physician's efficiency. There is an assumption that how fast a provider completes a task (efficiency) is associated with their productivity (outcome of task completion). DEA can be helpful to identify an optimal performance frontier (benchmark of the best-performing physicians) and is helpful in its ability to proficiently oversee numerous resources and monitor results throughout the change process.3 A systematic review of the literature around measuring emergency medicine provider productivity found that most studies measured productivity using patients per hour (PPHR), relative value units per hour, or provider-to-disposition time. These measures factor in common elements of productivity such as patient volume, complexity, and processing time.6 They do not, however, include operational barriers that physicians do not have control over such as scheduling, crowding, or higher-level hospital practices and are not standardized across institutions. These shortcomings are rarely taken into consideration despite the increasing focus of administrators when determining compensation.7 If productivity is being used to determine physician compensation, it is important to ensure efficiency mitigate the effects of operational barriers on productivity measures. Accurate measurements of physician productivity based on variables physicians have the power to manipulate could result in an enhanced perception of fair pay8. This study's goal was to propose a novel measure of physician efficiency that incorporates physician behavior metrics. This was a retrospective analysis evaluating attending emergency physicians from a single academic emergency physician group who worked at least one shift at one of two hospitals over a 1-month period (November 2022). Data related to the work by the attending physicians were included such as number of shifts worked, time of day of each shift worked, whether the physician worked with residents, type of shift worked (critical care or low acuity), patients seen, patients admitted, and time from door to disposition. For the development of the DEA model, the physicians were designated as the decision-making units (DMUs). The input variables included the room to discharge, room to admission, and percentage of patients admitted. The output variable was PPHR. A VRS model was used to estimate efficiencies in the environment: how efficiently a physician transformed a decrease in input variables into an increase in PPHR. Efficiency scores were interpreted on a scale from 0 to 1, with 1 representing full efficiency. Additionally, a linear regression was performed on variables outside of physician control to determine if these are significantly associated with PPHR, including the time of the shift, whether the provider worked with residents, and the type of the shift (critical care). The VRS score was also included in the model to determine if it was independently associated with PPHR. Multicollinearity was assessed by variance inflation factor (VIF) values after executing each multiple linear regression model for each urban hospital and university site. A VIF of <5 is considered as an absence of potential multicollinearity. Kendall's rank correlation was calculated for VRS scores in physicians working at both facilities. All calculations were done using Stata 17.0 MP parallel edition (Stata Corp). This study was approved by the institutional review board as exempt. A DEA model was constructed for the large urban hospital with 80 of the physicians participating in the study. Of these 80, a total of 39 worked at both the urban and the academic center. The VRS scores of physicians ranged from 0.48 to 1 with a median score was 0.77. The linear regression analysis showed that percentage of night shifts and critical care shifts worked as well as the VRS score were significantly associated with PPHR. Higher VRS scores were associated with a 0.68-unit increase in PPHR (95% confidence interval [CI] 0.08–1.28, p = 0.03), and working a night shift was associated with a 0.01-unit increase in PPHR (95% CI 0.005–0.01, p < 0.001). In contrast, working a critical care shift was associated with a 0.01 unit decrease in PPRH (95% CI −0.02 to −0.007; p = 0.00). The model explained 38% of the variance in PPHR. At the university hospital, 61 physicians were included in the construction of the DEA model. VRS scores ranged from 0.51 to 1. The median (IQR) VRS score was 0.76 (0.68–1). The linear regression showed that working a night shift, provider efficiency, type of pod, and working without residents were not associated with PPHR. We did not find any evidence of multicollinearity in the final linear regression model for each of the urban as well as university hospitals (VIF < 5). Due to such nonsignificant findings, the observed R2 was relatively smaller for the university hospital model. In those physicians who worked at both the urban and the academic hospital, there was poor correlation between VRS scores at each institution (τ = 0.11). The efficiency of frontier graph shows physicians with the most efficient combinations of input variables (Figure 1). Many studies have explored utilizing DEA to analyze efficiency at a department or hospital level, several of which have found that operational barriers have a large impact on inefficiencies of departments. These operational barriers include things like aggregate emergency department visits10 or the function of a hospital as part of a larger health care system.11 However, few studies have focused on evaluating the efficiency of individual health care providers. Of those, one found that DEA could provide valuable insights into provider efficiency and give meaningful targeted feedback on areas causing inefficiency such as ordering unnecessary imaging studies.9 The results of this study have significant implications on how the field of emergency medicine measures productivity of physicians. Neither the operational variables nor the VRS score fully account for differences in the number of patients physicians saw per hour. While there are factors that are considered universal, that is, patient volume or treatment time, other factors such as shift types can be unique to a specific institution. The benefit of DEA analysis is that it can be adjusted to incorporate both the universal and unique factors. In systems where efficiency is not an independent predictor of PPHR like the university site, basing physician compensation on PPHR will not adequately identify high performers and the use of an alternative method such as DEA may be indicated. This study was limited in that only two hospitals were analyzed over a month-long period. Time to interpretation of testing and consultation completion were unable to be quantified, so door to disposition was used as a surrogate all-inclusive time point. It was not possible to account for boarding. Given the low R2 on the university site regression model, additional factors appear to impact PPHR unique to that location which raises questions regarding the validity of that model. Thus, further work is needed, including performing the analysis over time, to assess the stability of the results in terms of efficiency. In addition, the DEA is a novel concept and physicians may not appreciate or value the results. Expansion of this investigation at hospitals with a different set of operational barriers would provide understanding of the versatility of this analysis in other environments. In conclusion, the results of this study showed that using DEA analysis may provide insight on whether physician behaviors can impact PPHR. The VRS score provided by the DEA analysis was inconsistently associated with PPHR, a common metric of clinical productivity and the score was not the only variable shown to impact physician productivity. Since many factors outside of a physician's control affect productivity the optimization of efficiency might not necessarily mean an optimization of productivity. Deborah Diercks is on the editorial board of Academic Emergency Medicine. The other authors declare no conflicts of interest. The data that support the findings of this study are available from the corresponding author upon reasonable request.