Evaluating Socio-Ecological System Performance Under Uncontrollable Environmental Risks: Implications for Subnational Evaluation Practice

Performance evaluation increasingly informs policy learning and accountability in complex socio-ecological systems (SES). However, when efficiency-based tools are applied to SES, evaluation outcomes are shaped not only by observed performance but also by evaluation-design assumptions. In particular, whether environmental shocks are treated as controllable or uncontrollable influences judgments of performance and responsibility. This study examines how alternative treatments of environmental risk affect SES evaluation outcomes. Using Indonesian provinces as a disaster-prone, environmentally heterogeneous case, we compare a conventional CCR DEA model, a super-efficiency data envelopment analysis (DEA) model, and an enhanced super-efficiency DEA model treating floods and landslides as uncontrollable outputs. The results show that treating environmental risks as uncontrollable substantially changes performance assessments, rankings, and responsibility attribution. By reframing SES efficiency analysis as an evaluation-design problem rather than a technical exercise, this study demonstrates how controllability assumptions shape evaluation outcomes and provides transferable insights for designing fair, transparent evaluations in complex SES.