Designing causal evaluations of analytics-prompted interventions in higher education: three case studies of interventions designed to improve wellbeing

Learning analytics systems are increasingly used in higher education to monitor student engagement and identify those at risk of poor wellbeing. However, there is limited causal evidence on the effectiveness of student support interventions prompted by analytics. Drawing on three case studies from universities in England, this paper documents how randomised controlled trials can be feasibly designed and implemented within university settings to rigorously assess analytics-prompted interventions. It highlights some key considerations that should be reflected in the design of causal evaluations of interventions prompted by analytics, such as developing theories of change to practically inform the selection of outcomes. It also discusses the suitability of different trial designs, depending on whether new or existing interventions are being evaluated, evaluation pre-registration, and the value of complementing statistical studies of intervention’s effects with implementation and process evaluations.