Capturing the impact of integrated care: challenges & opportunities from evaluation practice

The evidence underpinning integrated care remains uncertain, with numerous examples of evaluations finding little robust evidence of intended impacts on health or reporting conflicting results. Reasons for this include problems with the design and implementation of integrated care initiatives and issues with integrated care evaluations themselves, such as lack of suitable comparator or poorly defined outcome measures. In this interactive workshop we will focus on outcomes associated with integrated care, drawing on examples from our own evaluations and the experience of participants. Discussions aim to strengthen our collective understanding of how the impacts of integrated care might be better defined and measured. This workshop is intended for anyone with a role in designing integrated care initiatives or an interest in the evaluation of integrated care. Part 1: Welcome(10-mins) We will start with a brief introduction of the aims and a small number of polling questions for the audience to set the scene. Part 2: Evaluating integrated care initiatives: common challenges and what we’ve learned (40-mins) A common challenge for integrated care programmes is providing clarity on expected outcomes and how designated activities will lead to these in practice. A recent review of the last 10-years of articles within the International Journal of Integrated Care (IJIC) found that there has been little consistency in how evaluations have defined and measured the postulated impacts of integrated care. Also, few studies have co-designed the methods or the outcome measures with service users, yet improving the experience of people accessing services is often seen as a key outcome. Following an overview of the IJIC review as a starting point, we will illustrate challenges our interdisciplinary research team have faced in evaluating major integrated care initiatives, drawing on three studies in particular, which examined: (1) the governance and accountability arrangements of integrated delivery systems in different countries; (2) progress towards integration by the newly formed English Integrated Care Systems; and (3) progress towards patient-level health and care service integration using primary care-based community multi-disciplinary teams. We will also consider whether and how we could have approached the evaluations differently. This will be followed by a brief Q&A session. Part 3: Group work (20-mins) Facilitated breakout groups will consider the issues raised. Participants will be able to share experiences of commissioning integrated care programmes and/or conducting evaluations of integrated care, and to identify (routes towards) solutions. Groups will address questions such as: What outcomes are integrated care initiatives trying to change? What are the causal pathways to the intended outcomes? What innovative methods can we use to better capture intended outcomes? Are we measuring the outcomes that matter to service users, and how do we best involve them in the co-design of pertinent outcome measures? How do we better capture and account for the context in which initiatives are implemented? Part 4: Close(20-mins) Discussion will be summarised on padlet to triangulate across groups and identify common themes. A final plenary discussion will aim to describe key priorities for the evaluation com