Effective Evaluation and Unintended Consequences of Stem Interventions: A Case Study from North East England

The evaluation of many STEM (Science, Technology, Engineering and Mathematics) interventions focuses on planned outcomes. However, the impact of unintended consequences from interventions remains largely unexplored in the literature. This case study presents the design and evaluation of a school-based workshop about an engineering-adjacent career: environmental planning. The authors used a community of inquiry action research approach focusing on knowledge and aspirational outcomes from outreach, particularly around broadening children's knowledge and understanding of careers. The developed evaluation tools allowed identification of intended and unintended consequences from the intervention. Participants were children aged 10-11 years (<tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathrm{n}=83$</tex>) from two schools in North East England. Results showed an increase in knowledge of environmental planning. The qualitative data indicate unanticipated consequences due to children's temporal misunderstanding of timescales involved in coastal changes. Findings are discussed in the context of 'lessons learned' and highlight the importance for engineering and technology outreach interventions of incorporating a well-designed evaluation strategy and feedback mechanism to identify and mitigate unintended and unanticipated consequences.