Eyes on the ball: Teachers’ rating processes when assessing a national L2 speaking test through the lens of their scoring document

The administration of national second language (L2) tests in schools has increased during the last decades. Speaking is often missing from tests, possibly because it is particularly challenging to assess in a reliable and standardized way, not least when communicative competence is part of the test construct. This study examines how teachers attend to the challenges of L2 speaking assessment, by studying why and how scoring documents are used in the rating process of a national L2 English speaking test in Sweden. Data consist of stimulated recall interviews with 13 secondary-school teachers. The method of analysis was qualitative thematic analysis guided by the Anthropological Theory of the Didactic (ATD, Chevallard, 2007). Findings show that teachers reported on a three-step rating process, where scoring documents were used in relation to the purpose of each step. In Step One, the documents were regarded as beneficial for a focus on relevant criteria and for quick notetaking. In Step Two, students’ spoken performances were reflected in notes to which teachers referred to analytically decide the score. A third step was added after the score was determined, when notes were passed on to stakeholders. Analysis of interview data indicates washback effects from the test on teaching, and illuminates teachers’ assessment responsibilities in a system increasingly affected by accountability.