Development and validation of a fluency rating scale for Swiss German Sign Language

Introduction Sign language fluency is an area that has received very little attention within research on sign language education and assessment. Therefore, we wanted to develop and validate a rating scale of fluency for Swiss German Sign Language ( Deutschschweizerische Gebärdensprache , DSGS). Methods Different kinds of data were collected to inform the rating scale development. The data were from (1) focus group interviews with sign language teachers ( N = 3); (2) annotated DSGS data from users/learners with various levels of proficiency (i.e., deaf native signers of DSGS, hearing sign language interpreters, and beginning learners of DSGS, approximately CEFR level A1-A2) ( N = 28) who completed different signing tasks that were manipulated by preparation time; (3) feedback from raters ( N = 3); and (4) complimented with theory from spoken and sign language fluency. Results In the focus group interview, sign language teachers identified a number of fluency aspects. The annotated DSGS data were analyzed using different regression models to see how language background and preparation time for the tasks can predict aspects of fluency (e.g., number and duration of pauses). Whereas preparation time showed only a slight effect in the annotated data, language background predicted the occurrence of fluency features that also informed the scale development. The resulting rating scale consisted of six criteria, each on a six-point scale. DSGS performances ( N = 162) (same as the annotated data) from the different groups of DSGS users/learners were rated by three raters. The rated data were analyzed using multi-facet Rasch measurement. Overall, the rating scale functioned well, with each score category being modal at some point on the continuum. Results from correlation and regression analysis of the annotated data and rated DSGS performances complemented validity evidence of the rating scale. Discussion We argue that the different sources of data serve as a sound empirical basis for the operationalized “DSGS fluency construct” in the rating scale. The results of the analyses relating performance data to ratings show strong validity evidence of the second version of the rating scale. Together, the objective fluency measures explained 88% of the variance in the rating scores.