Beyond monologues – Examining L2 Finnish learners’ interpersonal behavior during dialogue speaking tasks using the computer-joystick method

Although second language (L2) speaking assessment usually seeks to cover interactional aspects, in practice, it is often limited to assessing L2 learners' individual audio-recorded performance or focusing merely on linguistic criteria. This study experiments with a new method for assessing interactional competence. The computer-joystick method allows observing both verbal and nonverbal features while rating an individual's behavior from video. The present study seeks to find out whether the method can be used to assess speakers' interpersonal behaviors reliably in the context of L2 learners' dialogue speaking tasks. The interpersonal behavior of academic L2 Finnish learners (N = 44) was examined during three spoken interaction tasks designed for A2–B2 level speakers. The study is part of a larger project which develops ways to measure and assess oral language skills automatically. Each learner's interpersonal behaviors (using two variables of dominance and affiliation, rating both linguistic and nonverbal features) were continually coded over time while watching audiovisual recordings of their speaking performance. The reliability of the ratings ranged from moderate to good. The resulting time series data were used to analyze changes in the variables and their variation according to task-level, dyad-level and participant-level predictors. Nonindependence metrics, actor and partner effects of language proficiency, and measures of synchronization between participants were investigated. In sum, it was found that the coding system can be used reliably for assessing interpersonal behaviors in the context of language assessment. Finally, potential implications for assessing spoken interaction in a second language using both humans and algorithms are discussed. • Joystick coding was used to rate interpersonal behaviors from video recordings. • Video recordings enable assessment of nonverbal and verbal features. • We analyzed language learners' dominance and affiliation in dialogue performances. • Time series data enables changes in the variables and their interplay to be analyzed. • The findings can be used to assess spoken interaction between humans and algorithms.