Assessment of Task-Based Pragmatic Performance: Measuring Pragmatic Complexity and Accuracy

Assessing pragmatic performance is crucial in second language (L2) pedagogy, and it has recently gained attention within task-based research. Current studies commonly employ holistic or analytic rating scales to evaluate pragmatic performance, yet the reliability and interpretive consistency of such ratings remain debatable. This study proposes a refined approach by distinguishing between dimension-specific holistic ratings and quantitative index-based measures. It also introduces the concepts of pragmatic complexity and pragmatic accuracy to examine task-based pragmatic outcomes. Thirty-seven Czech L2 learners of English completed a simple and a complex decision-making task designed to elicit suggestion speech acts. In line with the Cognition Hypothesis, these tasks varied in cognitive complexity as reflected in the number of elements and reasoning demands. Raters used dimension-specific holistic rubrics to provide scores for pragmatic accuracy and complexity. Complementary indices—ratios of suggestion types and inaccuracies relative to total suggestions—yielded empirically derived indicators of pragmatic performance. Findings indicate that fine-grained, analytic ratings can capture aspects of pragmatic performance that more global assessments may overlook. Moreover, increased cognitive task demands may direct learners’ attentional resources in ways that facilitate improved pragmatic performance. The study concludes with implications for task design and multi-method approaches to L2 pragmatic assessment.