Agreement and Alignment in Binary Rating Tasks: Strategic Convergence as an Equilibrium Outcome

Agreement between raters does not, by itself, show how that agreement is produced. In binary judgement tasks, raters may classify cases similarly either because they rely on similar decision thresholds or because those thresholds operate under favourable marginal conditions. The present study examined whether convergence in raters’ decision thresholds must be imposed exogenously or can emerge endogenously through strategic adjustment. Within an equal-variance signal-detection framework, raters were modelled as choosing thresholds that balance expected classification accuracy against an incentive to align with the rest of the panel. Equilibrium thresholds were obtained by Nash best-response updating, and threshold variance, the Strategic Convergence Index (SCI), and mean pairwise Cohen’s <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" overflow="scroll"> <mml:mrow> <mml:mi>κ</mml:mi> </mml:mrow> </mml:math> were examined as functions of alignment pressure and prevalence. Increasing alignment pressure produced a sharp collapse in threshold dispersion and drove SCI towards 1 under the chosen normalisation, whereas <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" overflow="scroll"> <mml:mrow> <mml:mi>κ</mml:mi> </mml:mrow> </mml:math> increased only within a comparatively narrow range. When alignment pressure was held constant, SCI remained effectively unchanged across prevalence conditions, while <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" overflow="scroll"> <mml:mrow> <mml:mi>κ</mml:mi> </mml:mrow> </mml:math> varied substantially. These findings indicate that convergence in decision thresholds can arise as an equilibrium outcome of strategic interaction yet remains analytically distinct from observable agreement. They also clarify the status of Cohen’s <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" overflow="scroll"> <mml:mrow> <mml:mi>κ</mml:mi> </mml:mrow> </mml:math> in this setting: it tracks agreement at the level of realised classifications and therefore remains sensitive to marginal conditions, whereas SCI captures convergence in the latent decision structure from which those classifications arise. SCI is thus not proposed as a replacement for agreement coefficients but as a model-based quantity that makes explicit a form of alignment that <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" overflow="scroll"> <mml:mrow> <mml:mi>κ</mml:mi> </mml:mrow> </mml:math> only partially reflects.