Measure letter recognition performance: A subjective evaluation method

Yu & Watson (2022) found that letter recognition performance can be estimated using a guided subjective self-report, and there was a high correlation between the subjective estimation and the performance-based (objective) measure obtained independently with psychophysical method. However, in their study, the objective and the subjective data were collected from different observers, and the correlation between measurements was only assessed at the group level. Here we are examining whether similar results can be observed at the individual level. Nine normally sighted young adults completed five testing blocks (four objective blocks and one subjective block). Target stimuli were single lowercase letters presented at 10° eccentricity left and right of the fixation. In the objective blocks, observers performed a letter recognition task following each letter presentation (10 trials/letter/block). In the subjective block, observers viewed each target letter ten times in a row, and then reported all the letters that they perceived the target to be and assigned a percentage value for each reported letter (1 trial per letter). Consistent with the findings from Yu & Watson (2022), there was a good agreement between subjective (81%) and objective measurement (85%) for overall letter recognition accuracy, and the correlation between subjective and objective measure was significant for recognition accuracy of individual letter (r=0.74, p<0.001) and distribution of letter confusion (r=0.78, p<0.001). When examining individual observers, we found significant correlations between subjective and objective measure for overall letter recognition accuracy (r=0.65, p=0.03), recognition accuracy of individual letter (r=0.36 to 0.72, ps≤0.03), and distribution of letter confusion (except one observer, r=0.24 to 0.65, ps≤0.04). All tests were one-tailed tests. Our results showed that the subjective evaluation method can provide estimation on letter recognition performance with much fewer trials and has potential for serving as a viable alternative to performance-based evaluation.