Measuring naturalistic speech comprehension in real time

Speech comprehension has been described as an effortless and robust process; yet, in real-world contexts, it is common for a listener to misunderstand what was said or fail to derive meaning entirely. Typically, methods of measuring speech comprehension are applied 'post hoc' - that is, after the comprehension has happened. This approach fails to capture comprehension as it occurs, limiting the field's understanding of the cognitive processes involved in real-time comprehension. To overcome these challenges, we designed and tested a novel method of measuring real-time speech comprehension during naturalistic listening. We built a slider device that synchronizes with experimental software and provides millisecond read-out. In three experiments, participants listened to audiobook segments while providing continuous comprehension ratings using the slider. To vary comprehension success, we presented speech segments at speed factors of 1-5 times faster than normal. We validated the time-resolved slider data against established speech comprehension assessment methods. Overall, our findings validate our novel time-resolved comprehension measure and demonstrate that it is possible to derive an online behavioral measure of real-time speech comprehension. We also confirmed numerous limitations of static post hoc assessments, including challenges with multiple-choice question design and the confounding of potential effects due to recency bias and comprehension for summarization. The measure proposed here overcomes the constraints of static post hoc assessments and can be effectively integrated with neuroimaging techniques, offering a valuable tool for future research on dynamic processes during naturalistic listening.