Linguistic Features Predicting Math Word Problem Readability Among Less-Skilled Readers

Solving a math word problem (MWP) requires understand- ing the mathematical components of the problem and an ability to decode the text. For some students, lower reading comprehension skills may make engagement with the math- ematical content more difficult. Readability formulas (e.g., Flesch Reading Ease) are frequently used to assess read- ing difficulty. However, MWPs are typically shorter than the texts traditional readability formulas were designed to analyze. To identify metrics relevant to assessing the read- ing difficulty of MWPs, we identified 28 candidate features which may predict MWP readability. We then assessed the performance of 297,072 middle and high school students completing word problems in an intelligent tutoring system as part of standard educational practice. From this, we identified 4,446 (out of 9,421) problems where performance gaps between predicted less- and more-skilled readers were significantly larger than typical gaps between the groups. Finally, we tested how well the readability metrics could identify problems with performance gaps. Of five models tested, a random forest had the best predictive accuracy, AUC = .75. The findings suggest readability of the text played some role in less-skilled readers decreased performance and provide a path towards better understanding how to assess the read- ability of MWPs and make them more accessible to less- skilled readers.