Fairness Implications of Unmodeled Reading Skills in Adaptive Learning Systems

Students' reading ability affects their outcomes in learning software even outside of reading education (such as in math education) which can result in unexpected and inequitable outcomes. We analyze an adaptive learning software using Bayesian Knowledge Tracing (BKT) to understand how the fairness of the software is impacted when reading ability is not modeled. We tested BKT model fairness by comparing two years of data from 8,549 students who were classified as either "emerging" or "non-emerging" readers (i.e., a measure of reading ability). While BKT was unbiased on average in terms of equal predictive accuracy across groups, specific skills within the adaptive learning software exhibited bias related to reading level. Additionally, there were differences between the first-answer mastery rates of the emerging and non-emerging readers (M = .687 and M = .776, difference 95% CI = [0.075, 0.095]), indicating that emerging reader status is predictive of mastery. We then conducted an intersectional fairness analysis of emerging reader status and traditional demographic groups. We found that students of different demographics, when considering emerging reader status, have varying rates of mastery of up to 10%. We demonstrate significant group differences in BKT models regarding reading ability, exhibiting that it is important to consider--and perhaps even model--reading as a separate skill that differentially influences students' outcomes.