Lessons Learned from Usability Evaluation with a Low Literacy Population

Developing accessibility solutions for functional illiterates (FXI) is challenging. In a laboratory experiment, we compared the effectiveness of two guide format (paper and video), with a no-guide condition, at helping FXI complete mobile banking tasks. Eleven FXI participants performed three tasks, and both objective performance and self-reported measures were collected. Our findings show that while task performance was similar across conditions, participants showed a slight preference for the paper guide. Additionally, task familiarity emerged as a factor influencing the performance of the participants. The study also revealed several methodological considerations for future work: simplifying the experimental design, adding control variables to account for the heterogeneity of the FXI population, and promoting participant comfort through increased familiarity and human presence. This study advances research in digital accessibility for FXI and offers methodological recommendations for future work.