Identifying the Information Gap for Visually Impaired Students during Lecture Talks

Visual slides have become a common tool in university classrooms, assisting students in following along with lectures. However, visually impaired individuals face an information gap when visual slides are the primary lecture material. This paper aims to identify which information is most likely to be withheld from visually impaired individuals. To analyse this information gap, the study first extracts textual elements from lecture slides and converts visual elements into textual descriptions on multiple levels that discriminate between different semantic depths of visuals. This information is compared to the transcribed audio of the lecture talk by a semantic similarity analysis. The study tests how a transformer model can be used for an automatic semantic similarity analysis, which is validated through human evaluation. The results indicate variations in the level of detail and semantic depth with which visuals are described across different lectures. As shown, direct references to visuals and elemental details of visuals are often omitted in speech. However, this information is important for enabling mental visualization processes for visually impaired individuals and for following along with the lecture. The identified information gap can serve as valuable feedback for lecturers and assistive systems, helping them to more effectively provide visually impaired students with the information that they may have otherwise missed during the lecture.