A Hybrid Approach for Usability Evaluation of Learning Management Systems Using Machine Learning Algorithms

This research full paper describes a hybrid approach based on data, questionnaire answers, and machine learning algorithms to predict usability scores in Learning Management Systems (LMSs) to improve student learning and satisfaction. Students need to achieve their learning goals by interacting with LMSs. To attain these goals, usability evaluation ensures effectiveness (task completion), efficiency (time measurement), and satisfaction (positive attitude). Usability evaluation usually follows questionnaires, user testing of the LMS, and expert reviews. Although these methods are widely used due to several benefits, they face challenges related to trying these software systems multiple times until the system satisfies student needs, human subjectivity perception, and lack of software system adaptability. We propose this hybrid approach to face these challenges, promote student engagement with the system, and create a better design in the LMS courses. The aim is to identify features extracted from the LMS to predict usability scores with machine learning techniques. We evaluated this strategy through a case study with data collected from undergraduate students at a public university in the United States. The students' tasks were answering a quiz, posting in a forum, and uploading an assignment. These activities in the LMS allow the extraction of ten features into the machine learning algorithms. These attributes are time quiz, time forum, time assignment, grade quiz, word count message post, file size, file type, clicks module quiz, clicks module forum, and clicks module assignment. The four targets are from scores of the System Usability Scale and UseLearn questionnaires. Random Forest produces the best performance of average mean square error and root mean square error among machine learning algorithms. The results are promising, though there are alternatives for improvements. Our proposed approach contributes to the engineering and computing education field by providing a predictive tool for usability scores to improve the student learning experience and the components of the LMS.