Solaria-GPT: A Tailored ChatGPT Tool for Usability Inspection

Usability defects in software systems result in challenges for users during their interactions with the software. To address these challenges, usability inspection is key for detecting defects during software development, allowing them to be fixed before the defects reach end users. It also plays a critical role after software implementation, during the software maintenance phase. This paper presents Solaria-GPT, a tool based on a tailored version of ChatGPT, designed to assist software engineers in identifying and classifying usability defects following Nielsen’s heuristics. Solaria-GPT facilitates two interactions: the detection of usability defects in user interfaces and the classification of each defect based on the violated heuristic. To evaluate the tool, we conducted a study to assess the performance of Solaria-GPT across textual and media-based inputs (screenshots and videos), focusing on its accuracy rate (correct heuristic violation), utility rate (valid defect identified by Solaria-GPT), and new defect rate (new defect identified by Solaria-GPT). The results indicated a 96.23% accuracy rate in heuristic classification from textual inputs. For utility rate and new defect rate, the Solaria-GPT achieved 86.67% and 86.67%, respectively, for screenshots, and 16.13% and 87.10%, respectively, for videos. Comparisons with other large language models (Claude, Qwen, Gemini, and Deepseek) demonstrated that Solaria-GPT outperformed all alternatives across metrics. These findings suggest that Solaria-GPT is a promising tool for enhancing software usability during the software development lifecycle. Demo Video: https://doi.org/10.5281/zenodo.15275798