Breaking Barriers in Mobile Accessibility: A Study of LLM-Generated Native Android Interfaces

Rapid advancements in artificial intelligence, particularly in large language models (LLMs), have opened new opportunities for automating software development tasks, including the generation of code for mobile applications. This paper examines the potential of LLMs, such as ChatGPT, to improve the accessibility of native Android apps. It evaluates whether LLM-generated code aligns with established accessibility standards, focusing on various screen layouts and prompt designs. Two studies were conducted to assess the accessibility of the generated interfaces. The first study analyzed the generation of seven types of mobile interfaces using less restrictive prompts. In contrast, the second study required the generated code to adhere to a specific interface programming model. After evaluation, 540 accessibility issues were identified, with interfaces generated using the Jetpack Compose framework showing better performance, although still in need of improvement. Interestingly, prompts explicitly requesting accessibility often produced more errors than standard prompts, highlighting the difficulties that LLMs face in comprehending and implementing accessibility requirements. This research emphasizes the need to refine LLM outputs when it comes to interface descriptions, as accessibility issues may be introduced into code intended for use by developers.