Large Language Model-based Method for Mobile App Accessibility Enhancement

Mobile application accessibility refers to the degree to which mobile applications are designed and implemented to ensure that any user can easily access the application.However,only a small fraction of the vast number of applications in the domestic mobile application market support accessibility features,which contradicts to the vision of breaking the digital divide and enjoying the benefits of the digital age for the growing elderly and visually impaired population.Large language models(LLMs) have demonstrated significant potential for achieving human-level intelligence.Through prompts guidance,they can engage in simple logical reasoning and decision-making.In addition,shortening the interactive pathway is an intuitive strategy for enhancing mobile application accessibility.Inspired by the aforementioned facts,we propose an innovative method for enhancing mobile application accessibility based on LLMs.This method creatively applies accessibility services and LLMs,aiming to improve security,automation,and intelligence.We have implemented a mobile application accessibility tool called AccessLink.Under the premise of non-invasiveness and user authorization,AccessLink perceives and interacts with the graphical user interface of mobile applications.Additionally,we have developed a dataset construction approach based on automated methods.Experimental validation is conducted using the constructed dataset with large models such as GPT-3.5,GPT-4.0,QianWen and Baichuan,demonstrating the effectiveness of the proposed method.