More than meets the eye: Feature concerns and suggestions in mobile XR app reviews

This study aims to address quality concerns in mobile Extended Reality (XR) apps by developing tools to classify user reviews from the Google Play Store. Our first major contribution is a holistic approach to coding thousands of reviews, offering a unified framework to assess feature concerns and feature suggestions, thereby enhancing the understanding of user expectations. We also introduce a novel toolset for extracting key terms related to these concerns and suggestions, facilitating rapid analysis and improvement of app functionalities. To validate our approach, we compare the performance of this automated coding method with sentiment analysis, deep learning-based BERT, and several large language models (LLMs). Additionally, our study extends the understanding of user satisfaction by analyzing quantitative indicators such as app ratings and review helpfulness, offering new perspectives on user perceptions of XR app quality. This research has discovered vital areas for feature improvement without implementing complex infrastructure and using advanced technical expertise, enabling app firms to better target their efforts in improving XR app experiences. • Developed a novel framework for classifying user reviews of mobile XR apps to identify feature concerns and suggestions. • Compared explainable smoke term-scoring techniques with deep learning-based BERT classification to enhance feedback analysis. • Conducted an exploratory qualitative analysis comparing XR and non-XR app feedback, revealing distinct user concerns around immersion, safety, and device limitations. • Found XR users are more likely to suggest updates to existing features and propose new safety features, highlighting the unique demands of immersive technologies. • Demonstrated how automated feedback tools can improve XR app development by streamlining issue detection and guiding feature refinement.