Deep-Learning-Based Mobile Application for Real-Time Recognition of Cultural Artifacts in Museum Environments

Dissemination and conservation of cultural heritage have been challenged by continued accessibility in museums, where traditional information delivery systems are at times ineffective in terms if interaction with visitors. The current paper investigates RumiArt IA, a mobile application, to identify cultural objects in real-time, remaining fully in the scope of this line of research without relying on internet connectivity. The system, which is developed based on the Rumiñahui Museum and Cultural Center, Ecuador, uses transfer learning in the MobileNetV2 architecture with INT8 post-training quantization to identify 21 cultural artifacts spread across six thematic rooms. The experiment involved building a dataset of 36,000 images under diverse lighting conditions, viewing angles, and distances; furthermore, artificial transformations were explicitly crafted to simulate real museum conditions such as glass reflections and non-frontal capture angles. Quantization was used to reduce each model to 775 KB as compared with the 2.4 MB, with accuracy loss not reaching more than 0.5 percent (DKL < 0.05). Assessment of 9450 validation images yielded a general accuracy of 92.2%, with an inference time of 63 ms on current devices with a high throughput and 215 ms on mid-range hardware from 2020. Practical validation involving 50 visitors of the museum showed a success rate of 93.7%, with average user satisfaction at 8.5/10 and 87%, indicating they would recommend the application. An in-depth error study of the most difficult room (88.3% accuracy) indicated that 47% of the errors were due to the angles of the camera, which blocked out distinguishing features, and 22% were caused by display case reflections and the shadows of the visitors. These results indicate that end-to-end machine learning can provide consistent cultural heritage recognition in resource-constrained settings but its efficiency is susceptible to physical capture factors that cannot be resolved by data augmentation. Offline mode and low memory footprint (less than 90 MB when loaded on six models) of the system are especially relevant to application in situations where there is no guarantee of cloud connectivity.