GUIA uditor : Enabling Post-hoc Child Safety Forensics via Action-Guided GUI Provenance on Mobile Devices
The proliferation of smart devices exposes children to online risks like grooming and financial scams that are deeply embedded within legitimate applications. Current approaches rely on automated prevention and detection, a paradigm that is fundamentally limited by its inherent fallibility. Whether rule-based or AI-driven, they inevitably produce false positives and negatives, failing to provide reliable protection. In this paper, we argue for a complementary, human-in-the-loop, post-hoc forensic paradigm. We present GUIA uditor , the first system designed to realize this vision by creating GUI Provenance: a queryable, semantic record of a child's interaction sequence. To generate this, GUIA uditor leverages a Multimodal Large Language Model (MLLM) to translate the temporal sequence of GUI events into a human-understandable narrative. To make this practical on mobile devices, a novel evidence distillation pipeline reduces the data requiring analysis by over 89.2% compared to periodic sampling approaches adopted by industry standards, with negligible impact on accuracy. On a new dataset of 295 interaction clips, GUIA uditor achieves a 95.23% Macro-F1 Score in logging significant events and, crucially, its two-stage forensic query engine successfully retrieves the correct evidence as the top result for over 90.20% of natural language questions. An end-to-end evaluation on three modern smartphones shows that the full pipeline, including on-device MLLM inference, adds 2.1W of power draw and 7.4s of per-event latency, with a peak memory footprint of ~3.1GB. These results show that post-hoc GUI forensics can run on modern mobile devices and provide useful context for guardian-led safety review.
Authors
- Shaofei Li (ORCID: https://orcid.org/0009-0001-6530-5935)
- Yao Guo (ORCID: https://orcid.org/0000-0001-5064-5286)
- Ding Li (ORCID: https://orcid.org/0000-0001-7558-9137)
- Yifeng Cai (ORCID: https://orcid.org/0000-0002-0049-6670)
- Junlin Liu (ORCID: https://orcid.org/0000-0001-5917-2251)
- Zhineng Zhong (ORCID: https://orcid.org/0009-0007-2490-5671)
- Xiangqun Chen
- Jiacheng Liu
- Shuai Wang
- Yuanchun Li
- Ziqi Zhang
Institutions
- University of Illinois Urbana-Champaign (US)
- Peking University (CN)
- Tsinghua University (CN)
Publication Details
- Journal
- Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies
- Published
- 2026-09-30
- DOI
- https://doi.org/10.1145/3831978
- Primary Topic
- Digital and Cyber Forensics
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- National Natural Science Foundation of China
- Ministry of Education of the People's Republic of China