GUI-ARP: Enhancing Grounding with Adaptive Region Perception for GUI Agents
Existing GUI grounding methods often struggle with fine-grained localization in high-resolution screenshots. To address this, we propose GUI-ARP, a novel framework that enables adaptive multi-stage inference. Equipped with the proposed Adaptive Region Perception (ARP) and Adaptive Stage Controlling (ASC), GUI-ARP dynamically exploits visual attention for cropping task-relevant regions and adapts its inference strategy, performing a single-stage inference for simple cases and a multi-stage analysis for more complex scenarios. This is achieved through a two-phase training pipeline that integrates supervised fine-tuning with reinforcement fine-tuning based on Group Relative Policy Optimization (GRPO). Extensive experiments demonstrate that the proposed GUI-ARP achieves state-of-the-art performance on challenging GUI grounding benchmarks, with a 7B model reaching 60.8% accuracy on ScreenSpot-Pro and 30.9% on UI-Vision benchmark. Notably, GUI-ARP-7B demonstrates strong competitiveness against open-source 72B models (UITARS-72B at 38.1%) and proprietary models.
Authors
- J. Ren
- Xiantao Zhang (ORCID: https://orcid.org/0000-0002-2983-7889)
- Xuehan Ye (ORCID: https://orcid.org/0009-0000-4181-288X)
- Yang Wen (ORCID: https://orcid.org/0000-0003-0561-6229)
- Wei Dai (ORCID: https://orcid.org/0000-0002-0631-1892)
- Yiqing Li (ORCID: https://orcid.org/0000-0002-7036-717X)
- Ziyuan Chen (ORCID: https://orcid.org/0000-0003-4729-0781)
- Zhi Hao Jin (ORCID: https://orcid.org/0000-0002-7918-1574)
Institutions
- Sun Yat-sen University (CN)
- Wuhan University (CN)
- Alibaba Group (United States) (US)
Publication Details
- Published
- 2026-04-21
- DOI
- https://doi.org/10.1109/icassp55912.2026.11461773
- Primary Topic
- Context-Aware Activity Recognition Systems
- Type
- article
- Field-Weighted Citation Impact
- 0.00