Cost-Efficient Observation for LLM-Based UI Grounding: Measuring Predicate-Relative Attribute Sufficiency Within a Symbolic Representation
A large language model (LLM)-based agent uses text observations to find UI elements. Their content affects token cost and grounding success. Observation-reduction work asks how much to remove; we ask which attribute subsets are sufficient under a fixed success predicate. We compare 6000 events across 24 synthetic pages, varying attributes and serialisations. Removing volatile class and id values increases success by 9.48 pp (95% CI [+4.90, +15.52] pp). The secondary metric P_now (unique correct match) changes little where a stable attribute exists but decreases otherwise. The improvement comes from removing unstable locator candidates, not adding information. Abbreviating attribute names reduces success by 12.08 pp, saving 0.2% of tokens. Removing JSON syntax while keeping attribute names reduces cost without a significant change in success. The cost-optimal profile (COP), best among 21 main configurations, achieves 10.00 pp higher success with 34.0% fewer tokens than full-attribute JSON. In a one-attempt comparison, the accessibility snapshot uses half the tokens but performs worse; neither dominates. The direction persists across three gpt-4.1 generations and four models under non-identical decoding settings (for o4-mini, via the COP). A 33-site archived-page pilot shows a smaller gain in that direction. Results concern predicate-relative sufficiency in HTML DOM with CSS locators.
Authors
- Milena Kirilova Lazarova (ORCID: https://orcid.org/0000-0002-8256-0831)
- Anton Angelov (ORCID: https://orcid.org/0009-0007-6417-7701)
Institutions
- Technical University of Sofia (BG)
Publication Details
- Journal
- Applied Sciences
- Published
- 2026-11-11
- DOI
- https://doi.org/10.3390/app162211180
- Primary Topic
- Web Data Mining and Analysis
- Type
- article
- Field-Weighted Citation Impact
- 0.00