A 149M Text Encoder as the First Tier for Phone Agents: Pre-Registered Routing to Frontier Models
Phone agents send every step to a frontier model, although many steps are easy. We ask whether Milo Pointer, a 149M-parameter text encoder that reads the screen's accessibility tree, can answer the steps it is confident about and defer the rest, offline on AndroidControl with a human-written instruction for each step. Each registered hypothesis, with its threshold and sample, was fixed in a dated log before any frontier model saw its steps. The small model answers 71-75% of steps on the device. Routing the rest to Claude Opus 5.5, given 16 retrieved training examples, is non-inferior to Opus alone in every registered comparison at under a third of its cost, and beat it in a registered confirmatory test with either deferral rule (+2.2 to +2.8 points, p <= 0.02). Pooled over 1,650 test steps the gain is 1.5 points ([+0.5, +2.4]; 1.1 on corrected labels), although the small model alone is not better than Opus: the two models err on different steps. Routing to Claude Sonnet 5.5 lifts Sonnet by 2.4-2.7 points. Alone, the text-only model scores 86.1% on AndroidControl-Low under the screenshot-agent protocol, above the 85.2% reported for OS-Atlas-7B on a slightly different step set. In a registered test with Claude Haiku 5.5 on fresh in-domain steps, a fallback shown only the small model's five best elements matched the full screen at half the input. A retrain that never used the test split reproduces the small model's accuracy. The evidence covers single steps: a related cascade lost on live multi-step tasks. We release the model, every system's per-step outputs and the registration log.
Authors
- Chaitanya Talapalli (ORCID: https://orcid.org/0009-0000-8811-3349)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-10-30
- DOI
- https://doi.org/10.5281/zenodo.23818332
- Primary Topic
- Speech and dialogue systems
- Type
- preprint