A 149M Text Encoder as the First Tier for Phone Agents: Pre-Registered Routing to Frontier Models

Phone agents send every step to a frontier model, although many steps are easy. We ask whether Milo Pointer, a 149M-parameter text encoder that reads the screen's accessibility tree, can answer the steps it is confident about and defer the rest, offline on AndroidControl with a human-written instruction for each step. Each registered hypothesis, with its threshold and sample, was fixed in a dated log before any frontier model saw its steps. The small model answers 71-75% of steps on the device. Routing the rest to Claude Opus 5.5, given 16 retrieved training examples, is non-inferior to Opus alone in every registered comparison at under a third of its cost, and beat it in a registered confirmatory test with either deferral rule (+2.2 to +2.8 points, p <= 0.02). Pooled over 1,650 test steps the gain is 1.5 points ([+0.5, +2.4]; 1.1 on corrected labels), although the small model alone is not better than Opus: the two models err on different steps. Routing to Claude Sonnet 5.5 lifts Sonnet by 2.4-2.7 points. Alone, the text-only model scores 86.1% on AndroidControl-Low under the screenshot-agent protocol, above the 85.2% reported for OS-Atlas-7B on a slightly different step set. In a registered test with Claude Haiku 5.5 on fresh in-domain steps, a fallback shown only the small model's five best elements matched the full screen at half the input. A retrain that never used the test split reproduces the small model's accuracy. The evidence covers single steps: a related cascade lost on live multi-step tasks. We release the model, every system's per-step outputs and the registration log.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-10-30
DOI
https://doi.org/10.5281/zenodo.23818332
Primary Topic
Speech and dialogue systems
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

A 149M Text Encoder as the First Tier for Phone Agents: Pre-Registered Routing to Frontier Models

Chaitanya Talapalli
Zenodo (CERN European Organization for Nuclear Research)
Speech and dialogue systems
preprint

A 149M Text Encoder as the First Tier for Phone Agents: Pre-Registered Routing to Frontier Models

Chaitanya Talapalli
preprint en

Abstract

Phone agents send every step to a frontier model, although many steps are easy. We ask whether Milo Pointer, a 149M-parameter text encoder that reads the screen's accessibility tree, can answer the steps it is confident about and defer the rest, offline on AndroidControl with a human-written instruction for each step. Each registered hypothesis, with its threshold and sample, was fixed in a dated log before any frontier model saw its steps. The small model answers 71-75% of steps on the device. Routing the rest to Claude Opus 5.5, given 16 retrieved training examples, is non-inferior to Opus alone in every registered comparison at under a third of its cost, and beat it in a registered confirmatory test with either deferral rule (+2.2 to +2.8 points, p <= 0.02). Pooled over 1,650 test steps the gain is 1.5 points ([+0.5, +2.4]; 1.1 on corrected labels), although the small model alone is not better than Opus: the two models err on different steps. Routing to Claude Sonnet 5.5 lifts Sonnet by 2.4-2.7 points. Alone, the text-only model scores 86.1% on AndroidControl-Low under the screenshot-agent protocol, above the 85.2% reported for OS-Atlas-7B on a slightly different step set. In a registered test with Claude Haiku 5.5 on fresh in-domain steps, a fallback shown only the small model's five best elements matched the full screen at half the input. A retrain that never used the test split reproduces the small model's accuracy. The evidence covers single steps: a related cascade lost on live multi-step tasks. We release the model, every system's per-step outputs and the registration log.

Zenodo (CERN European Organization for Nuclear Research)
Speech and dialogue systems
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon