AST-Compressed Code Graphs for Gemma 4 Agents: What a Small Local Model Gains from a Budgeted Repository Map

AST-Compressed Code Graphs for Gemma 4 Agents: What a Small Local Model Gains from a Budgeted Repository Map (Gemma 4 Developer Agent Paper Track) Code and data: https://github.com/maroofiums/astgraph-gemma Abstract Running a coding agent on a single consumer GPU means the repository cannot be pasted into the prompt, and a small model gets lost when it is. We build a repository map from tree-sitter ASTs (signatures, line ranges, optionally call edges and control-flow counts), rank symbols against the issue text, and fill a fixed token budget by score per token. We test it with Gemma 4 E4B (4-bit, one T4) on SWE-bench Lite issue localization, 150 test instances, with the ranker chosen on a separate dev split without any LLM calls. At a 2k-token budget a plain signature map reaches 0.527 file Acc@1 against 0.380 for function-snippet RAG and 0.280 for raw top-k files, and doubles BM25 file ranking (0.267). Our richer variant, which adds call edges and control-flow counts, did not help: it is 0.04 lower on file accuracy (not significant) and 0.09 lower on function accuracy (pHolm = 0.002). At function level, snippet RAG matches the signature map, and a two-stage prompt (map first, then candidate files) lifts function accuracy over the full variant (0.042 to 0.229 on the ablation subset) but not significantly over the plain signature map (0.188; difference +0.042, p = 0.54). Memory measurements show that for this model the KV cache is a small part of what a long prompt costs; logits and activations dominate, and chunked prefill with last-token logits is what lets an 8k prompt be prefilled on a T4 (measured for prefill only). We also report the failure analysis, the negative results, and how the index can be used either as a prompt layer or as a set of tools. Motivation and task Most agent work on repository-level tasks assumes a large hosted model. When the model must run locally, for privacy or cost, the prompt budget is small and the model is easily distracted. Pasting the files that a keyword search ranks highest spends the budget on method bodies the model does not need yet. This work asks three questions on one model and one task: which parts of a repository-map representation matter for a small model, how ranking interacts with the token budget, and what the memory side looks like on about 15 GB of VRAM. Task (issue localization). Given an issue text and a repository checkout, the model names up to three files and one function where the fix belongs. The gold files and touched lines come from the reference patch. This is the localization step of systems such as Agentless; no patches are generated. Method Symbol extraction. Every .py file is parsed with tree-sitter. For each class, method and function we record the qualified name, one-line signature (with annotations), line range, base classes, called names, and counts of if/for/while/try/with/match nodes. Files are keyed by SHA-1, so a re-index after a commit only re-parses changed files. Symbol graph. Calls are resolved by name only (no type inference). An edge a → b means "a calls something called b", weighted by 1/(number of candidates), so each call name spreads a total weight of at most one. Rankers (four). bm25 (BM25 over a small document per symbol: identifier words, file path, callees); bm25m (BM25 plus boosts when the issue mentions the file path, including tracebacks, or the symbol name); ppr (personalised PageRank on the undirected call/inheritance graph, seeded with normalised BM25 scores, α = 0.5); hybrid (0.7 × max-normalised bm25m + 0.3 × max-normalised PPR). Budgeted rendering. The top 3000 symbols are candidates. Each has a token cost (its rendered line, plus the file path the first time its file appears). A greedy knapsack-style fill takes symbols in decreasing score per token and keeps each one while it fits. The final text is re-counted and trimmed, so the token budget is a hard limit. Files are listed by their best symbol score. Prompt. JSON-only output (up to three files and one qualified symbol). The map goes first, then the issue (truncated to 1024 tokens for every method). Greedy decoding, at most 200 new tokens, lenient parsing. Experimental setup Model and hardware: google/gemma-4-E4B-it via Hugging Face Transformers, bitsandbytes NF4 with fp16 compute, one NVIDIA T4 (14.56 GiB usable), SDPA attention. Data: SWE-bench Lite (300 test instances, 12 repositories), sorted by id and shuffled with seed 0. The last 60 form the dev split (used only to choose the ranker, without any LLM calls); the first 150 of the remainder form the test split. Ablations and the budget sweep use the first 60 test instances. Function accuracy is defined on the 119 of 150 test instances (48 of the first 60) where every touched line sits inside a function or class. Methods at equal budget (B = 2048 tokens): issue only; BM25 over files (no LLM); raw top-k (top ten BM25 files pasted whole and cut at B); snippet RAG (BM25 over function bodies, first 40 lines each); SIGMAP (signatures only, an approximate re-implementation of the Aider repository-map idea); OURS (signatures + callees + control-flow counts). Statistics: File Acc@1/@3 and function Acc@1; 95% bootstrap intervals (10,000 resamples); paired bootstrap differences with Holm-corrected p-values. Main results (test split, n = 150, B = 2048) Method File@1 File@3 Func@1 (n = 119) Issue only 0.207 0.253 0.109 BM25 files (no LLM) 0.267 0.460 – Raw top-k 0.280 0.293 0.109 Snippet RAG 0.380 0.447 0.227 OURS (edges + flow) 0.487 0.627 0.143 SIGMAP (signatures) 0.527 [.447, .607] 0.647 0.235 [.160, .311] SIGMAP is 0.147 higher than snippet RAG on file Acc@1 (95% CI 0.060 to 0.233, pHolm = 0.004) and 0.247 higher than raw top-k (pHolm < 0.001). SIGMAP is about 1.97× BM25 file ranking on File@1 (0.527 vs 0.267), which is the "doubling" in the abstract. At function level, snippet RAG is as good as the signature map (0.227 vs 0.235, p = 0.93): a signature map shows names, not code. Graph methods cost about 3–4 s more median end-to-end latency (longer prefill and longer answers). Ranker selection on dev (no LLM) Rankers were compared by how often the prompt contains the gold file and the gold function definition, using J(ρ) = ½ Σ over B ∈ {1024, 2048} of [Covfile + Covfunc]. The hybrid ranker won (J = 0.998, against 0.973 for bm25m, 0.889 for bm25 and 0.443 for ppr). PPR alone is poor at small budgets because it spreads mass over graph hubs and ignores what the issue names. At 2048 tokens, 0.65 of gold files appear in a signature map, against 0.47 in snippet RAG and 0.17 in raw top-k. Negative and ablation results Call edges and control-flow counts did not help. OURS is 0.040 below SIGMAP on file Acc@1 (not significant, pHolm = 0.26) and 0.092 below on function Acc@1 (pHolm = 0.002). With a fixed budget, every callee list pushes another symbol out of the map; dev coverage of the gold function at 2048 tokens drops from 0.42 (signatures) to 0.29 (OURS). Ablations (first 60 test instances): SIGMAP 0.500 / 0.650 / 0.188 (File@1 / File@3 / Func@1); + edges 0.467 / 0.667 / 0.125; + control flow 0.533 / 0.683 / 0.208 (within noise); OURS 0.417 / 0.617 / 0.042; SIGMAP with PPR ranker 0.333 / 0.483 / 0.062. Format and ordering effects (JSON vs compact rows, alphabetical, shuffled or relevance-ordered files) are unresolved at n = 60; alphabetical and shuffled orders scored at least as well as relevance order. Two-stage prompt (stage 1: file and symbol names only at half the budget; stage 2: full map restricted to candidate files) lifts function accuracy over OURS from 0.042 to 0.229 (pHolm = 0.003), at about 1191 extra prompt tokens and about 4.5 s extra median latency. Against SIGMAP (0.188) the difference is +0.042 (p = 0.54), so a second stage is not shown to beat a single-stage signature map. Budget sweep: SIGMAP file accuracy is flat from about 1.1k to 4.7k prompt tokens (0.533, 0.550, 0.500, 0.517); 512 tokens of map already gives most of the benefit. Snippet RAG keeps improving with budget and has the highest function-level point estimate at every budget (intervals overlap). Failure analysis On the first 60 test instances the gold file is in the prompt for 78% of issues with OURS (47/60) and 80% with SIGMAP (48/60). When it is absent the model never gets it right (0/13 and 0/12). When it is present, the model picks it first 53% (OURS) and 62% (SIGMAP) of the time. Most remaining errors are plausible neighbours (for example django/db/models/query.py instead of django/db/models/sql/query.py, Dataset instead of DataArray in xarray, or a test file instead of the module under test). Memory and prefill on a T4 The "4-bit" checkpoint is not small: resident weights are 9081 MiB, because the 5376 MiB of per-layer embeddings and 1280 MiB token embedding stay in bf16; only 2146 MiB of compute weights are quantized. The analytic fp16 KV cache is tiny (14 MiB at 256 tokens, 116 MiB at 6144; 42 layers, 35 sliding-window with window 512, 7 full-attention, 2 KV heads, 18 layers sharing KV). Measured memory above the weights is far larger (93 MiB at 256 tokens, 727 MiB at 2048, 4026 MiB at 6144). The KV cache is about 3% of that at 6144 tokens and at most about 20% at any length; activations and logits dominate (vocabulary of 262,144). At 4096 tokens, full logits cost 4340 MiB above weights, last-token logits 2069 MiB, and chunked prefill (512) with last-token logits 624 MiB. At 6144 tokens full logits run out of memory; at 8192 only the chunked path runs (1191 MiB, 15.4 s). Prefill time grows faster than linearly: 0.51 s at 512, 2.4 s at 2048, 9.0 s at 4096 tokens. These probes measure prefill memory on random token ids; answer quality at 8k tokens and decoding after a chunked prefill were not checked. Compression ratio Over ten test instances, the full symbol graph is on average 4.8× smaller than the Python source (ρAST). After ranking and budgeting to 2k tokens