ABACUS: Adapting Unified Foundation Model for Count Image Understanding and Generation
We present ABACUS, a unified vision-language model that jointly addresses object counting, crowd counting, referring-expression counting, and count-faithful image generation within a single 3B-parameter model. ABACUS introduces three contributions: density-aware adaptive zooming paired with an objectness map from multi-head self-attention decomposition to spatially ground count predictions; a boundary-aware count policy trained via GRPO with nested local, boundary, and global rewards to eliminate over- and undercounting at crop boundaries; and a cycle-consistent GRPO strategy in which the frozen understanding branch scores generated candidates on count-deviation and aesthetic quality, closing the understanding-generation synergy gap without any external critic or annotation. ABACUS achieves state-of-the-art results across seven benchmarks spanning object counting (FSC-147, CARPK), crowd counting (ShanghaiTech A/B), referring-expression counting (REC-8K), count-faithful generation (CoCo-Count, T2I-CompBench, GenEval), and count reasoning (CountQA), surpassing both task-specific specialists and larger generalist models. Project page is at https://mondalanindya.github.io/pages/ABACUS/.
Authors
- Sauradip Nag (ORCID: https://orcid.org/0000-0002-2943-6663)
- Anjan Dutta
- Anindya Mondal
Institutions
- Simon Fraser University (CA)
- University of Surrey (GB)
Publication Details
- Journal
- ACM Transactions on Graphics
- Published
- 2026-11-09
- DOI
- https://doi.org/10.1145/3842572
- Primary Topic
- Multimodal Machine Learning Applications
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- Nvidia
- Government of the United Kingdom
- UK Research and Innovation
- University of Bristol
- University of Surrey
- Science and Technology Facilities Council