<i>XStreamVGGT</i> : Extremely Memory‐Efficient Streaming Vision Geometry Grounded Transformer With KV Cache Compression

ABSTRACT Learning‐based 3D visual geometry models have significantly advanced with the advent of large‐scale transformers. Among these, StreamVGGT leverages frame‐wise causal attention to deliver robust and efficient streaming 3D reconstruction. However, it suffers from unbounded growth in the Key–Value (KV) cache due to the massive influx of vision tokens from multi‐image and long‐video inputs, leading to increased memory consumption and inference latency as input frames accumulate. To address this gap, we propose XStreamVGGT, a tuning‐free approach that seamlessly integrates pruning and quantization to systematically compress the KV cache, enabling extremely memory‐efficient streaming inference. Specifically, redundant KVs generated from multiframe inputs are first pruned to satisfy a fixed KV memory budget. This process employs an efficient token importance identification mechanism, ensuring full compatibility with high‐performance attention kernels, such as FlashAttention. Furthermore, dimension‐adaptive KV quantization is incorporated into the pruning pipeline to further reduce memory consumption. By exploiting the intrinsic distribution patterns of KV tensors, this method effectively preserves numerical accuracy. Extensive evaluations show that XStreamVGGT achieves mostly negligible performance degradation while substantially reducing memory usage by 4.42 and accelerating inference by 5.48, enabling practical and scalable streaming 3D applications. The code is available at https://github.com/ywh187/XStreamVGGT/ .