Hybrid-ACT: scaling robot imitation learning under few-demonstration settings with vector-quantized cross-attention

Learning effective policies from few demonstrations remains a key challenge in robot imitation learning. Although Transformer-based policy models are effective for long-horizon tasks, scaling model size does not always improve performance under few-demonstration settings. In this paper, we propose Hybrid-ACT, a scalable imitation learning model based on a Transformer architecture with Vector-Quantized Attention (VQ-Attention). The proposed method integrates visual observations, robot states, and actions using continuous and discrete representations by quantizing only the key vectors in cross-attention. This design introduces discrete key representations with negligible overhead in both parameter count and full-model peak memory. Overall, Hybrid-ACT uses fewer parameters and less peak inference memory than ACT while maintaining comparable inference latency. To address codebook collapse, we introduce Gumbel-Softmax reparameterization that improves codebook utilization and stabilizes the training of deeper decoder architectures. We evaluate the proposed method on short-horizon and long-horizon bimanual manipulation tasks in the RoboTwin benchmark. Hybrid-ACT achieves better performance than existing baseline methods, with particularly substantial improvements on several long-horizon manipulation tasks. We further confirm that the performance gains become more pronounced as model scale increases, demonstrating the effectiveness of Hybrid-ACT for robot imitation learning under few-demonstration settings.