Efficient Visuo-Tactile Learning Via Fine-Grained Alignment and Importance-Aware Token Retention

Visuo-tactile learning is crucial for robotic manipulation, yet current methods are limited by coarse-grained feature alignment and computational inefficiency. To address this, an efficient visuo-tactile learning framework with dual-level optimization is proposed for representation learning and inference. At the representation level, generalizable and fine-grained features are enhanced by combining dynamic mask-based token alignment with the alignment of physics-based salient tactile foregrounds to attribute-specific semantic text. At the inference level, an Importance-aware Token Retention mechanism is introduced that prunes non-salient tokens, reducing computational cost quadratically and enabling real-time application. Extensive experiments show performance comparable to or surpassing state-of-the-art accuracy across multiple downstream tasks while maintaining a balance between performance and efficiency.