Object Attribute Recognition Method Integrating Visual–Tactile Data and Multitask Learning

Accurate object attribute recognition is crucial for intelligent robots to efficiently accomplish various tasks. However, current methods suffer from limitations such as reliance on single-modality inputs and poor inter-task correlation across attribute recognition tasks. To address these challenges, this study proposes a multi-dimensional object attribute recognition framework by integrating visual-tactile data fusion (VTDF) with multi-task learning (MTL) techniques. First, we design a spatio-temporal attention-based VTDF module, enabling the model to fully process, interact, and fuse cross-modal visual and tactile information, thereby significantly improving the precision of single-attribute recognition. Second, we introduce a MTL method grounded in cross-task attention queries, which facilitates efficient inter-task information sharing and interaction while ensuring task balance. This allows each task to leverage shared knowledge from other tasks while preserving its own task-specific features, ultimately enabling robots to acquire more accurate classification results across all target attributes. Experiments demonstrate that the proposed VTDF module effectively integrates features from visual and tactile modalities, substantially enhancing single-attribute recognition accuracy. The MTL module successfully achieves inter-task information interaction and balanced performance. Finally, by combining these two modules, our framework achieves efficient and precise recognition of multi-dimensional object attributes, empowering intelligent robots with comprehensive perception of environmental objects. Our code can be available at https://github.com/CdpLab/ObjectAttrRecogn.