An Auxiliary Fine-tuning Transformer for HumanObject Interaction Detection

The detection of human-object interactions (HOI) can be approached through two primary methods: two-stage and one-stage approaches. Two-stage methods involve dividing the task into object detection and interaction classification, which, although effective, still face several challenges. In this paper, we propose a novel Fine-Tuning Transformer (FFT) to address these challenges within a one-stage framework. FFT introduces a new module that assists the decoder in identifying the most relevant HOI pairs in an image. Additionally, we introduce a reweighting method to tackle the issue of long-tail distribution. Our method incorporates various enhancements and has demonstrated improved accuracy, achieving state-of-the-art results on the HICO-DET and VCOCO datasets.