Effective Scene Graph Generation by Statistical Relation Distillation

Annotating scene graphs for images is a time-consuming task, resulting in many instances of missing relations within existing datasets. In this paper, we introduce the Statistical Relation Distillation (SRD) method to enhance scene graph datasets. SRD leverages human-annotated relations alongside object-to-object and predicate-to-predicate similarities to reinforce the existence likelihood of scene graph relations. Moreover, SRD can augment relational frequency using relations of non-selected object and predicate categories that are usually omitted by scene graph generation (SGG) task. The output from SRD derives the prior probability which is combined with model-predicted probabilities to annotate missing relations in training images and subsequently re-train SGG models on the augmented dataset. We evaluate our proposed method on Visual Genome and GQA-200 datasets. Experimental results show that training on the augmented dataset enhances the performance of prominent scene-graph generation models. The implementation code is at https://github.com/LUNAProject22/SRD.