Scene Graph Generation Approach with Integrated Environmental Contextual Information

Scene graphs depict entities and their relationships in images, enabling tasks like semantic understanding. Current research focuses on entity and relationship recognition and generation efficiency. However, image environmental information is significant but often overlooked. Integrating it with scene graphs enhances computer proficiency in multiple domains. This paper addresses the lack of environmental information in scene graphs. It uses multimodal pre-training models to recognize street-view image environmental info and incorporates it into scene graphs. After introducing related tech basis, it proposes a recognition method and verifies it via experiments.