Hierarchical Convolution and Graph Net for Utilizing Structural Information of Image

Graph neural network (GNN) models are capable of capturing the intrinsic structure and semantic relationships within data and this mechanism grants them substantial potential advantages in the field of computer vision. However, GNN models lack the ability to learn local fine-grained features, which limits their current applications in computer vision. Given the superior capability of convolutional neural networks (CNNs) to learn local detail features, this paper proposes a new module that integrates graph computation layers and convolution computation layers, aiming to learn patch representations that encompass both global and local features, thereby enhancing the performance of GNN models in computer vision tasks. Specifically, (1) the image is divided into patches which are treated as graph nodes, and nodes are connected to their neighbors using KNN, thus representing the image as a graph structure; (2) a module (GraphConv) consisting of a graph network computation layer and a convolution network computation layer is designed, where the former encodes large receptive field information based on the connections between nodes, and the latter learns the fine-grained features within patches using small convolution kernels; (3) the GraphConv module is stacked in a deep model using a residual structure. On the ImageNet-1K dataset and PASCAL VOC object detection dataset, our method significantly improves the performance of the original GNN model. The PyTorch code is available at https://github.com/yuan0038/GraphConvNet.