From Patch to Graph: Dual Graph Transformer Network for Few-Shot Metal Surface Defect Semantic Segmentation

Recently few-shot segmentation (FSS) of metal surface defects has been extensively developed. Most previous methods strive to achieve generalization through the pixel-based prototypes directly derived from support-query pairs. However, these methods are biased towards the global context without the intrinsic object details, thus hindering the recognition of minute defects. To this end, we propose a dual graph transformer network (DGTNet) to fully capture the aforementioned intrinsic details for accurate metal surface defect FSS. Specifically, in DGTNet, a patch-to-graph (P2G) encoder is firstly proposed to generate patch-based local and global graphs from dual perspective of support-query pairs, while simultaneously guiding global context and intrinsic details. Moreover, we equip DGTNet with a P2G transformer and a graph-to-mask (G2M) decoder to reason the relations among different nodes of the two graphs and generate the pixel-level mask of the query images, respectively. Our DGTNet also manifests remarkable adaptability within the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i>-shot FSS setting. Extensive experiments on metal surface defect datasets show that DGTNet yields superior performances under both 1-shot and 5-shot settings. Our source code is available at: https://github.com/DingH2E/DGTNet.