Click-based interactive image segmentation with global hints and local corrections

The aim of click-based interactive image segmentation is to obtain pixel-level segmentation masks by only a small number of manual clicks. This approach streamlines the process of pixel-level annotation and image editing. Much research has focused on this area. In particular, SimpleClick, which utilizes Vision Transformers, has implemented a straightforward design that has demonstrated its effectiveness in interactive image segmentation. However, two issues remain: First, this simple design does not fully utilize the global guidance provided by the interaction map. Second, treating all clicks equally fails to maximize the benefits of the new clicks’ guidance in each iteration. To address these challenges, we propose a novel interactive image segmentation network called Glclick. Specifically, we introduce a Global Hint Module (GHM) that integrates global information from clicks into the transformer backbone. In addition, Glclick incorporates a Local Correction Module (LCM) that performs local optimization on the target masks generated by the backbone network. Extensive experiments on four generic datasets and three medical datasets demonstrate the superiority and generalizability of Glclick. • Designed a brand-new interactive image segmentation model called “Glclick”. • Designed a Global Hint Module called GHM and a Local Correction Module called LCM. • Advanced performance was achieved on generic datasets and three medical datasets.