Toward Universal Instance Shadow Detection Based on Pairwise Grouping With Contrastive Morphological Alignment

Instance shadow detection, a recently proposed task, is more challenging than traditional shadow detection, as it aims to detect shadow instances paired with the associated objects that cast shadows. Previous works focus on outdoor natural light scenes, featuring unique shadow per object. These methods utilized an offset-based bidirectional relational learning mechanism requiring dual matching, which struggled in scenarios with artificial light where an object may cast multiple shadows. In this paper, we propose a pairwise grouping strategy enabling the model detecting objects and their corresponding shadows in conjunction, eliminating the need for dual matching. The model incorporates the pairwise query, serving as the matching entities for objects and their corresponding shadows, and the pairwise masked attention to enforce the network focus more on the relational features between object and shadow. Additionally, we introduce a morphological representation of object-shadow pairs, and employ contrastive morphological alignment to further refine the pairwise relational features. In this way, the Pairwise Grouping with Contrastive Morphological Alignment (PGCMA) enables the model to detect instance shadows through single-time matching. Moreover, we propose a Universal Scene Shadow-Object dataset (USSO), which is more comprehensive and includes a broader range of scenes than previous datasets. Extensive experiments conducted on both existing and our newly proposed benchmarks demonstrate the superiority of our method over current state-of-the-art approaches. The source code and our proposed dataset are publicly available at https://github.com/ssecv/USSO/.