SPL-PlaneTR: Lightweight and Generalizable Indoor Plane Segmentation Based on Prompt Learning

Single-image plane segmentation plays an important role in understanding 3D indoor scenes, including applications such as 3D indoor reconstruction. In recent years, PlaneTR, a transformer-based architecture, has achieved remarkable performance in single-image plane instance segmentation. It has garnered significant attention from researchers and remains one of the most advanced algorithms in this field. However, PlaneTR has the following two major limitations: its ineffective utilization of line segment information within images and the high number of parameters. In this study, we propose an improved version of PlaneTR, named Spatial Prompt Learning PlaneTR (SPL-PlaneTR), to address these issues. Our approach effectively balances model complexity and performance. Specifically, to more effectively leverage structural information provided by line segments, we replace the original line segment's transformer branch with a lightweight line segment prompt module and line segment prompt adapter. Additionally, we introduce spatial queries to replace conventional position queries, enabling the network to accurately localize planes across diverse indoor scenes. The experimental results demonstrate that our model, with fewer parameters, outperforms PlaneTR on both the original and noise-corrupted ScanNet datasets. Furthermore, SPL-PlaneTR achieves superior zero-shot transfer performance on the Matterport3D, ICL-NUIM RGB-D, and 2D-3D-S datasets compared to PlaneTR. Notably, our lightweight SPL-PlaneTR also surpasses several state-of-the-art algorithms in this domain. Our code and model have been publicly available.