Differentiable Rendering based Part‐Aware Occlusion Proxy Generation

Abstract Software occlusion culling has become a prevalent method in modern game engines. It can significantly reduce the rendering cost by using an approximate coarse mesh (occluder) to cull hidden objects. An ideal occluder should use as few faces as possible to represent the original mesh with high culling accuracy. In contrary to mesh simplification, the process of generating a high quality occlusion proxy is not well‐established. Existing methods, which simply treat the mesh as a single entity, fall short in addressing complex models with interior structures. By leveraging advanced neural segmentation techniques and the optimization capabilities of differentiable rendering, in combination with a thoughtfully designed part‐aware shape fitting and camera placement strategy, our approach can generate high‐quality occlusion proxy mesh applicable across a diverse range of models with satisfactory precision, recall and very few faces. Moreover, extensive experiments compellingly demonstrate that our method substantially outperforms both state‐of‐the‐art methodologies and commercial tools in terms of occlusion quality and effectiveness.