Innovative Approaches to Real-Time Occlusion Handling for Enhanced Augmented Reality Experiences

Occlusion is a critical aspect of augmented reality and mixed reality systems, profoundly impacting realism and the user experience. This research paper explores various techniques and optimizations to generate better occlusion output on augmented and mixed reality systems in real time, the proposed techniques helped us build better occlusion effects suitable for dynamic environmental objects and human subjects. Our approach successfully integrated dynamic and stable occlusion through the integration of Lightship ARDK which leverages the depth sensors in real-time producing stable occlusions. Additionally, various optimizations were applied to enhance occlusion output without sacrificing performance or accuracy. Notably, the implementation of occlusion by semantically segmenting and suppressing unwanted objects from the environment yielded better results compared to other techniques, which further improved upon the performance of the occlusion on mobile devices.