Correction of lighting effects in diminished reality for concealing light equipment

Diminished Reality (DR) is a technology that conceals real-world objects by blending them with background information. This technology has applications in various fields, including simulating furniture arrangements at home before purchase. In such a system, existing furniture is first concealed using DR, and then new virtual furniture is displayed onto the concealed scene using Augmented Reality (AR). This allows users to visualize how new furniture will look in their home, even in spaces where items are already placed, therefore it makes it easier to make informed purchasing decisions. However, existing approaches are difficult to accurately account for and correct the environmental effects of concealed objects, particularly those that influence lighting, such as light equipment. This study proposes a method to not only conceal light equipment within a scene but also to correct the associated lighting effects, and it enhances the realism of the DR process. The method first estimates the light source’s 3D position using depth information captured by an RGB-D camera and applies a brightness correction based on the estimated position and its distance from the light source. Then, Generative Adversarial Network (GAN) is used to perform the DR process and conceal the light equipment effectively within the corrected image. The results show that applying the proposed brightness correction reduces the inconsistency in lighting effects and leads to a more realistic concealment of the light equipment and improves the overall visual quality of the scene.