AI-Driven Style Transfer Framework Based on 3D Gaussian Splatting for Immersive Experiences

With the recent advancement of virtual try-on (VTO) technology, it is being applied to various fields. Although advancements in VTO technology have enabled not only 2D but also 3D visualization, applying style transfer to hyper-local regions remains challenging due to the complex surface curvature and ambiguous boundaries of 3D objects. To address these challenges, we propose an immersive 3D style transfer framework based on 3D Gaussian splatting. A segmentation model is employed to accurately segment target regions, and a large-scale specialized dataset is constructed to capture the morphological diversity of human hands. Furthermore, neural style transfer is integrated with the 3D representation to enable precise style application to hyper-local regions. The proposed framework achieves a mean intersection of union (mIoU) of 0.806 in segmentation and high-fidelity stylization with learned perceptual image patch similarity (LPIPS) and reference-based LPIPS (Ref-LPIPS) scores of 0.1472 and 0.0196, respectively. These results indicate that the proposed framework can provide the quality requirements and immersive VTO experience.