LB-VTON: Lower-Body Virtual Try-On via Differential Constraints and Adversarial Training Strategies

Recently, significant progress has been made in image-based virtual try on systems, but their recent advances have mainly focused on upper-body cloths at the expense of lower-body cloth try on. Problems such as the complexity of leg poses and inappropriate regularization parameters lead to the remaining challenges in virtual try on of lower-body cloths. Here, we propose a novel lower-body virtual try on network called LB-VTON, which synthesizes realistic images given images of the wearer and the target cloth. To better preserve cloth and body features, LB-VTON employs a four-stage design strategy. First, it preprocesses the image to delineate body parts and cloth regions using methods such as human parsing. Secondly it uses a regression network approach to improve the resolution and fidelity of the try on images by estimating the parameter L of the thin-sheet spline transform using second-order differential constraints derived from actual mesh data. In the third step, to overcome the problem of fabric blurring and texture loss, it employs a generative adversarial training strategy that seamlessly merges the deformed cloths with the target model by utilizing the U-Net architecture as the generator and the enhanced DC-GAN as the discriminator. Finally, all images are zoomed to higher resolution <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$(768\times 1024)$</tex> using Real-ESRGAN super-resolution technique. In addition, we present a novel lower body cloth try on dataset - the first of its kind specifically designed to improve the accuracy of lower body cloth try on. Experiments with this method on CP-VTON and its dataset show that the results produced by LB-VTON are significantly better than existing methods.