Face generation combining text and sketch semantics

With the booming development of deep learning and image generation technology, the research on sketch-generated face images has achieved remarkable results, however, there are still deficiencies in some scenarios that require high face image fidelity, and it is not possible to generate images that are semantically and geometrically consistent with the input sketches. A semantically controllable sketch-generated face image method is proposed, where some modules is designed to extract the sketch semantics, merge them with the text semantics into a more expressive semantics, and feed them into the generator along with the sketches in order to achieve semantic and geometric alignment. The proposed method is experimentally validated on open-source datasets and homemade datasets, and the experimental results show that the method effectively improves the quality of the generated images.