I2IP: image to image editing with prompt control using guided diffusion models

Real image editing has been one of the important research topics in the field of image generation, with the proposal of diffusion model, real image editing based on the guided diffusion model has become the mainstream, which edits the original real image through different conditions (text, image, sketch et al.), and outputs the edited image that align well with input conditions. Among them, the image-based real image editing task often can only migrate the style of the reference image to the original image, which is not controllable enough and lacks the ability of fine-grained image translation. In this paper, we propose I2IP, which introduces text prompt in the image translation process to control the image translation process in a fine-grained way, to improve the generalization and refinement of the image translation algorithm. Compared to text-based real image editing, I2IP can achieve image style transfer with reference image, while compared to image-based real image editing, I2IP takes the advantage of text-based image editing and control the editing result.