Patrick Star: A comprehensive benchmark for multi-modal image editing

Generative image editing enhances and automates traditional image designing methods. However, there is a significant imbalance in existing research, where the development of sketch-guided and example-guided image editing has not been sufficiently explored compared to text-guided image editing, despite the former being equally important in real-world applications. The leading cause of this phenomenon is the severe lack of corresponding benchmark datasets. To address this issue, this paper proposes a comprehensive and unified benchmark dataset, Patrick Star, which consists of approximately 500 test images, to promote balanced development in this field across multi-task and multi-modal settings. First, theoretical analysis grounded in Evaluatology highlights the importance of establishing a balanced benchmark dataset to advance research in image editing. Building on this theoretical foundation, the dataset’s construction methodology is explained in detail, ensuring it addresses critical gaps in existing studies. Next, statistical analyses are conducted to verify the dataset’s usability and diversity. Finally, comparative experiments underscore the dataset’s potential as a comprehensive benchmark, demonstrating its capacity to support balanced development in image editing. • Semantic tag specification. • Multi-dimensional task taxonomy. • Unified Evaluation Test Dataset for Image Editing Tasks