Semantic unit guided image understanding for human AI collaborative image editing via multimodal deep learning

This paper addresses natural-language-feedback-assisted image editing by examining how semantic role decision-making can reduce unintended changes to unspecified subjects, backgrounds, styles, and contextual relations. Existing instruction-based methods commonly represent editing intent or spatial scope without explicitly distinguishing content to be edited, preserved, or confirmed. This paper therefore proposes a semantic-unit-guided framework that assigns target, protected, and uncertain roles before editing. Target units guide the requested modification, protected units impose spatial or feature-level consistency constraints, and uncertain units support confirmation-based role updating. Experiments on InstructPix2Pix and MagicBrush, with zero-shot evaluation on a 500-sample RealEdit subset, compare the framework with nine representative baselines and binary target/non-target masking. The three-role model improves Protected Semantic Preservation over binary masking by 2.3% points on InstructPix2Pix and 2.9 points on MagicBrush. Its preservation advantage remains 4.5 and 6.5 points under matched-Edit-Success conditions. Protected Semantic Preservation correlates with human preservation ratings at 0.718. Alternative annotation criteria preserve the main performance pattern, while failure and sensitivity analyses identify ambiguous instructions, grounding quality, confidence thresholds, and preservation weight as influential factors. Feedback results are obtained under a benchmark-based uncertainty-confirmation protocol using subsequent instructions and annotation-derived responses; direct user comprehension, usability, and engagement are not evaluated.

Authors

Institutions

Publication Details

Journal
Discover Artificial Intelligence
Published
2026-11-21
DOI
https://doi.org/10.1007/s44163-026-02544-x
Primary Topic
Generative Adversarial Networks and Image Synthesis
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Semantic unit guided image understanding for human AI collaborative image editing via multimodal deep learning

Xiaoping Gu, Xiaoqian Yang
Discover Artificial Intelligence
Generative Adversarial Networks and Image Synthesis
article

Semantic unit guided image understanding for human AI collaborative image editing via multimodal deep learning

Xiaoping Gu, Xiaoqian Yang
article en

Abstract

This paper addresses natural-language-feedback-assisted image editing by examining how semantic role decision-making can reduce unintended changes to unspecified subjects, backgrounds, styles, and contextual relations. Existing instruction-based methods commonly represent editing intent or spatial scope without explicitly distinguishing content to be edited, preserved, or confirmed. This paper therefore proposes a semantic-unit-guided framework that assigns target, protected, and uncertain roles before editing. Target units guide the requested modification, protected units impose spatial or feature-level consistency constraints, and uncertain units support confirmation-based role updating. Experiments on InstructPix2Pix and MagicBrush, with zero-shot evaluation on a 500-sample RealEdit subset, compare the framework with nine representative baselines and binary target/non-target masking. The three-role model improves Protected Semantic Preservation over binary masking by 2.3% points on InstructPix2Pix and 2.9 points on MagicBrush. Its preservation advantage remains 4.5 and 6.5 points under matched-Edit-Success conditions. Protected Semantic Preservation correlates with human preservation ratings at 0.718. Alternative annotation criteria preserve the main performance pattern, while failure and sensitivity analyses identify ambiguous instructions, grounding quality, confidence thresholds, and preservation weight as influential factors. Feedback results are obtained under a benchmark-based uncertainty-confirmation protocol using subsequent instructions and annotation-derived responses; direct user comprehension, usability, and engagement are not evaluated.

Discover Artificial IntelligenceVol. 6(1)
Zhixing College of Hubei University (CN)
Generative Adversarial Networks and Image Synthesis
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon