PASTEL: Palette‐Affinity Soft Color Segmentation with Texture‐Preserving Editing Layers
Abstract Palette‐based localized recoloring is a common operation in texture‐rich design workflows, such as textile recoloring, fashion colorway generation, interior surface editing, product material customization, and decorative pattern design. In these scenarios, users need to modify colors associated with selected palette components while preserving fine textures, printed structures, material details, soft transitions, and boundary appearance. Soft color segmentation provides a natural representation for this task by decomposing an image into palette‐associated alpha layers. However, existing methods still face a trade‐off among reconstruction fidelity, alpha‐layer editability, and inference efficiency. Optimization‐based and geometric methods can produce faithful decompositions but are often computationally expensive or mainly driven by color‐space geometry, whereas learning‐based methods are efficient but may lose high‐frequency details and often do not explicitly model the relationship between image pixels and palette colors. We propose PASTEL , a palette‐conditioned soft color segmentation framework for editable localized recoloring. PASTEL decomposes an image into normalized palette‐associated alpha layers and residual‐enhanced color layers. The core of PASTEL is a pixel–palette affinity learning formulation. Given an input image and its palette, we first encode image pixels and palette colors into a shared feature space through decoupled image–palette encoding. We then compute pixel–palette affinity scores and normalize them along the palette dimension to obtain interpretable alpha layers, where each layer is directly associated with a palette color. This design makes the decomposition consistent with palette‐based compositing and suitable for localized recoloring. To preserve details beyond spatially constant palette colors, PASTEL introduces palette‐constrained residual refinement: a lightweight predictor estimates layer‐wise residual maps from the rough palette reconstruction and image‐encoder features, and these residuals are attached to their corresponding palette maps before alpha compositing. The model is optimized with reconstruction, sparsity, and residual‐distance objectives, keeping residual corrections tied to palette‐associated layers. Experiments show that PASTEL achieves a favorable balance among reconstruction fidelity, localized editability, alpha‐layer compactness, and inference efficiency, with palette retrieval serving as an additional application of the learned color representation.
Authors
- Yanghong Zhou (ORCID: https://orcid.org/0000-0002-0372-996X)
- P.Y. Mok (ORCID: https://orcid.org/0000-0002-0635-5318)
- Qichen Xie (ORCID: https://orcid.org/0009-0001-1025-2872)
- N. M. Cheung
Institutions
- Hong Kong University of Science and Technology (HK)
Publication Details
- Journal
- Computer Graphics Forum
- Published
- 2026-11-13
- DOI
- https://doi.org/10.1111/cgf.70638
- Primary Topic
- Generative Adversarial Networks and Image Synthesis
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- Innovation and Technology Fund