Prompt‐Conditioned Cue Matching for Black‐Box Procedural Material Fitting

Abstract Many procedural materials are implemented in legacy or production renderers where differentiable optimization is difficult to integrate, making black‐box parameter fitting still relevant. In this setting, the optimizer receives only a scalar feedback value, so the objective function determines which visual cues are observable to the search. Contrastive Language‐Image Pre‐training (CLIP) image matching is a strong baseline, but its global comparison can still underweight residual cues such as material color, feature scale, stripe spacing, gloss, or cellular structure. We propose Prompt‐Conditioned Cue Matching, a target‐relative CLIP response loss that uses a text prompt as a comparison axis between the target and candidate render. The prompt does not replace the target image; it specifies which cue should be compared more carefully. Because cue‐only optimization is weakly anchored, we combine this term with CLIP image matching in an image‐dominant hybrid loss. Experiments on checker, noise, wave, and Voronoi shaders, together with a real‐capture verification, show that language‐defined cue responses can provide targeted refinement when a visible residual cue remains.

Authors

Institutions

Publication Details

Journal
Computer Graphics Forum
Published
2026-11-13
DOI
https://doi.org/10.1111/cgf.70632
Primary Topic
Computer Graphics and Visualization Techniques
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Prompt‐Conditioned Cue Matching for Black‐Box Procedural Material Fitting

Juraj Fulir, Christoph Garth, Runzhou Mao
2 links
Computer Graphics Forum
Computer Graphics and Visualization Techniques
article

Prompt‐Conditioned Cue Matching for Black‐Box Procedural Material Fitting

Juraj Fulir, Christoph Garth, Runzhou Mao
article en

Abstract

Abstract Many procedural materials are implemented in legacy or production renderers where differentiable optimization is difficult to integrate, making black‐box parameter fitting still relevant. In this setting, the optimizer receives only a scalar feedback value, so the objective function determines which visual cues are observable to the search. Contrastive Language‐Image Pre‐training (CLIP) image matching is a strong baseline, but its global comparison can still underweight residual cues such as material color, feature scale, stripe spacing, gloss, or cellular structure. We propose Prompt‐Conditioned Cue Matching, a target‐relative CLIP response loss that uses a text prompt as a comparison axis between the target and candidate render. The prompt does not replace the target image; it specifies which cue should be compared more carefully. Because cue‐only optimization is weakly anchored, we combine this term with CLIP image matching in an image‐dominant hybrid loss. Experiments on checker, noise, wave, and Voronoi shaders, together with a real‐capture verification, show that language‐defined cue responses can provide targeted refinement when a visible residual cue remains.

Computer Graphics Forum
Fraunhofer Institute for Industrial Mathematics (DE), Rheinland-Pfälzische Technische Universität Kaiserslautern-Landau (DE)
Computer Graphics and Visualization Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon