Momentum in diffusion samplers: why a "momentumized" DPM-Solver++ fails on a flow model, and what a negative momentum coefficient does at equal NFE

Summary. The "momentumized" samplers of the ComfyUI-Extra-Samplers pack (2023) are advertised as momentum variants of DPM-Solver++. On a flow-matching model (Anima, bf16) one of them, dpmpp_dualsde_momentumized, returns a washed-out silhouette instead of an image, and through the standard KSampler its momentum coefficient is zero. We ported it exactly, took it apart, and then asked a constructive question: is there a form of momentum that lowers the error against a converged solution at one network evaluation per step and equal evaluation count (NFE)? Nine hypotheses were registered before any measurement; five of them were refuted. Diagnosis. The failure is in the noise step, not in momentum. The sampler uses variance-exploding formulas on a flow model, so with eta = 1 the final latent has a standard deviation of 3-4.5 instead of about 1.2; with eta = 0 the same code draws a recognisable image. The evidence is indirect (the same 2S SDE algorithm in the flow-correct form of ComfyUI core gives a clean picture; the mechanism was not isolated by a separate experiment). Two further defects: a multistep 3M correction is added on top of an already second-order 2S step (the sampler diverges even without noise, 2-35 times worse than DPM-Solver++(2M) at equal NFE), and the function parameter r is overwritten inside the loop. The port is checked against the original to 1e-10 on a toy model. What momentum has to be. In the exponential-integrator frame, momentum on the data prediction is an exponential moving average D_bar = m D_bar_prev + (1 - m) D. A positive coefficient is a low-pass filter: it adds lag and increases the error monotonically (up to +61% at m = 0.6). A negative coefficient is a recursive extrapolation and lowers the error; the recursion is stable only for |m| < 1 (m = -0.9 gives no gain, m = -1.2 diverges). Main results (Anima, bf16, 576x768, CFG 5, 15 units = 5 prompts x 3 seeds, 8-24 evaluations, paired bootstrap intervals, reference = converged solution of the same kind): Deterministic sampling: EMA momentum with m = -0.6 (or the time-scaled form with m = -0.5) has 11-21% lower error than DPM-Solver++(2M) at equal NFE, i.e. about 1.2 times fewer steps for the same error; the intervals exclude zero in every cell. LPIPS to the decoded reference is lower in 14 of 15 units at 8 steps. Stochastic sampling: strong negative momentum loses its advantage. Damped extrapolation (beta = 0.5) and EMA with m = -0.3 give 6-15% lower strong error than dpmpp_2m_sde (shared Brownian path); at CFG 8 and 1216x832 the milder EMA gave colour casts and damped extrapolation did not. One evaluation per step beats two at equal NFE: the best one-evaluation SDE variant is 16-29% better than the two-evaluation dpmpp_sde at half the steps. Of the nine registered hypotheses, four were supported (positive momentum only hurts; time-scaled form not better than the best constant for a positive coefficient; LPIPS agrees with latent error; one evaluation beats two) and five were refuted (including that the stacked correction is the cause of the failure). A static step grid from an earlier study cut the error by half on the prompts it was fitted on and transferred across prompts of the same family, but it did not help a held-out natural-language prompt and broke an 8-step picture at CFG 8; it is shipped as an experimental node only (post hoc analysis). Prior work. Momentum in diffusion sampling is not new: Heavy Ball momentum inside existing numerical solvers to enlarge their stability regions (Wizadwongsa et al., ICLR 2024, arXiv:2307.11118), an Adam-style momentum sampler (Wang et al., arXiv:2308.11941), momentum for peculiar images (Jang et al., AAAI 2024), TADA (arXiv:2506.21757), and the ComfyUI packs ComfyUI-sonar and Seren Sampler. Only abstracts and READMEs were read; the code of these packages was not run and no comparison was made. What we could not find is an explanation of why the shipped sampler fails on a flow model, and a measurement of the sign and size of EMA momentum against converged references at equal evaluation count; this is a statement about our search, not about the field. Software. Exact port of the original sampler, the new samplers, ComfyUI nodes (a momentum sampler node, a filter node that wraps any one-evaluation sampler, a sigma-grid node), KSampler entries, 13 CPU tests, all scripts, tables (results/) and figures are in the repository: https://github.com/YakovSava/momentum-cfg-samplers-research. The README (English, Russian, Chinese, Meadow Mari) is the full record with the hypothesis table and all intervals. The filter wrapper for 19 additional samplers was added as an engineering extension and has not been evaluated on the model. Limitations. One model, one latent size and CFG in the main sweeps; effects below about 5% are not detectable; a 3-seed check at CFG 8 cannot resolve differences below about 15%. The main metric is relative L2 distance of the final latent to a converged solution of our own (about 5% own error); LPIPS only for the deterministic winners at 8 and 16 steps. Strong error of an SDE sampler is not image quality. The sweep sections beyond the registered grid, the grid transfer analysis and the CFG 8 check are post hoc. The Meadow Mari translation is machine-assisted and not reviewed by a native speaker.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-14
DOI
https://doi.org/10.5281/zenodo.24214605
Primary Topic
Generative Adversarial Networks and Image Synthesis
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Momentum in diffusion samplers: why a "momentumized" DPM-Solver++ fails on a flow model, and what a negative momentum coefficient does at equal NFE

Iakov Savelev
Zenodo (CERN European Organization for Nuclear Research)
Generative Adversarial Networks and Image Synthesis
preprint

Momentum in diffusion samplers: why a "momentumized" DPM-Solver++ fails on a flow model, and what a negative momentum coefficient does at equal NFE

Iakov Savelev
preprint en

Abstract

Summary. The "momentumized" samplers of the ComfyUI-Extra-Samplers pack (2023) are advertised as momentum variants of DPM-Solver++. On a flow-matching model (Anima, bf16) one of them, dpmpp_dualsde_momentumized, returns a washed-out silhouette instead of an image, and through the standard KSampler its momentum coefficient is zero. We ported it exactly, took it apart, and then asked a constructive question: is there a form of momentum that lowers the error against a converged solution at one network evaluation per step and equal evaluation count (NFE)? Nine hypotheses were registered before any measurement; five of them were refuted. Diagnosis. The failure is in the noise step, not in momentum. The sampler uses variance-exploding formulas on a flow model, so with eta = 1 the final latent has a standard deviation of 3-4.5 instead of about 1.2; with eta = 0 the same code draws a recognisable image. The evidence is indirect (the same 2S SDE algorithm in the flow-correct form of ComfyUI core gives a clean picture; the mechanism was not isolated by a separate experiment). Two further defects: a multistep 3M correction is added on top of an already second-order 2S step (the sampler diverges even without noise, 2-35 times worse than DPM-Solver++(2M) at equal NFE), and the function parameter r is overwritten inside the loop. The port is checked against the original to 1e-10 on a toy model. What momentum has to be. In the exponential-integrator frame, momentum on the data prediction is an exponential moving average D_bar = m D_bar_prev + (1 - m) D. A positive coefficient is a low-pass filter: it adds lag and increases the error monotonically (up to +61% at m = 0.6). A negative coefficient is a recursive extrapolation and lowers the error; the recursion is stable only for |m| < 1 (m = -0.9 gives no gain, m = -1.2 diverges). Main results (Anima, bf16, 576x768, CFG 5, 15 units = 5 prompts x 3 seeds, 8-24 evaluations, paired bootstrap intervals, reference = converged solution of the same kind): Deterministic sampling: EMA momentum with m = -0.6 (or the time-scaled form with m = -0.5) has 11-21% lower error than DPM-Solver++(2M) at equal NFE, i.e. about 1.2 times fewer steps for the same error; the intervals exclude zero in every cell. LPIPS to the decoded reference is lower in 14 of 15 units at 8 steps. Stochastic sampling: strong negative momentum loses its advantage. Damped extrapolation (beta = 0.5) and EMA with m = -0.3 give 6-15% lower strong error than dpmpp_2m_sde (shared Brownian path); at CFG 8 and 1216x832 the milder EMA gave colour casts and damped extrapolation did not. One evaluation per step beats two at equal NFE: the best one-evaluation SDE variant is 16-29% better than the two-evaluation dpmpp_sde at half the steps. Of the nine registered hypotheses, four were supported (positive momentum only hurts; time-scaled form not better than the best constant for a positive coefficient; LPIPS agrees with latent error; one evaluation beats two) and five were refuted (including that the stacked correction is the cause of the failure). A static step grid from an earlier study cut the error by half on the prompts it was fitted on and transferred across prompts of the same family, but it did not help a held-out natural-language prompt and broke an 8-step picture at CFG 8; it is shipped as an experimental node only (post hoc analysis). Prior work. Momentum in diffusion sampling is not new: Heavy Ball momentum inside existing numerical solvers to enlarge their stability regions (Wizadwongsa et al., ICLR 2024, arXiv:2307.11118), an Adam-style momentum sampler (Wang et al., arXiv:2308.11941), momentum for peculiar images (Jang et al., AAAI 2024), TADA (arXiv:2506.21757), and the ComfyUI packs ComfyUI-sonar and Seren Sampler. Only abstracts and READMEs were read; the code of these packages was not run and no comparison was made. What we could not find is an explanation of why the shipped sampler fails on a flow model, and a measurement of the sign and size of EMA momentum against converged references at equal evaluation count; this is a statement about our search, not about the field. Software. Exact port of the original sampler, the new samplers, ComfyUI nodes (a momentum sampler node, a filter node that wraps any one-evaluation sampler, a sigma-grid node), KSampler entries, 13 CPU tests, all scripts, tables (results/) and figures are in the repository: https://github.com/YakovSava/momentum-cfg-samplers-research. The README (English, Russian, Chinese, Meadow Mari) is the full record with the hypothesis table and all intervals. The filter wrapper for 19 additional samplers was added as an engineering extension and has not been evaluated on the model. Limitations. One model, one latent size and CFG in the main sweeps; effects below about 5% are not detectable; a 3-seed check at CFG 8 cannot resolve differences below about 15%. The main metric is relative L2 distance of the final latent to a converged solution of our own (about 5% own error); LPIPS only for the deterministic winners at 8 and 16 steps. Strong error of an SDE sampler is not image quality. The sweep sections beyond the registered grid, the grid transfer analysis and the CFG 8 check are post hoc. The Meadow Mari translation is machine-assisted and not reviewed by a native speaker.

Zenodo (CERN European Organization for Nuclear Research)
Generative Adversarial Networks and Image Synthesis
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon