- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper incorporates the speculative decoding in autoregressive models to the sampling of flow-matching based generative models. By re-introducing the diffusion score and schedule, the stochastic term is added back to the flow-based models, so that the speculative decoding can be applied. Experiments across different setups such as text-to-image, image editing, and video generation are conducted, with wall-clock computational efficiency provided to demonstrate the effectiveness of the SpecFlow.
Critical strengths
The idea of incorporating the autoregressive speculative decoding is novel in the flow-matching based model regime.
Critical weaknesses
See the questions down below.
Questions for authors
Here are my questions:
- Based on the setup of the paper, it is natural to apply the speculative decoding to noise-based or score-based diffusion models, instead of the flow-based models; is there any reasons the flow-based models were chosen to be the primary backbone models?
- Several classical baseline models are missing from the paper, for example the DPM-Solver++ [1] and the UniPC [2] solvers; it is beneficial to compare to at least one of them, in at least one of the setups (eg. text-to-image, image editing, etc.).
- I'm a bit surprised that the Euler–Maruyama solver attains the same image quality as the Euler's method across NFE=5, 10, 25, 50, as shown in Table 1. Although the marginal probability distributions are the same, the stochastic differential equation solver needs more timesteps (therefore more NFEs) than the ODE solver to sample stably due to the Wiener process term; one not identical but similar case example is DDIM [3] (Euler) v.s. DDPM [4] (Euler–Maruyama). Can you provide an intuitive explanation of why would the SpecFlow be able to avoid this issue? If I understand correctly, the speculative decoding should only accelerate the sampling but not improve the quality under the same budget.
[1] Lu, Cheng, et al. "DPM-Solver++: Fast solver for guided sampling of diffusion probabilistic models." Machine Intelligence Research 22.4 (2025): 730-751.
[2] Zhao, Wenliang, et al. "UniPC: A unified predictor-corrector framework for fast sampling of diffusion models." Advances in Neural Information Processing Systems 36 (2023): 49842-49869.
[3] Song, Jiaming, Chenlin Meng, and Stefano Ermon. "Denoising diffusion implicit models." arXiv preprint arXiv:2010.02502 (2020).
[4] Ho, Jonathan, Ajay Jain, and Pieter Abbeel. "Denoising diffusion probabilistic models." Advances in neural information processing systems 33 (2020): 6840-6851.
5 comments
Gentle Reminder
Dear Reviewer TFqq,
It is a gentle reminder to acknowledge our rebuttal and, if there are no further concerns, we request to increase the scores.
Best Regards,
Authors
Thanks the authors for the reply. However, for the DPM-Solver++ baseline comparisons in the Q2 response, I'm still not convinced.
- It seems like the DPM-Solver++ at 20 NFE is suspiciously worse than the vanilla Euler at 20 NFE for most of the metrics. Could the authors kindly provide the configs passed to the DPM-Solver++ experiment (I suppose the HuggingFace Diffuser version of DPM-Solver++ is used)? I'm a bit afraid that some mis-configurations caused unfair comparisons.
- Do you mind providing the LPIPS and PSNR values for the Full-50 experiment as well?
Rebuttal
We thank the reviewer for recognizing the novelty of applying speculative decoding to flow-matching models and for the constructive questions. We address each point below.
Q1. Why focus on flow-matching models rather than score-based diffusion models?
A1: We focus on Flow Matching (FM) because extending exact speculative decoding (SD) to its standard deterministic sampling procedure presents a distinct challenge that is not present in stochastic diffusion samplers. There are existing methods that have extended SD to diffusion and score based models but the major concern why it was not extendable to FM is due to deterministic paths from source distribution to target distribution.
As explained in Section 3.2 (Eq. 6), deterministic FM transitions are Dirac measures. Consequently, conventional likelihood-ratio verification cannot meaningfully accept a proposed next state unless it exactly matches the deterministic transition. In contrast, score-based diffusion samplers already have non-degenerate stochastic transitions, for which exact speculative sampling methods have been developed (Section 2).
Our contribution is to bridge this gap for pretrained FM models. We construct a marginal-preserving stochastic realization of the FM probability flow (Section 3.3, Eq. 7), derive the required Gaussian target transitions (Eq. 10), and recover the score estimate directly from the pretrained FM velocity (Eqs. 8–9), without an auxiliary score network or retraining.
Also, This is particularly relevant because modern image, editing, and video generators, including BAGEL, FLUX, and HunyuanVideo, use flow-based architectures. Our framework addresses these existing pretrained models rather than requiring a separately trained stochastic generator.
The underlying speculative verification principle can also be applied to score-based diffusion models with evaluable stochastic transitions. However, the deterministic nature of standard FM sampling is precisely the additional challenge addressed by SpecFlow.
Q2. Can you compare against classical higher-order solvers ...
A2: Higher-order solvers are relevant baselines, but they address a different computational objective. Methods such as DPM-Solver++ and UniPC improve numerical integration efficiency, whereas SpecFlow targets sequential sampling dependencies through speculative drafting and verification.
To address the requested numerical comparison, we additionally evaluated DPM-Solver++(2M) on BAGEL-7B-MoT at 1024×1024, using all 553 GenEval prompts and identical initial latents. We compare fidelity relative to deterministic Full-50.
At comparable speedup, SpecFlow achieves substantially lower LPIPS and higher PSNR than DPM-Solver++(2M). This indicates closer per-image agreement with Full-50.
We will include this baseline and clearly distinguish the two verification settings in the revision.
Q3. Why does Euler–Maruyama appear to maintain ...
A3: We would like to clarify that similar generation quality does not imply that deterministic Euler and Euler--Maruyama (EM) produce identical trajectories or distributions at finite NFEs. Our stochastic formulation is constructed to preserve the marginal distributions of the original FM probability flow in continuous time, under the ideal velocity and score assumptions (Appendix A.1.1). At finite sampling steps, discretization and learned-velocity errors can introduce differences between the two samplers.
Importantly, Table 1 compares Full-N sampling with SpecFlow-N, rather than directly comparing deterministic Euler and serial EM. The observed similarity in generation quality therefore should not be interpreted as numerical equivalence between the two solvers.
We already investigate this distinction in Appendix B.2 (Table 6), where deterministic and stochastic sampling exhibit close empirical agreement at finer discretizations, and Appendix B.5 (Table 9), which shows that larger diffusion scales can degrade stochastic-target quality.
Thus, the observed quality preservation is consistent with our stochastic construction, but is not assumed or guaranteed at arbitrary finite NFEs. SpecFlow's exactness guarantee applies specifically to its chosen discretized stochastic target, while its agreement with deterministic FM is evaluated separately.
Final Remarks
We sincerely thank the reviewer for their constructive feedback and thoughtful evaluation. Based on these clarifications and the additional evidence, we respectfully ask the reviewer to reconsider their assessment and increase their score.