- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies training-free acceleration of flow-matching diffusion transformers, where full-Transformer (FT) evaluations are performed only at selected timesteps and the velocity at the remaining steps is forecast as a calibrated linear combination of recent velocities (as in L2P). The authors argue that error accumulates because forecasted velocities never leave the span of the historical velocities. They propose SGSE, which also linearly forecasts the next sampling state, maps it through the inverse sampler update to a “state-implied velocity”, projects it onto the orthogonal complement of the history span, and adds it with a per-timestep calibrated scale γ_i. All coefficients are calibrated offline on 50 FT trajectories. A projection argument shows that the minimum achievable error decreases by ‖g_i‖²A_i². Experiments on FLUX.2 Klein 4B (DrawBench) and Wan2.1-1.3B (VBench) show modest fidelity gains over L2P, TaylorSeer, VDE and MeanCache at matched FT budgets, with ablations on FLUX.2 under the 11-FT schedule.
Critical strengths
- The diagnosis is clear and well supported. The paper decomposes the forecasting error into in-span and out-of-span parts (Eq. 6, Fig. 2) and shows that linear forecasting is limited by the history span. This gives a useful lens for the growing family of forecasting-based caching methods.
- The method is simple and cheap. It adds only one calibrated scalar per timestep, requires no retraining, and yields consistent fidelity gains (SSIM/PSNR) over the strongest baseline L2P at all FT budgets on both an image and a video model, with negligible overhead.
Critical weaknesses
- The “state forecasting” narrative obscures what the method actually adds, and the key baseline is missing. Appendix E shows that under Euler, SGSE reduces exactly to , and directly calibrating η_i performs almost identically (Table 7). Since equals the initial noise plus the accumulated past velocities, is essentially the component of the initial noise, together with velocities older than the window, that lies outside the history span. In the “All” window setting (Table 6), it reduces to the out-of-span component of the initial noise alone. Under rectified flow, the velocity is closely tied to the initial noise and the current state, so this is a natural extra basis vector. The paper should therefore compare against the straightforward baseline of adding the initial noise ε and/or the current state as additional regression bases. Without this comparison, both the role of state forecasting and the novelty relative to L2P are unclear.
- Gains are small, measured only as fidelity to FT, and the evaluation is narrow. Over L2P, SSIM improves by about 0.01, LPIPS is slightly worse at 9 FT on FLUX.2, and VBench is lower than L2P at both budgets on Wan2.1. A random Gaussian direction with FT calibration already recovers part of the gain (0.8294 vs. 0.8265 without guidance and 0.8358 for SGSE; Table 5), and the alignment A_i is small (about 0.05–0.1) for most intermediate timesteps (Fig. 2d). The coefficients are shared across all samples and calibrated on only 50 prompts, yet no robustness to prompt distribution, CFG scale, resolution or step count is tested. No variance across seeds or per-sample worst-case errors are reported, and all ablations are on FLUX.2 with 11 FT only.
Questions for authors
- How does SGSE compare with simply adding the initial noise ε, or the current state , as an extra basis vector in the calibrated linear forecaster, on both FLUX.2 and Wan2.1? If these baselines perform comparably, what does the state forecast contribute?
- Can you show the calibrated γ_i values across timesteps and repeat the key ablations (Tables 3 and 5) on Wan2.1 with UniPC? Fixing γ_i = 1 performs much worse than no guidance at all (0.7693 vs. 0.8265), which suggests the direction is unreliable without per-step rescaling.
5 comments
Response to W2 part1
W2: ① Gains are small, ② measured only as fidelity to FT, and ③ the evaluation is narrow.
We appreciate your detailed comments on the performance gains, evaluation, and robustness of SGSE. Below, we provide point-by-point responses to your eight concerns.
① We conducted additional evaluations using different generation seeds (page 15, lines 795–809), providing further evidence of the stability of SGSE's performance gains. As shown in Table R3.2, SGSE achieves higher mean SSIM and PSNR than L2P across all five evaluated configurations on FLUX.2 and Wan2.1, along with lower mean LPIPS in four of them. The evaluation metrics also exhibit relatively limited variation across generation seeds. These results further highlight the value of state-guided span expansion: leveraging complementary information beyond the historical velocity span at minimal computational cost offers a valuable direction for further improving generation quality in efficient diffusion inference.
Table R3.2: Multi-seed evaluation (mean ± standard deviation over three seeds). Results are from Tables 7 and 8 (page 15, lines 757–775).
| Model | FT | L2P SSIM ↑ | SGSE SSIM ↑ | L2P PSNR ↑ | SGSE PSNR ↑ |
|---|---|---|---|---|---|
| FLUX.2 | 18 | 0.8728 ± 0.0046 | 0.8791 ± 0.0044 | 22.2788 ± 0.2201 | 22.5673 ± 0.2139 |
| FLUX.2 | 11 | 0.8141 ± 0.0038 | 0.8262 ± 0.0041 | 20.2964 ± 0.0384 | 20.5455 ± 0.0778 |
| FLUX.2 | 9 | 0.7640 ± 0.0035 | 0.7723 ± 0.0031 | 18.3504 ± 0.0662 | 18.5022 ± 0.0817 |
| Wan2.1 | 18 | 0.7536 ± 0.0030 | 0.7628 ± 0.0022 | 20.4336 ± 0.0413 | 20.6845 ± 0.0387 |
| Wan2.1 | 11 | 0.6401 ± 0.0040 | 0.6445 ± 0.0030 | 17.6226 ± 0.0505 | 17.7511 ± 0.0870 |
② Following prior work L2P, we adopted SSIM, PSNR, and LPIPS to evaluate fidelity, ensuring a consistent comparison. In addition, we introduced CLIPScore and VBench to evaluate generation quality. We also measured inference latency and floating-point operations (FLOPs) to analyze the trade-off between quality and efficiency, as shown in Fig. 4 (page 8, lines 378–390). Furthermore, we provided qualitative visual comparisons of representative samples in Figs. 5 and 6 (page 9, lines 432–466) and Appendix D (pages 16–17, lines 858–863). Thus, our evaluation covers fidelity, generation quality, inference efficiency, and qualitative results, rather than focusing solely on fidelity.
③ In the original manuscript, we evaluated SGSE on both text-to-image and text-to-video generation in terms of fidelity, generation quality, inference efficiency, and qualitative results. We also examined performance across different FT budgets and conducted ablations on key components, method mechanisms, and history-window size. In the revised manuscript, we further extended our evaluation by adding comparisons with reduced-step FT on both models (Tables 1 and 2, page 7, lines 325–366), component and mechanism ablations on Wan2.1 with Flow-UniPC (Tables 4 and 11, pages 10 and 17, lines 496–503 and 892–902), multi-seed evaluations and per-sample worst-case analysis (Tables 7–9, page 15, lines 756–790), and direct current-state and initial-noise span expansion baselines under Euler and Flow-UniPC (Tables 12 and 13, page 20, lines 1026–1041). These additional experiments provide further evidence of SGSE's effectiveness across different generation tasks, FT budgets, random seeds, and samplers, while offering a more comprehensive evaluation of state-guided span expansion.
Response to W2 part2
W2: ④ A random Gaussian direction with FT calibration already recovers part of the gain (0.8294 vs. 0.8265 without guidance and 0.8358 for SGSE Table 5), and ⑤ the alignment A_i is small (about 0.05–0.1) for most intermediate timesteps (Fig. 2d). ⑥ The coefficients are shared across all samples and calibrated on only 50 prompts, yet no robustness to prompt distribution, CFG scale, resolution or step count is tested.
④ A random Gaussian direction with FT calibration can yield performance gains, which is consistent with our theoretical analysis that expanding the historical velocity span can reduce the approximation error even when the additional direction exhibits weak alignment with the missing velocity component. Specifically, a random direction may have a nonzero projection onto the missing velocity component, which can be exploited through calibration. In the FT-calibrated Gaussian, we replace the current state in Eq. (10) with a random Gaussian vector while retaining the subsequent inverse sampler update, orthogonal projection, and FT calibration. The timestep-specific coefficient is learned to adjust the sign and magnitude of the resulting correction direction, allowing the baseline to exploit its limited complementary information. However, state guidance achieves better results. The SSIM for velocity-only forecasting, FT-calibrated Gaussian guidance, and SGSE are 0.8265, 0.8294, and 0.8358 on FLUX.2, and 0.6438, 0.6440, and 0.6450 on Wan2.1, respectively (Tables 10 and 11, page 17, lines 879–902). These results show that calibration can partially benefit from an additional direction, but the complementary information provided by sampling states leads to better performance.
⑤ measures the directional alignment between the state-guided direction and the missing velocity component. According to Eq. (19) (page 6, lines 304–310), the reduction in the minimum achievable squared approximation error is Thus, the theoretical improvement depends on both the alignment and the magnitude of the missing velocity component . Even when is small, nonzero alignment indicates that the state-guided direction can capture part of the missing velocity component beyond the historical velocity span.
⑥ We followed prior work L2P in using shared calibration coefficients and a small calibration set. L2P specifically studied the effects of calibration set size. We adopted its 50-sample setting for a fair comparison. Moreover, our calibration and evaluation sets are disjoint, providing evidence of cross-dataset generalization. Specifically, FLUX.2 was calibrated on 50 PartiPrompts samples and evaluated on DrawBench, while Wan2.1 was calibrated on 50 FETV prompts and evaluated on VBench, as described in Section 4.1 (page 8, lines 398–402). We followed the evaluation settings of image generation used in L2P, including CFG scale, resolution, and total sampling steps, and evaluated all methods under the same configurations. We agree that further robustness evaluations of the calibration coefficients across different prompt distributions, CFG scales, resolutions, and sampling-step counts would be valuable. However, we cannot complete all these additional experiments within the rebuttal period. We will investigate these settings and report more detailed results in future work.
Response to W1
W1: The paper should compare against the straightforward baseline of adding the initial noise ε and/or the current state as additional regression bases.
Thank you for your comment. State forecasting provides a general method to construct the state-guided direction under different samplers (like Flow-UniPC), without relying on a state update rule specific to the Euler sampler. Compared with L2P, which forecasts velocities using linear combinations of historical velocities, our contribution is to identify the representational limitations of this approach and expand the historical velocity span with complementary information from sampling states.
We added two baselines that separately use the current state zi and initial noise ϵ as additional basis vectors. As shown in Table R3.1, both direct baselines achieve performance close to SGSE on FLUX.2, consistent with our analysis. However, on Wan2.1 with Flow-UniPC, SGSE outperforms both direct baselines across all four metrics. Moreover, both direct baselines perform worse than velocity-only forecasting, whereas SGSE achieves better results. These results show that directly using the current state or initial noise as an additional direction does not improve performance under Flow-UniPC, while SGSE consistently improves performance.
Table R3.1: Comparison of velocity-only forecasting, direct current-state and initial-noise span expansion, and SGSE. Velocity-only results are from Tables 3 and 4 (page 10, lines 487–503), while direct-state and direct-noise results are from Tables 12 and 13 (page 20, lines 1027–1041).