- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
DifferenceFlow introduces a training curriculum for one-step generative models built on MeanFlow. It derives a target from a finite-difference approximation of the MeanFlow identity, replacing AlphaFlow’s mixing weight with . The curriculum progressively reduces and ultimately switches back to JVP-based MeanFlow training. Its analysis shows that the proposed target cancels a leading velocity-gap bias and achieves second-order accuracy in interval length under the stated assumptions. Evaluation covers Swiss-roll and ImageNet-256×256, including training-recipe ablations and baseline comparisons. DiT-B/2 achieves a 1-NFE FID of 5.41, improving to 2.29 with additional FD-loss fine-tuning.
Critical strengths
-
Principled target design and error analysis. The paper derives its mixing coefficients from MeanFlow identity and shows that they cancel a leading velocity-gap bias present in AlphaFlow’s target under the stated assumptions. This provides a clear technical rationale for the modification and improves understanding of target accuracy in MeanFlow-based formulation.
-
Empirical gains before additional fine-tuning. Using DiT-B/2, the method achieves an ImageNet 1-NFE FID of 5.41, compared with 5.60 for AlphaFlow and 6.14 for MeanFlow in the authors’ reproductions, with improvements across several feature-space metrics. These results provide evidence that the proposed curriculum has practical value independently of the additional FD-loss fine-tuning.
Critical weaknesses
-
Limited novelty and an insufficient connection between theory and final performance. DifferenceFlow primarily modifies AlphaFlow’s finite- target weights and ultimately switches back to JVP-based MeanFlow for final optimization. The ideal-target bias analysis does not establish whether the proposed correction explains the gains after this shared MeanFlow stage, particularly with bootstrapped predictions and adaptive loss weighting. This limits the methodological advance and leaves the practical significance of the theoretical correction insufficiently demonstrated.
-
Incomplete comparisons with relevant recent methods. The controlled experiments focus on MeanFlow and AlphaFlow with DiT-B/2, omitting relevant developments such as iMF. Although Table 3 includes broader benchmark results, it does not provide matched training and fine-tuning controls. In particular, the best FID of 2.29 uses additional FD-loss fine-tuning without corresponding MeanFlow or AlphaFlow controls. These omissions weaken both the assessment of competitiveness and the attribution of the strongest results to the proposed target.
Questions for authors
- For Table 4, were training budgets, time sampling, flow-matching ratios, and loss weighting matched across methods? Did AF and DF use identical schedules and the same duration of final JVP-based MeanFlow training?
- How consistent is the FID improvement from 5.60 to 5.41 across random seeds?
- Were AF and MF evaluated with the same FD-based fine-tuning as DF? If so, what FIDs did they achieve?
- What practical advantages does the proposed formulation offer over Improved MeanFlow and pixel MeanFlow, and what evidence supports them?
2 comments
Thank you for the clarification and additional experiments. The matched FD fine-tuning comparison addresses one of my concerns: DF retains an advantage over AF and MF when all three receive the same fine-tuning. For Q3, could you also report iMF’s 1-NFE FID on ImageNet-256 before and after applying the same FD fine-tuning protocol, ideally at a comparable model size and training budget? This would help assess whether DF’s advantage extends to a stronger baseline under the same fine-tuning conditions.
The CIFAR-10 comparison is less convincing. DF improves over iMF from 19.63 to 18.84 FID in your matched setup, but the original MeanFlow paper reports 2.92 FID for unconditional CIFAR-10 at 1 NFE (Table 3). I understand that the training setups may differ. Could you specify the backbone, training budget, and FID evaluation protocol, and clarify whether the advantage persists in a setup that reproduces a competitive MeanFlow baseline?
The matched AF–DF comparison supports the practical value of changing the early target weights. My remaining concern is whether the bias cancellation in the theoretical analysis explains the gains observed after both methods switch to the same JVP-based training schedule.
We thank the reviewer for the careful review and constructive feedback. We appreciate your recognition of our principled target design and error analysis, particularly the cancellation of the leading velocity-gap bias under the stated assumptions. We also appreciate your acknowledgment of the empirical gains across multiple feature-space metrics before additional FD-loss fine-tuning. We address your concerns below. All changes in the revised manuscript are highlighted in red.
Q1: Fairness of training configurations in the AF–DF comparison.
A: For comparisons within the same training setting in Table 4, training budgets, time sampling, flow-matching ratios, and loss weighting are matched. AF and DF use the same α schedule and switch to JVP-based MeanFlow training at the same stage, with identical final JVP training durations. Thus, the primary difference is the target interpolation coefficients during finite-difference training.
Q2: Multi-seed consistency of FID improvements.
A: In additional evaluations, we found an existing DifferenceFlow checkpoint achieving a lower FID than the originally reported 5.41. Keeping this checkpoint fixed and varying only the sampling seed, DifferenceFlow achieved 5.2406±0.0436 (mean ± SD) across ten seeds with 50,000 samples each (Table R3-Q2). Scores ranged from 5.1851 to 5.3085, all below AlphaFlow's reported 5.60. AlphaFlow's FIDs across sampling seeds were also consistent with its reported value. These evaluations demonstrate robustness to sampling randomness, but not to independent training seeds.
Table R3-Q2: FID across different seeds.
Mean ± SD: 5.2406±0.0436
Q3: Matched Fréchet-distance fine-tuning for AF, MF, and DF.
A: We applied identical Fréchet-distance fine-tuning to AF, MF, and DF. As shown in Table R3-Q3, DF achieves an FID of 2.29, outperforming AF (2.49) and MF (2.67). DF also leads on 15 of 16 metrics across multiple representation spaces, confirming its advantage under matched fine-tuning.
Table R3-Q3: Comparison after Fréchet-distance fine-tuning under the B/2+ setting with CFG.
Q4: Practical advantages over Improved MeanFlow and pixel MeanFlow.
A: Improved MeanFlow and pixel MeanFlow use larger training scales and different configurations, making fully fair comparisons challenging. Matched experiments would require retraining and tuning beyond our computational resources. Our contribution combines finite-difference targets with a dynamic curriculum that could extend to other MeanFlow-based methods. Progressively shrinking the difference interval transitions training from explicit velocity supervision toward local MeanFlow constraints, forming an easy-to-hard process intended to mitigate gradient conflicts and improve optimization. Our MF/AF comparisons and curriculum ablations support this strategy.
For a controlled comparison, we evaluated DifferenceFlow and Improved MeanFlow on unconditional CIFAR-10 with identical model sizes and training budgets (Table R3-Q4). At 1 NFE, DifferenceFlow achieves an FID of 18.84, outperforming Improved MeanFlow (19.63), providing additional evidence under matched conditions.
Table R3-Q4: Unconditional generation performance on CIFAR-10.