- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes Flow-Guided Wasserstein Compensation Distillation (FWCD), which introduces a distribution-aware correction to existing flow distillation guidance using unpaired reference samples. The method learns a scalar potential to estimate the correction field and incorporates it into student training. The paper provides theoretical analysis and evaluates the approach on ImageNet-256 and SANA-1.6B, reporting improvements over Mean Flow Distillation (MFD) under four-step inference.
Critical strengths
S1. Empirical improvements over MFD. The reported results indicate some improvements in generation quality on ImageNet-256 and SANA-1.6B, suggesting the potential usefulness of distributional compensation for few-step distillation.
Critical weaknesses
Overall, the paper lacks sufficient clarity and coherence in its presentation. The motivation, significance, and intended contributions of the proposed approach are not clearly established, and the roles and connections of individual sections are difficult to follow.
Furthermore, the purposes of different baselines and experimental settings, as well as the interpretations of the reported results, are insufficiently explained.
As a result, understanding the paper often requires readers to infer connections and assumptions that should be explicitly communicated. These issues substantially affect the overall readability and completeness of the manuscript.
W1. Insufficient motivation and problem formulation. The paper does not clearly explain the practical limitations of existing methods, why the prescribed density evolution is necessary, or how the proposed formulation addresses these limitations.
W2. Unclear notation and terminology. Several quantities (e.g., , and student particles) are introduced without sufficient definitions. The use of multiple related notations further complicates understanding.
W3. Lack of coherent methodological explanations. The transitions between flow guidance, density constraints, potential learning, and student optimization are insufficiently motivated. Readers must infer why individual components are introduced and how they relate to one another.
W4. Insufficient experimental setup and evaluation scope. The main paper does not clearly establish all baseline configurations, experimental protocols, or the meaning of R1–R5. Furthermore, evaluation is restricted to NFE=4, leaving generalization across inference budgets unexplored.
W5. Limited interpretation of experimental results and figures. The paper provides limited explanations of the purpose and findings of individual ablations. In particular, Figure 3 lacks meaningful qualitative analysis, and the implications of the differing performance between FWCD and Additive D0 are not sufficiently discussed.
Questions for authors
Please refer to the weaknesses outlined above.
3 comments
Dear Reviewer,
The discussion deadline is approaching. Please review the authors’ responses and clearly indicate whether they adequately address your concerns. Kindly update your score if appropriate.
Best regards,
AC
Response to Authors
We thank the authors for the additional comments and experiments. However, the following points do not yet appear to be fully addressed.
1.The implication of the gap between FWCD and Additive D0 remains unclear. (W5)
- In Tables 1, 3, 10, and 16, Additive D0 often appears competitive with the proposed β-objective on ImageNet. The manuscript does not explain why, or when each objective should be preferred.
2. Generalization across inference budgets is still not established. (W4)
- Evaluation remains limited to 4 steps. 1 and 2 step scores appear to be at a level where the models themselves are not properly trained, rather than being directly comparable. The response also does not explain why the ranking reverses between 1 and 2 steps, or why the variance across seeds is so large.
For these reasons, I will maintain my score.
I hope this feedback is helpful for the next version of the manuscript.
Response to Reviewer ZC4J
We thank the reviewer. We clarified the motivation, notation, training workflow, and experiments.
W1. Motivation
MFD supplies guidance at student-generated queries, whose distribution can differ from the reference early in training. FWCD prescribes linear relaxation toward the reference and learns the response still required after the scaled MFD field. This choice permits estimation from unpaired sample expectations and explicit ideal dynamics with squared-Wasserstein contraction.
Revision: Sections 1 and 3.1–3.2.
W2. Notation
vj(x,t,c) is model j's instantaneous velocity at query x, noise time t, and condition c. With flow map Φr←tj, mean velocity is
\begin{cases} [x-\Phi^j_{r\leftarrow t}(x,c)]/(t-r),&r<t,\\ v_j(x,t,c),&r=t. \end{cases}$$ The base is $b=u_{\rm aux}-u_{\rm teacher}$; $g_t$ is total guidance. A student particle is a sample from the student query law. Noise time $t$, ideal evolution time $\tau$, and discrete optimizer updates are distinct. *Revision: Section 2; Appendix A, Table 4.* ### W3. How the method fits together 1. **Base:** MFD supplies direction $b$. 2. **Target:** For student/reference laws $\mu,\nu$, the constraint $-\nabla\cdot(\mu g)=\lambda(\nu-\mu)$ prescribes their distributional response. Its weak form, $\mathbb E_\mu\langle g,\nabla f\rangle=\lambda(\mathbb E_\nu f-\mathbb E_\mu f)$, uses unpaired samples. 3. **Potential:** Minimum-energy fitting gives $g_\beta[h]=(\beta b+\nabla h)/(1+\beta)$. The potential fits the target response minus that already supplied by the scaled base. 4. **Student:** We read the EMA potential's correction, apply magnitude control, and hold guidance fixed during backpropagation through the student query. ImageNet separately generates fitting/update queries. SANA fits on 16 queries; the first is detached during fitting and reused with its retained graph for the student update. *Revision: Sections 3.2–3.4 and 4, Figure 2; Appendix E.5.* ### W4. Experimental settings and inference steps R1–R3 are the main ImageNet runs; R4–R5 are matched-relative-magnitude replay runs. Seeds are listed in Appendix F. Original ImageNet comparisons use SiT-XL/2, qkv LoRA 64/64, 560 student updates, and four-step inference, with method-specific auxiliary/discriminator schedules and samplers. Original SANA training uses 272 student and 2,728 auxiliary updates. One- and two-step SANA models are trained separately for 3,000 global updates with three seeds. Evaluation shares 5,000 prompts, noises, and teacher references; FWCD uses natural magnitude and $\beta=9$. | Inference steps | MFD FID ↓ | FWCD FID ↓ | |---|---:|---:| | 1 | **257.948 ± 12.996** | 304.223 ± 8.839 | | 2 | 383.635 ± 21.679 | **86.592 ± 42.250** | Values are means ± sample SD. MFD has lower one-step FID in all pairs; FWCD has lower two-step FID and higher scores on all four image–text metrics in all pairs. A single-seed check changes only the prompt prefix and preserves both rankings. *Manuscript: Section 7.1; Appendices F and G.1/G.5. NFE and prefix-check results are supplied in this response.* ### W5. Ablations and Figure 3 Attenuation tests base scaling: ImageNet FID changes from **4.551 to 4.549**. D0/D9 test objectives on the same $0.9b$ base and 10% cap: FIDs are **4.059/4.222** at $M=5$ and **4.134/4.061** at $M=1$. Shuffle tests assignment, raising D9 FID from **4.222 to 4.442**. Appendix I.4 quantifies objective/readout and correspondence contributions separately; objective preference depends on the setting. Figure 3 uses shared classes/noise at eight evenly spaced evaluation indices. Its caption identifies the unresolved small foreground face in column (e), enlarged in **Figure 10(a)**. Quantitative results use the full evaluation set; selected images do not estimate failure rates. *Revision: Figure 3 caption in Section 7.2; Sections 7.3–7.4, Tables 1/3; Appendix I.4 (Table 16, Figure 8); J.4 (Figure 10(a)).*