acceptodds
Withdrawn from ICLR 2027

GFD-OPD: Guidance-Folded On-Policy Distillation of Diffusion Models Across Scales

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

On-policy distillation (OPD) has demonstrated two important capabilities in language models: compressing large teachers into smaller students and merging expert models into a single model. Existing diffusion OPD, however, mostly focus on the latter, with teachers and students sharing the same backbone and scale. We investigate large-to-small diffusion opd from large teachers to a small student and find that the standard recipe fails. To find the underlying cause, we propose Fixed-State KL, an effective and fair way to measure the distribution gap between student and teacher during OPD training for diffusion models. We are the first to clarify why large-to-small OPD is challenging for diffusion models: a smaller student struggles to perfectly match the distribution of a larger teacher, while classifier-free guidance can accumulate and amplify the distributional discrepancies between the student’s conditional and unconditional branches and those of the teacher. To solve this problem, we propose GFD-OPD, a simple yet effective method that reduces the student–teacher gap while avoiding the error amplification of the CFG composition. Across numerous experiments, GFD outperforms previous baselines in both training efficiency and final performance, achieving state-of-the-art results on all benchmarks.

Primary area:
generative models
Keywords:
Diffusion, On-Policy-Distillation, Text to image

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_joz1OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies on-policy distillation (OPD) of diffusion models from a large teacher to a smaller student, and finds that the standard DiffusionOPD recipe produces severe artifacts in this setting. To diagnose this, the authors propose Fixed-State KL, which evaluates the student–teacher KL divergence on a shared set of samples from teacher trajectories and noised real images. Their analysis attributes the failure to classifier-free guidance (CFG): a smaller student cannot match the teacher's conditional and unconditional branches, and the CFG composition amplifies their errors. They propose Guidance Folding Distillation (GFD), which trains the student's conditional branch to directly match the teacher's CFG-guided velocity, combined with a reward-extrapolated target adopted from ExOPD. Empirically, GFD-OPD achieves promising performance on SD3.5 (Large to Medium) and FLUX.2 (9B/32B to 4B) over OPD baselines.

Critical strengths

The paper studies an important and practical setting, large-to-small on-policy distillation of diffusion models.

Critical weaknesses
  • Limited Novelty: My main concern lies in the actual contribution of this paper. Training the student's conditional branch to match the teacher's CFG-guided output is essentially guidance distillation [1], which is well established in the diffusion distillation literature, yet the paper neither cites nor discusses it. The reward-extrapolated target is directly adopted from ExOPD [2]. In this regard, GFD-OPD reads as combining guidance distillation [1] and ExOPD [2] within the DiffusionOPD framework. The paper would benefit from positioning the method with respect to the guidance distillation literature and clarifying what is new in the on-policy, large-to-small setting.
  • Inconsistency between Motivation and Deployment: The method is motivated by removing the CFG composition so that "nothing is left to amplify". However, GFD is deployed with CFG at w=2w=2, where the unconditional branch is never trained, which reintroduces the composition the method aims to avoid. The claim that the unconditional branch "co-evolves" with the conditional branch is only asserted, not examined.
  • Fairness of Comparison: The baselines are evaluated at w=4.5w=4.5, while GFD-OPD uses w=2w=2 and an extrapolated target with λ=1.25\lambda=1.25. Since reward extrapolation is orthogonal to guidance folding, it could also be applied to the baselines. For a fair comparison, the baselines should also be evaluated with their best guidance scale on the task rewards and with the same extrapolated target.

[1] Meng, Chenlin, et al. "On distillation of guided diffusion models." CVPR (2023).

[2] Yang, Wenkai, et al. "Learning beyond teacher: Generalized on-policy distillation with reward extrapolation." arXiv preprint arXiv:2602.12125 (2026).

Questions for authors
  • In Table 5, the "Standard OPD" row (on the SD3.5-Medium student) does not match the DiffusionOPD L→M results in either Table 2 or Table 3. Could you clarify the setting of this ablation and the difference between "Standard OPD" and DiffusionOPD?
  • Since GFD trains the conditional branch to absorb the guidance, it can be deployed at w=1w=1 with a single forward pass, halving the inference cost. Could you report the task rewards and image quality at w=1w=1 in the main tables?
  • How do DiffusionOPD, PDM and split-KL perform when combined with the same extrapolated target (λ=1.25\lambda=1.25)? Table 5 adds extrapolation only on top of guidance folding, so it remains unclear how much of the gap to the baselines in Tables 2–4 is due to extrapolation alone.
Reviewer_g2m4OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presented On-Policy Distillation (OPD) for text-to-image rectified flow models in the cross-scale regime, i.e., distilling a large teacher into a smaller student. The authors claimed that recent two-branch diffusion OPD methods suffer from severe visual degradation when applied across model scales.
First, the authors provide a diagnostic analysis showing that two-branch OPD relies on error cancellation between the unconditional error and the conditional increment error, which breaks down when the student is capacity-constrained and gets amplified by the CFG weight. Second, to bypass this amplification, they propose "Guidance Folding Distillation (GFD)", training the student's conditional branch directly on the teacher's CFG-guided velocity, alongside "Fixed-State KL" for evaluation and "Reward Extrapolation (ExOPD)" (a linear extrapolation between a teacher and a reference model's velocities) to further boost performance on SD3.5-Medium and FLUX.

Critical strengths
  1. The mathematical decomposition of the guided error and the empirical cross-term analysis clearly explain why two-branch on-policy distillation works for same-scale alignment but collapses in cross-scale distillation,

  2. Strong Final Empirical Checkpoint: Combining on-policy rollouts with single-branch CFG distillation and velocity extrapolation achieves strong quantitative metrics on SD3.5-Medium and FLUX benchmarks.

Critical weaknesses
  1. I feel that there is an over-claimed novelty of "Guidance Folding" (GFD): The paper frames matching the student's single conditional branch directly to the teacher's CFG-guided output as a primary methodological contribution. However, folding the teacher's CFG prediction into a single student pass is standard guidance distillation and is already the default paradigm in modern diffusion distillation. Applying standard guidance distillation within an on-policy rollout loop is a sensible engineering fix for DiffusionOPD, but it is difficult to classify as a novel algorithmic contribution.

  2. Marginal Empirical Impact of the Remaining Method (ExOPD): Once Guidance Folding is viewed as a standard baseline practice, the remaining methodological contribution is Reward Extrapolation (ExOPD), which simplifies to a linear extrapolation of velocities. In the ablation study (Table 5), Guidance Folding accounts for almost the entire performance jump over Standard OPD

Questions for authors
  1. Distilling a CFG-guided teacher directly into a single student conditional pass has been standard and is widely used across recent distillation framework. Can the authors clarify how GFD differs conceptually from applying standard guidance distillation to on-policy rollouts?

  2. Given that Table 5 shows GFD alone achieves most of the total gain over Standard OPD, while ExOPD adds a relatively small improvement at the cost of evaluating an additional frozen reference model at every training step, what is the wall-clock training overhead of ExOPD, and does it justify the extra compute?

  3. Did you evaluate applying ExOPD directly to Standard OPD (or same-scale OPD) to isolate whether velocity extrapolation is broadly effective on its own?

Reviewer_pWjmOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies on-policy distillation from larger diffusion teachers to smaller students. It observes that directly matching CFG-composed predictions can degrade image quality and links this behavior to branch approximation errors and weakened error cancellation. The proposed GFD trains the student’s conditional branch to match the teacher’s guided prediction, with target extrapolation providing further improvements. The paper also introduces Fixed-State KL to compare teacher–student discrepancies on shared states. Experiments on SD3.5 and FLUX.2 include component ablations and generation-quality analyses.

Critical strengths

1.Transferring capabilities from larger teachers to smaller students is valuable for deployment. The reported results show improved task performance while mitigating the image-quality degradation observed with standard cross-scale OPD. 2.Beyond benchmark scores, the paper provides branch-error decomposition,

Critical weaknesses

My main concern is the limited methodological novelty. The central operation of GFD is training a single student prediction to match the teacher’s CFG-composed output. This has a clear precedent in On Distillation of Guided Diffusion Models (Meng et al., CVPR 2023). Moreover, DanceOPD already demonstrates the absorption of CFG-defined velocity fields through on-policy distillation. Neither of these closely related works is discussed or directly compared with in the manuscript.

The large-to-small setting is practically relevant, and the reported improvements are encouraging. However, the paper does not sufficiently establish what GFD contributes beyond applying an existing guidance-distillation operation to this setting.

The cross-scale setting and error analysis may be valuable contributions, but the current presentation leaves the methodological advance over combining existing guidance distillation with OPD unclear.

The full method combines guidance folding, target extrapolation, and inference-time CFG, but the incremental ablations do not fully isolate their contributions. In particular, the main experiments still use student CFG at w=2. It therefore remains unclear how much of the improvement comes from absorbing teacher guidance into the conditional branch versus subsequent guidance adjustment.

Questions for authors

1.Does “Standard OPD” in Table 5 correspond to DiffusionOPD_{L->M} in Table 2? If so, why do the results differ? Otherwise, which training or evaluation settings differ? 2.How does GFD differ from prior guidance distillation in its objective, training-state distribution, and student parameterization? Is the main contribution methodological, an application to cross-scale OPD, or an analysis of the error mechanism? 3.The extrapolation expression in Figure 3 appears inconsistent with Eq. (8): the teacher term seems to have an additional factor of lambda. Which expression is implemented? 4.Under what conditions should the proposed CFG error mechanism and GFD extend to other generation modalities, such as video or audio? If this has not been validated, please clarify the scope of the conclusions. Experiments on additional modalities would be a future-work suggestion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.