acceptodds
Under review as a conference paper at ICLR 2027

Rethinking Classifier-Free Guidance in On-Policy Diffusion Distillation

Abstract

On-policy distillation (OPD) adapts diffusion models by querying a teacher along trajectories generated by the current student, but how it should behave under classifier-free guidance (CFG), a default component of modern diffusion systems, remains poorly understood. Existing OPD methods naturally extend velocity matching to the CFG-composed prediction, directly matching teacher and student guided velocities. We show that this objective is under-identified at the branch level: positive- and negative-branch errors can compensate in the guided prediction. Through two contrasting cases, we find that naive matching remains effective under shared negative conditioning, where both branch errors decrease jointly. When the model's native CFG schema retains privileged information in the teacher's negative branch that is unavailable to the student, however, this joint reduction breaks down and the composed objective induces antagonistic branch-error dynamics, reducing the positive-branch error while increasing the negative-branch error. We term this failure mode Negative Branch Asymmetry (NBA). To address NBA, we introduce Positive–Direction Matching (PDM), a branch-aware OPD objective that separately constrains the positive prediction and the CFG conditional direction. We apply PDM to dense-to-sparse video control, where naive guided matching is highly sensitive to inference guidance scales, while branch-aware supervision enables more robust and effective knowledge transfer.

Primary area:
generative models
Keywords:
On-Policy Distillation, Classifier-Free Guidance, Diffusion

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_czaJOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper characterizes the branch-error dynamics, negative branch asymmetry (NBA), under teacher’s privileged negative conditioning during the on-policy distillation of diffusion models with classifier-free guidance. To mitigate the issue, the paper proposes positive direction matching (PDM) that separately match the positive branches between teacher and student, and the CFG conditional direction.

Critical strengths
  • The paper is well-written and easy to follow.

  • The analysis of branch error in Fig 2 validates the problem.

  • The paper demonstrates the proposed method with diverse experiments.

Critical weaknesses
  • The theoretical justification of PDM is limited to zero-loss case. The paper argues that zero PDM loss forces e+=e−=0e_+=e_-=0. However, in the “privileged” setting, zero loss may be unattainable since the student's conditioning does not fully determine the teacher's predictions. Since the student’s negative branch is conditioned only on the null prompt, whenever E[vT−∣x,y]≠E[vT−∣x]E[v^-_T|x,y]\neq E[v^-_T|x] the student can match the teacher's expected guided output at most at a single γ\gamma, regardless of the objective. Therefore, zero-loss identifiability alone does not explain PDM's robustness across γ\gamma scales.
  • The proposed problem is valid (Fig 2). But the problem depends on the CFG configuration, where the teacher’s negative branch is privileged. If the teacher’s negative branch is not privileged, both negative branches share the same input, which the paper’s own analysis suggests may fall into the benign regime. In that case, the problem could be avoided without the proposed loss.
Questions for authors
  • Could the authors characterize what the minimizer of PDM is and why it is a reasonable approach? Otherwise, the results remain empirical.

  • Could the authors evaluate, for example, the teacher’s negative branch without reference? If it performs worse, this could strengthen the motivation.

3 comments
AuthorsOpenReview ↗

Weakness 1 & Question 1 — Branch-Level Optima Beyond Zero Loss

Thank you for drawing attention to the zero-loss assumption. We agree that exact teacher matching may be impossible when the teacher has privileged information. Our central claim is that PDM can still select a unique pair of student branch predictions when zero loss is unattainable. We have clarified this distinction in Sec. 3.2 and added a population-loss analysis in Appendix 7.1.

Setup. We fix the distribution of supervised states, assume finite second moments, and allow each student branch to be any function of its inputs. Let xx contain the inputs shared by the student branches and yy the additional positive-branch input. Write f(x,y)=vS+f(x,y)=\mathbf v_S^+ and h(x)=vS−h(x)=\mathbf v_S^-. For the teacher, define

b=\mathbb E[\mathbf v_T^-\mid x,y],\qquad \bar b=\mathbb E[\mathbf v_T^-\mid x].$$ If $b\neq\bar b$ with nonzero probability, $h(x)$ cannot reproduce $b(x,y)$ from its inputs alone, and exact matching of the expected guided output at two distinct guidance scales is impossible. **This does not settle the question above: naive matching and PDM can still differ in whether their population optima determine the student branches.** **Naive matching.** At a fixed $\gamma>0$, its population loss is minimized when $$\gamma f+(1-\gamma)h=\gamma a+(1-\gamma)b.$$ For any choice of $h(x)$, one can choose $f(x,y)=[\gamma a+(1-\gamma)b-(1-\gamma)h]/\gamma$ to satisfy this equation. Thus matching one guided prediction does not determine the student's negative branch. **Positive–Direction Matching.** The expected PDM loss can be written as $$J_{\mathrm{PDM}}=C+\mathbb E \left[\|[\|f-a[\|\|_2^2+\lambda[\|\|f-h-(a-b)[\|\|_2^2\right],$$ where $C$ is independent of $f$ and $h$. For a fixed $h$, minimizing over $f$ gives $$\begin{aligned} f_h^\star &= a-\frac{\lambda}{1+\lambda}(b-h),\\\\ J_{\mathrm{PDM}}(f_h^\star,h)&=C+\frac{\lambda}{1+\lambda}\mathbb E\|b-h\|_2^2. \end{aligned}$$ Since $h$ can depend only on $x$, minimizing the remaining term over $h(x)$ gives $$h^\star=\bar b,\qquad f^\star=a-\frac{\lambda}{1+\lambda}(b-\bar b).$$ For $\lambda>0$, this population minimizer is unique up to sets of probability zero. In particular, $h^\star=\bar b$ is the best negative-branch prediction available from $x$, while the positive branch balances matching $a$ against matching the conditional direction. The minimum loss remains positive when $b\neq\bar b$ with nonzero probability. The key distinction is that guided-only matching leaves $h$ free, whereas PDM identifies both branches at the population optimum even when exact teacher matching is impossible. This result does not guarantee that a parameterized student reaches the optimum or matches the teacher at every guidance scale; the guidance-sweep results in revised Table 3 provide the empirical evidence for robustness.
AuthorsOpenReview ↗

Weakness 2 & Question 2 - CFG Configuration

Thank you for raising these related points. We agree that NBA depends on the teacher's CFG configuration, and the revised paper now states this scope explicitly in Sec. 6. We address both the suggested negative-branch reconfiguration and its effect on the teacher in our response to Reviewer TSgv, “Weakness 1 & Question 1 - Isolate Negative-Branch Conditioning” (revised Fig. 2, Table 3, and Fig. 10).

Dear reviewer czaJ:

As the discussion period closes in a few hours, we wanted to thank you once again for your insightful review and constructive questions on the zero-loss argument and the teacher’s CFG configuration.

In case it is helpful for a quick final check, here is a brief pointer to our updates in the responses and revised paper:

  • Beyond zero loss: Sec. 3.2 and Appendix 7.1 now characterize PDM’s population minimizer under the stated assumptions.
  • CFG configuration: We added within-setting negative-branch comparisons and a teacher-output example (revised Fig. 2, Table 3, and Fig. 10).

We appreciate your time and would welcome any further questions.

Reviewer_TSgvOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper explains the Negative Branch Asymmetry (NBA) failure mode of on-policy diffusion distillation (OPD). It happens when the teacher receives privileged negative conditioning compared to the student under classifier-free guidance (CFG). The failure mode causes instable generation and poor quality. The paper proposed Positive–Direction Matching (PDM) that separately supervises the positive prediction and the CFG conditional direction to solve the issue. Experiments show improved generation quality and reduced sensitivity to changes in inference-time guidance scales.

Critical strengths
  • Clear and informative analysis. The paper is well written and explains the problem clearly. It identifies a key mismatch and provides a reasonable explanation and an intuitive yet effective solution. The paper is informative and provide useful guidance for designing branch-retaining distillation methods.
  • Extensive experiment support. The evaluation combines image-domain and video-control. Comparisons with different methods including Independent Branch Matching (IBM) provide meaningful proof for assessing the benefits of on-policy learning and branch-aware supervision. The results show improved control fidelity at the training guidance scale as well as better robustness during inference. The experiments provide convincing evidence of the approach’s usefulness.
Critical weaknesses
  • The comparison used to explain when NBA emerges does not isolate negative-branch conditioning. My main concern is the explanation of the emergence of NBA in Section 4.1. The text-rendering and reference-conditioned settings differ in both cT+c_T^+ and cT−c_T^-. How should I comprehend the two settings “differ only in negative conditioning”? I believe a comparison keeping cT+c_T^+, cS+c_S^+, cS−c_S^-, and the remaining training setup fixed, while varying only cT−c_T^- would provide more direct evidence.
Questions for authors
  1. Isolating the role of negative-branch conditioning in NBA. As mentioned in weakness, the two settings in Section 4.1 differ in positive and negative branch, could you provide a controlled comparison that keeps cT+c_T^+, cS+c_S^+, cS−c_S^-, and the remaining training setup fixed, while varying only cT−c_T^-? Specifically, does NBA still emerge when the teacher retains privileged information in its positive branch but shares the student’s negative-branch conditioning?

  2. Comparison with matching at multiple guidance scales. Could you compare PDM against an objective that matches teacher and student CFG-composed predictions at two distinct guidance scales on the same supervised state, while keeping the rollout guidance fixed? For example, during training, the loss could average guided-matching losses at γ=3\gamma=3 and γ=5\gamma=5 to avoid compensation. Both losses reuse the existing branch predictions, requiring no additional teacher forward passes, and jointly achieving zero loss forces both branch errors to zero. Does PDM retain an advantage over this alternative in control fidelity and guidance-scale robustness?

3 comments
AuthorsOpenReview ↗

Weakness 1 & Question 1 - Isolate Negative-Branch Conditioning

Thank you for pointing this out. We agree that the two original settings differ in more than the teacher's negative-branch conditioning, so comparing them alone does not isolate its effect. We have corrected that wording in Sec. 4.1 and added a comparison within each setting. In these comparisons, the teacher's positive condition and both student conditions remain unchanged while we change the teacher's negative condition.

Reference-conditioned setting. We keep the reference image in the teacher's positive branch, remove it from the negative branch, and train with naive matching at γ = 2. Both branch errors now decrease. In the original configuration, the positive error decreases while the negative error increases (revised Fig. 2, bottom row; Sec. 4.1.2).

Video (pose) setting. We make the analogous change by replacing dense pose with the student's sparse pose in the teacher's negative branch, then train with naive matching. Table R1 compares this run with the original naive run and PDM across inference guidance scales; the new rows also appear in revised Table 3 ("Naive, sparse c_T^-").

γMethodMPJPE All ↓MPJPE Key ↓[email protected] All ↑[email protected] Key ↑[email protected] All ↑[email protected] Key ↑FID ↓FVD ↓
5Naive (privileged c_T^-)4.623.2291.3595.0179.3989.5414.0567.44
3Naive (privileged c_T^-)4.943.3190.2394.5377.9288.2222.66134.09
1Naive (privileged c_T^-)8.986.2380.0887.4269.1481.5778.20507.70
5Naive, c_T^- = p_sparse4.182.7793.1796.5182.5191.8015.0664.61
3Naive, c_T^- = p_sparse4.202.8293.0796.3782.2691.4615.7778.54
1Naive, c_T^- = p_sparse5.543.8989.3593.3777.6087.6024.89119.89
5PDM4.272.7592.7196.3882.1291.6613.8358.65
3PDM4.172.7392.8196.6182.2791.8113.4655.85
1PDM4.482.9092.1495.6481.7291.1515.2560.97

At γ = 1, this change improves naive matching from 507.70 to 119.89 FVD and from 8.98 to 5.54 MPJPE-All; at γ = 3 and 5, its pose accuracy is comparable to PDM. Removing privileged information from the teacher’s negative branch substantially improves naive matching, but does not close the gap to PDM (revised Table 3). Figure 10 illustrates a likely reason: with the positive branch and sampling seed held fixed, replacing dense pose in the teacher’s negative branch with sparse or no pose produces visible quality degradation in the teacher’s own output. PDM instead uses the teacher’s original CFG configuration while addressing branch-error compensation in the student objective.

Together, these within-setting comparisons provide more direct evidence for the role of privileged negative conditioning in the observed failure. We have included them in Fig. 2 and Sec. 4.1.2 (reference-conditioned setting), and Table 3, Fig. 10, and Sec. 4.2.3 (video control) of the revised paper.

AuthorsOpenReview ↗

Question 2 - Multi Guidance Matching

Thank you for the suggestion. We implemented this objective in the video (pose) setting: the loss averages the guided-matching losses at γ = 3 and γ = 5 on the same supervised state, reusing the existing branch predictions, with the rollout guidance fixed at γ = 5. Table R2 compares it with naive matching and PDM across inference guidance scales; these rows are now in Table 3 of the revised paper and are discussed in Sec. 4.2.3.

γMethodMPJPE All ↓MPJPE Key ↓[email protected] All ↑[email protected] Key ↑[email protected] All ↑[email protected] Key ↑FID ↓FVD ↓
5Naive4.623.2291.3595.0179.3989.5414.0567.44
3Naive4.943.3190.2394.5377.9288.2222.66134.09
1Naive8.986.2380.0887.4269.1481.5778.20507.70
5Two-scale matching (γ ∈ {3, 5})4.332.9592.4895.8980.9890.3614.1567.27
3Two-scale matching (γ ∈ {3, 5})4.563.1791.6995.0580.0489.4217.9291.17
1Two-scale matching (γ ∈ {3, 5})5.804.1888.2292.5376.7886.9135.52189.21
5PDM4.272.7592.7196.3882.1291.6613.8358.65
3PDM4.172.7392.8196.6182.2791.8113.4655.85
1PDM4.482.9092.1495.6481.7291.1515.2560.97

PDM retains its advantage. It is better than two-scale matching on every metric at every γ, including γ = 3, one of the two matched scales (FVD 55.85 vs. 91.17). Away from the matched scales the gap widens: at γ = 1, FVD is 60.97 for PDM vs. 189.21, and MPJPE-All 4.48 vs. 5.80.

Dear reviewer TSgv:

As the discussion period closes in a few hours, we wanted to thank you once again for your insightful review and constructive questions on isolating negative-branch conditioning.

In case it is helpful for a quick final check, here is a brief pointer to our updates in the responses and revised paper:

  • Conditioning control: We added within-setting comparisons in the reference-conditioned and video experiments (revised Fig. 2 and Table 3).
  • Two-scale matching: We evaluated the baseline you suggested on the same student states and report it alongside PDM in revised Table 3.

We appreciate your time and would be glad to clarify anything that remains unclear.

Reviewer_T8miOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies a problem with CFG-based on-policy diffusion distillation: matching only the teacher’s and student’s combined predictions can let errors in the two branches cancel out. When the teacher has extra conditioning information, the positive branch may improve while the negative branch gets worse. The authors call this Negative Branch Asymmetry (NBA), which can cause generation to break down when the guidance scale changes. They propose Positive–Direction Matching, which separately matches the positive prediction and the difference between the two branches. Image experiments illustrate the failure, while video experiments show better control and more stable generation across guidance scales. Directly matching both branches (IBM) also works well. For efficiency, video training supervises only the first 8 of 50 denoising steps.

Critical strengths
  1. The guidance-scale results are striking and practically relevant. When γ changes from 5 to 1, naive OPD’s FVD jumps from 67.44 to 507.70, while PDM and IBM remain stable. Since users often adjust guidance at inference, this exposes a real weakness in naive distillation and is the paper’s strongest result.

  2. The fix is simple and requires no extra model evaluations, since the branch predictions are already available. Table 2 also shows better control accuracy at the training guidance scale across pose, depth and scribble, giving the method practical value beyond guidance-scale robustness.

Critical weaknesses

W1. PDM’s advantage over simpler alternatives is unclear. IBM already removes the branch ambiguity and performs similarly, sometimes with better generation quality. The small differences are hard to judge without repeated runs or confidence intervals. Positive-only matching (λ = 0) is also competitive at the training scale, but is missing from the guidance sweep—the key comparison needed to show whether PDM’s direction term actually helps.

W2. The cause of NBA is not cleanly isolated. The two image experiments differ in backbone, task, data size, training setup and teacher construction, so the different error curves cannot be attributed to negative conditioning alone. A cleaner test would change only the teacher’s negative-branch input within the same setting. Table 3 also lacks teacher results across guidance scales, making it harder to separate distillation-induced degradation from the teacher’s own CFG sensitivity.

W3. The compute comparison with SFT and off-policy baselines is unclear. The paper claims equal training budgets, but equal optimizer steps do not establish equal compute, especially given OPD’s rollout costs. Per-method GPU time, teacher-query counts and numbers of supervised transitions are not reported, so the gains over these baselines are difficult to assess at comparable cost.

Questions for authors

Q1. Please run the within-setting toggle described in Weakness 2, with everything else fixed: reference removed from the teacher's negative branch only in the style setting, and p_sparse in the teacher's negative branch in the video setting. Does the antagonistic branch-error pattern disappear? This is the experiment that would actually support the causal claim in the title of Sec. 4.1.

Q2. Please add to Table 3: Teacher and unadapted Student at γ ∈ {1, 3, 5}, and PDM with λ = 0. Does the direction term matter anywhere off-scale? If λ = 0 is also stable, the contribution should be restated as "anchor the positive branch" rather than PDM.

Q3. Please report at least three seeds with standard deviations for Naive, IBM and PDM in Tables 2 and 3. If IBM and PDM are statistically indistinguishable, please reframe the contribution accordingly.

Q4. Have you compared PDM with matching CFG-composed predictions at two fixed guidance scales on the same student states? This also removes branch ambiguity without extra model evaluations and would be a useful baseline.

Q5. How are training budgets matched across SFT, off-policy distillation and OPD? Please report GPU time, teacher-query counts and supervised transitions for each method, and clarify whether “equal budget” means equal training steps or comparable compute.

7 comments
AuthorsOpenReview ↗

Weakness 1 & Question 2 - PDM vs. IBM and Positive-Only Matching

Thank you for highlighting these comparisons. IBM and positive-only matching are strong baselines, and we agree that evaluating λ=0\lambda=0 only at the training scale left an important question open. We have added its guidance sweep, along with the Teacher and unadapted Student rows requested in W2/Q2. Table R3 below shows the results; the new rows also appear in revised Table 3 and are discussed in Sec. 4.2.3.

γMethodMPJPE All ↓MPJPE Key ↓[email protected] All ↑[email protected] Key ↑[email protected] All ↑[email protected] Key ↑FID ↓FVD ↓
5Teacher3.032.8296.0296.4691.1592.2314.1356.01
3Teacher2.912.7296.4196.9191.6692.5312.7742.23
1Teacher2.922.6596.1697.0691.4392.8212.0436.58
5Student (unadapted)5.924.1086.7891.8969.9782.5315.8181.41
3Student (unadapted)5.904.2286.9791.7370.2182.3214.2464.45
1Student (unadapted)6.464.6584.9290.2268.0879.8213.9359.81
5Naive4.623.2291.3595.0179.3989.5414.0567.44
3Naive4.943.3190.2394.5377.9288.2222.66134.09
1Naive8.986.2380.0887.4269.1481.5778.20507.70
5IBM4.282.9192.6795.9581.6991.0312.9953.83
3IBM4.423.0492.0995.5381.4190.8113.6956.85
1IBM4.913.4390.9294.6980.6689.9818.0071.17
5PDM, λ = 0 (positive-only)4.242.8192.7196.2481.8091.0513.9759.72
3PDM, λ = 0 (positive-only)4.242.8892.8896.2582.4491.5513.1555.89
1PDM, λ = 0 (positive-only)4.553.0291.9695.6681.8191.1514.7466.65
5PDM, λ = 14.272.7592.7196.3882.1291.6613.8358.65
3PDM, λ = 14.172.7392.8196.6182.2791.8113.4655.85
1PDM, λ = 14.482.9092.1495.6481.7291.1515.2560.97

The new sweep confirms that positive-only matching (λ=0\lambda=0) is itself stable across guidance scales. Full PDM (λ=1\lambda=1) gives lower keyframe MPJPE and FVD at each tested scale, with the clearest FVD gain at γ=1\gamma=1 (60.97 vs. 66.65). This suggests that positive-branch matching accounts for much of the stability, while the direction term provides an additional gain in these metrics. IBM is also competitive: it has lower FVD at the training scale, while PDM has stronger pose control and lower FVD at γ=3\gamma=3 and 11. We report three-seed results for Naive, IBM, and PDM in our response to Q3 so that the smaller differences can be assessed more carefully.

AuthorsOpenReview ↗

Weakness 2 & Question 1 - Isolation of the Cause of NBA and Missing Teacher in Table 3

Thank you for raising both concerns. For the within-setting tests of the teacher's negative-branch input, please see our response to Reviewer TSgv, “Weakness 1 & Question 1 - Isolate Negative-Branch Conditioning” (revised Fig. 2 and Table 3). For the teacher's behavior across guidance scales, please see our response to your W1/Q2, which reports the Teacher and unadapted Student at γ∈{1,3,5}\gamma\in\{1,3,5\} (Table R3; revised Table 3). These results help separate the teacher's own CFG response from the degradation introduced by naive distillation.

AuthorsOpenReview ↗

Weakness 3 & Question 5 - Computation Clarification

We appreciate the reviewer’s point that equal optimizer steps do not give SFT, off-policy distillation, and OPD equal compute budgets. The submission used step-matched baselines. Under our training setup, a step takes about 20 s for SFT and off-policy distillation, versus about 130 s for OPD because of its on-policy rollouts and teacher evaluations. We therefore trained the two cheaper baselines for about 6×6\times more steps to match wall-clock cost. Table R4 compares these checkpoints with the submitted ones and with PDM, using the same γ=5\gamma=5 evaluation as Table 2.

Pose

MethodMPJPE All ↓MPJPE Key ↓[email protected] All ↑[email protected] Key ↑[email protected] All ↑[email protected] Key ↑FID ↓FVD ↓
SFT (submission)5.023.3190.3694.7776.7288.5514.1163.90
SFT (matched compute)5.013.1590.3395.0876.6589.0315.8570.90
Off-policy (submission)5.143.4489.7894.0975.4687.4814.4968.50
Off-policy (matched compute)4.863.0991.0095.5577.3289.6213.9164.87
OPD with PDM (submission)4.132.8392.9896.1282.4891.0313.5262.77

Depth

MethodCorr All ↑Corr Key ↑RMSE All ↓RMSE Key ↓SI-RMSE All ↓SI-RMSE Key ↓FID ↓FVD ↓
SFT (submission)88.3996.1713.837.8011.966.5212.8758.13
SFT (matched compute)88.6296.4813.747.6011.806.2413.4360.02
Off-policy (submission)88.2596.0113.988.0012.106.6412.6659.35
Off-policy (matched compute)88.5796.2913.777.7111.886.3512.4354.44
OPD with PDM (submission)91.2497.1311.956.9110.315.6811.6460.60

Scribble

MethodRecall All ↑Recall Key ↑Precision All ↑Precision Key ↑F1 All ↑F1 Key ↑FID ↓FVD ↓
SFT (submission)67.2389.0369.5290.6668.2789.7910.6950.56
SFT (matched compute)69.9290.8971.8692.1470.7991.4611.2545.74
Off-policy (submission)70.9891.5572.1591.7371.4991.6011.6052.90
Off-policy (matched compute)70.2591.3172.5492.6571.2991.9311.4444.46
OPD with PDM (submission)76.4293.2576.1392.8676.2293.0310.3450.53

With this comparison, PDM retains the best primary control metric in each modality: pose MPJPE-All, depth SI-RMSE-All, and scribble F1-All. It also has lower FID than both matched-compute baselines in all three modalities. The baselines achieve lower FVD on depth and scribble, which the revised paper reports as well. We have replaced the step-matched SFT and off-policy rows in Table 2 with these matched-compute checkpoints and clarified the budget in Sec. 4.2.1.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We appreciate the reviewers' careful feedback. We have uploaded a revised PDF, with the changes highlighted in blue. The main revisions are:

  • Theory (Sec. 3.2 and Appendix 7.1): We distinguish zero-loss attainability from uniqueness of the branch-level population optimum and characterize PDM's minimizer under a fixed distribution of supervised states.
  • NBA diagnosis (Sec. 4.1, Fig. 2, and Sec. 6): We correct the earlier cross-setting comparison, add a within-setting negative-branch test in the reference-conditioned experiment, and clarify the role of the teacher's CFG configuration.
  • Video results (Sec. 4.2, Table 3, and Appendix Fig. 10): We add Teacher, unadapted Student, positive-only matching (λ=0\lambda=0), two-scale guided matching, and a run with sparse control in the teacher's negative branch. Fig. 10 shows teacher outputs under different negative-branch inputs.
  • Compute comparison (Table 2 and Sec. 4.2): We replace the SFT and off-policy rows with matched-compute checkpoints, trained for about 6×6\times as many optimizer steps as OPD.

A shared concern was whether privileged negative-branch conditioning, rather than other differences between the original settings, drives the observed NBA. We have corrected the cross-setting claim and added within-setting comparisons in the reference-conditioned and video experiments (revised Fig. 2 and Table 3). These results support the role of the teacher’s negative-branch input; Fig. 10 also illustrates the quality cost of reconfiguring the teacher. We discuss the evidence and its scope in our response to Reviewer TSgv, “Weakness 1 & Question 1 - Isolate Negative-Branch Conditioning.”

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.