acceptodds
Withdrawn from ICLR 2027

On-Policy Adversarial Flow Distillation for Autoregressive Video Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Autoregressive video generators are attractive for streaming, long-horizon, and interactive applications, but distilling strong black-box teachers into causal students remains difficult. The student must learn under its own rollout distribution, whereas practical teachers may expose only prompt-conditioned completed videos and may differ in architecture, capacity, temporal design, and sampling schedule. This interface makes supervised fine-tuning off-policy, score-based distillation inapplicable, and direct adversarial imitation too sparse for denoising-time credit assignment. We propose Adversarial Flow Distillation (AFD), an on-policy framework for heterogeneous black-box video distillation. AFD trains a prompt-paired Bradley–Terry discriminator on teacher videos and current student rollouts, then uses its rollout-specific advantage to weight forward-process flow-matching updates on the student's noised states. This provides dense velocity supervision without teacher scores, denoising trajectories, or step alignment. At the population optimum, the discriminator score recovers the teacher through exponential tilting, while its centered linear tilt yields strict local reverse-KL descent. Across two causal AR student families, AFD improves motion- and physics-sensitive generation while preserving general video quality. Independent physics evaluation and feedback ablations further support these gains and the importance of adaptive on-policy feedback. The method requires only clean teacher videos and student rollouts, providing a practical route for distilling proprietary or heterogeneous video generators into efficient autoregressive students.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
black-box distillation, on-policy distillation, autoregressive video generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_qn2mOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes AFD: Adversarial Flow Distillation for adapting pretrained autoregressive video generators using only completed videos from a black-box teacher. The paper trained a prompt-conditioned Bradley–Terry discriminator to distinguish teacher samples from current student rollouts. Its scores provide weights for a DiffusionNFT objective, supplemented by a reference-model regularizer. The paper also analyzes idealized density-ratio reweighting and its effect on conditional flow-matching targets. Experiments cover Self-Forcing and Causal-Forcing, with additional teacher and architecture variants, and report improvements on selected VBench dimensions and automatic physics evaluations.

Critical strengths
  1. The black-box teacher setting is practically relevant: adapting efficient students using only completed teacher videos avoids requiring access to proprietary model internals.

  2. Experiments cover multiple students and teachers, with an independent physics diagnostic and feedback ablations providing preliminary support for the effectiveness of adaptive discrimination.

  3. The population analysis provides a useful interpretation of the discriminator signal through density-ratio reweighting and explains how sample-level weights affect conditional flow-matching targets. This offers theoretical motivation for the method under idealized assumptions.

  4. The method supports distillation from teachers accessed only through a video-generation API, without requiring teacher scores or internal states. This makes it applicable to both open-source and proprietary models, including teachers with different architectures and sampling procedures.

Critical weaknesses
  1. The paper does not clearly establish its methodological distinction from APT2, One-Forcing, AAD-1, and Astrolabe. In particular, APT2 already combines autoregressive student-forcing rollouts with an online discriminator, using adversarial feedback to improve generation under student-generated histories. Astrolabe already applies NFT-based forward-process optimization to distilled autoregressive video models, including online rollouts, reward normalization, and positive/negative updates without backpropagating through complete sampling trajectories. AFD therefore appears to combine two established components: adaptive adversarial feedback, as in APT2, and NFT-based student optimization, as in Astrolabe. Relative to Astrolabe, its principal change appears to be replacing the externally specified reward with a teacher--student discriminator; relative to APT2, it changes how discriminator feedback updates the generator. These differences may be useful, but the paper does not sufficiently explain what additional methodological insight they introduce or demonstrate their value through controlled comparisons. The generic GAN baselines and limited discussion of these closely related methods leave the significance of the contribution unclear.
  2. The uniform-weight ablation is difficult to reconcile with the stated objective. Equation (4) defines the positive and negative branches using the current prediction with stop-gradient. Both branches therefore have the same forward value, while their derivatives have opposite signs. For fixed rollout samples and targets, the resulting NFT gradient is proportional to (2wi−1)(2w_i-1) and vanishes when wi=0.5w_i=0.5. However, Table 11 reports substantial improvements under this setting, with Physics VBench increasing from 68.49 to 81.74. If the student and reference models are initialized identically, the prior regularizer also initially provides zero gradient. Please clarify what produces the learning signal in this ablation, including whether the implementation uses an old/EMA predictor instead of the detached current prediction, a different initialization, or additional losses. This discrepancy limits confidence in the interpretation of the ablation.
Questions for authors
  1. What is the methodological contribution beyond combining an online teacher--student discriminator with existing NFT-based optimization? Astrolabe already uses online rollouts for NFT-based AR video optimization, while APT2 already performs student-forcing adversarial training. Please clarify what is novel about the claimed on-policy formulation and what controlled evidence distinguishes AFD from these approaches.

  2. The introduction motivates AFD through the difficulty of assigning video-level feedback to individual frames or denoising states. However, the NFT update appears to reuse a single rollout-level weight across its forward-noised training states. What credit-assignment mechanism does AFD introduce beyond standard NFT? Equation (12) explains how reweighting changes conditional flow targets, but does not identify which frames or steps caused the reward. Please clarify whether the claim concerns fine-grained credit assignment or only the propagation of sample-level feedback into dense training gradients.

  3. Few-step generators and adversarial training can both suffer from mode collapse. How does AFD behave when a few-step student produces low-diversity rollouts with similar discriminator scores? In this case, centered NFT advantages may provide limited learning signal. Could the authors clarify whether the NFT update and reference-model prior can help restore diversity, and whether this behavior has been evaluated?

Reviewer_HSaJOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Adversarial Flow Distillation (AFD), a method for finetuning autoregressive video generators by distilling black-box teachers. It trains an online discriminator to distinguish teacher-generated videos from current student rollouts, then uses the discriminator scores as rewards for DiffusionNFT-based flow-matching updates. The method is motivated as an on-policy method, as the authors claim that off-policy SFT is not effective. Experiments on Self-Forcing and Causal-Forcing students with Seedance 2.0 teacher demonstrate improvements in motion quality and physical consistency, beating SFT and GAN baselines.

Critical strengths
  • This paper effectively trains an online reward model and use DiffusionNFT to improve the student. This method appears technically sound, and also very practical and straightforward.
  • The experiments show consistent improvements over GAN baselines across multiple students. The online/frozen discriminator ablation further supports the benefit of adaptive teacher--student feedback.
  • Using black box video generators to improve AR students is an interesting and practical problem.
Critical weaknesses
  • My major concern is that the experiments are insufficient to support the central claim of the paper about the importance of on-policy distillation. This is why I lean towards a weak reject despite the method looking interesting.
    • The paper uses SFT as its primary off-policy baseline. However, the student models are already distilled few-step models. DiffusionNFT works on few-step models, but SFT with plain flow matching loss does not work for few step generation. Therefore, the poor SFT performance does not necessarily demonstrate the disadvantage of off-policy training.
    • A more meaningful ablation should use a more competitive off-policy baseline, such as further distilling the SFT model for few-step generation (e.g., using score-based DMD). Alternatively, consider using a multi-step student as the baseline for both SFT and DiffusionNFT.
  • It appears that training and evaluation prompts overlap. The authors report using 200 prompts from VideoPhy-2 for adaptation and subsequently evaluating using VideoPhy-2. This overlap could substantially weaken the evidence for generalization.
  • The novelty of the method itself is somewhat limited. The method primarily combines established DiffusionNFT with an online teacher--student discriminator. It would be better to discuss closely related methods such as Astrolabe and AdvDMD.
Questions for authors

Can the authors clarify the training and evaluation prompts?

Reviewer_vvWQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AFD, an on-policy autoregressive video distillation method for black-box video teacher models. The method first uses a Bradley–Terry discriminator to estimate the distribution discrepancy between teacher videos and student-generated videos to form dynamic rewards. Then based on DiffusionNFT, video-level rewards are converted into flow matching updates for the forward process of the student model, without requiring the score or denoising trajectory from the teacher model. Experiments are conducted on models such as Self-Forcing and Causal-Forcing, and some experiments are conducted on motion quality and physical consistency.

Critical strengths
  1. The research problem has practical value. This paper focuses on black-box teacher models, overcoming the limitations of traditional distillation methods that rely on teacher scores, latents, and denoising trajectories.

  2. The method design is reasonable. The authors combine the estimation of teacher-student distribution discrepancy with the student's own forward-process flow matching, and this design can train the student without the intermediate states of the teacher.

  3. The experiments cover many models, including self-forcing, causal forcing, Wan2.1, Seednace1.5, and Kling.

Critical weaknesses
  1. The forward-process is proposed in the DiffusionNFT, and using the forward-process RL in the autoregressive video model is also proposed in the Astrolabe, I think the authors should compare the performance with these existing methods.

  2. The authors use 200 prompts from VideoPhy-2 to obtain teacher videos, and adopted VideoPhy-2-PC to evaluate physical consistency. However, they do not state whether the adaptation prompts and the final evaluation prompts are strictly separated.

  3. In this paper, the Bardley-Terry discriminator is used to distinguish the teacher and the student videos. The authors also state that AFD improves motion- and physics-sensitive generation. I think since the two types of videos may differ in visual style, texture, generation quality, and motion performance, the experiments cannot determine whether the improvement in physics-related metrics stems from the effective transfer of motion and physical features or from teacher-student visual distribution alignment.

  4. Generalization in long-video generation. The paper states the potential of autoregressive video generation for long video generation tasks. But the experiments have not validated the performance of AFD on longer videos. I think the current experiments are insufficient to judge the stability of this method in long-video generation under error accumulation.

  5. Lack of user studies.

Questions for authors
  1. Can AFD maintain the same performance gains over longer autoregressive rollouts? I would like to see some results in long-video generation.

  2. Do different experiments use online teacher API calls or pre-cached teacher videos? What is the number of teacher samples required per training run and the API call volume?

  3. The authors should conduct a user study to show more results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.