acceptodds
Under review as a conference paper at ICLR 2027

Parallel Decoding Distillation for Fast Image and Video Generation

92 views

Abstract

Generation in video diffusion or flow models is computationally expensive due to the slow and iterative sampling process. Current state-of-the-art (SOTA) acceleration methods heavily rely on variational score distillation (VSD) and adversarial losses to distill diffusion models into few-step generators. Albeit achieving high-quality video generation, these training losses are notoriously hard to optimize and suffer from mode collapse, leading to loss of video diversity and lack of motion. In this paper, we introduce Parallel Decoding Distillation (PDD), a simplified and scalable trajectory-based distillation method for fast inference of diffusion and flow matching models. Our architecture and training procedure are compatible with any pre-trained model and support sampling with a varying number of function evaluations (NFE). PDD accelerates generation by predicting multiple denoising steps per network evaluation. Conceptually, it learns a representation of the mean velocity without regressing its derivative using JVPs or finite-difference approximations. Our method achieves SOTA performance with 4-8 NFE on LTX-2.3 Text-to-Video/Audio, Wan 14B Text-to-Video, and Qwen-Image Text-to-Image. Moreover, PDD presents a significant improvement in generated video diversity.

Primary area:
generative models
Keywords:
distillation, fast inference, flow maps, meanflow, diffusion models, flow matching.

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 4.00 over 3 reviews, above 99% of the others.
Reviewer_51mwOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Parallel Decoding Distillation(PDD), a trajectory-based distillation framework for accelerating image and video generation with diffusion and flow matching models. PDD introduces a parallel decoder that predicts the mean velocities of multiple consecutive denoising intervals within a single network evaluation. The student is trained through on-policy trajectory regression, using teacher velocities approximated by Euler or Midpoint solvers as supervision. During inference, multiple prediction heads are fused into a single linear layer, enabling efficient few-step sampling with flexible NFE budgets. The method avoids computationally expensive JVPs, finite differences, adversarial objectives, and multi-stage distillation. Experiments on ImageNet-256, Qwen-Image, Wan2.1, and LTX-2.3 demonstrate competitive or state-of-the-art generation quality at 4–8 NFEs, while preserving greater sample diversity than distribution-based distillation baselines.

Critical strengths
  1. The proposed method is theoretically well-grounded and technically elegant, featuring a simple, efficient on-policy regression objective and a parallel decoding architecture that supports flexible inference budgets.
  2. The experiments are comprehensive and convincing, covering multiple large-scale image and video generation models, with competitive generation quality and strong preservation of sample diversity.
Critical weaknesses

The methodological novelty relative to Pi-Flow remains insufficiently justified. The paper lacks a clear explanation of why discrete mean-velocity prediction and its regression objective are methodologically necessary or fundamentally advantageous compared to Pi-Flow's continuous policy parameterization.

Questions for authors

Thank you for the solid work. The paper is theoretically well-grounded and supported by comprehensive experiments. My main concern is the methodological novelty compared to Pi-Flow. Could the authors clarify why discrete mean-velocity prediction and its regression objective are necessary compared to continuous policy parameterization? Do these designs offer advantages beyond architectural simplicity and computational efficiency that continuous policy methods cannot easily achieve? If the authors can clearly justify these differences through theoretical analysis or targeted experiments, I would like to raise my score to 4.

2 comments
Reviewer_51mwOpenReview ↗

Concern Addressed

Thank you for the detailed rebuttal. The clarification of the unified time discretization, mean-velocity supervision, and state-independent policy with layer fusion has largely addressed my concerns about the methodological novelty relative to Pi-Flow. I particularly appreciate the explanation of how these design choices enable flexible NFE budgets and efficient inference. I encourage the authors to clarify the scope of the any-NFE claim and further discuss the role of policy parameterization in the final version. Overall, I am satisfied with the response and will raise my score to 4.

AuthorsOpenReview ↗

Authors' response

We are glad that the provided discussion addressed the reviewer’s concerns. In the final version of the paper, we will further clarify and stress these points. We are grateful to the reviewer for dedicating their time to carefully review our paper, for providing constructive comments that helped us better emphasize the paper’s contributions, and for raising their score.

Reviewer_NUkqOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces a novel step-distillation method which predicts multiple consecutive denoising steps in a single model evaluation called Parallel Decoding Distillation (PDD). PDD uses a simple regression loss to match the teacher’s mean velocities along the student's trajectory, avoiding adversarial and distribution matching losses, and derivative estimation. The method supports different inference budgets with one model and is evaluated on large-scale image, video, and audio-video generation models, where it achieves strong quality and diversity at 4-8 NFEs.

Critical strengths
  1. PDD provides a simple and practical alternative to existing step distillation methods. It uses a single regression objective and avoids adversarial training, VSD, JVPs, and other complications. This makes the method easier to train and potentially more scalable to large image, video, and audio models.

  2. The empirical results are strong and broad. PDD is evaluated on several large-scale image, video, and audio-video models, where it performs competitively at 4-8 NFEs while preserving sample diversity. The ability to support different inference budgets with a single distilled model is also practically useful.

Critical weaknesses
  1. The perceptual evaluation is limited. The paper includes no human evaluation or user study, even for video and audio-video generation, and instead relies on automated metrics and an AI-based judge. It is therefore unclear whether the reported improvements are consistently perceptible to human users.

  2. Evaluation in the extreme low-step regime is incomplete. The main large-scale video and audio-video results focus on 4-8 NFEs, with no 1-NFE results for the large text-conditioned models and no 1-2 NFE results for LTX-2.3. This leaves the method’s competitiveness at the lowest inference budgets unclear.

Questions for authors
  1. How does PDD perform at 1-2 NFEs on large text-to-image and video models, especially compared with adversarial/distribution-matching based methods?

  2. Could the authors provide a human evaluation of the video and audio-video results and comparisons to previous work?

  3. How important is training along the student trajectory? How does it compare with matching velocities only along the teacher trajectory?

  4. Since CFG is distilled at a fixed scale, can the guidance strength be adjusted at inference, or does this require training a separate model? Can CFG be distilled in such a way that its strength is still adjustable during inference (if it is not currently)?

  5. How sensitive is PDD to the time discretization and batch size? The paper identifies these as the most important hyperparameters, but it is unclear how much performance degrades when using smaller batches or different time schedules.

Reviewer_vvwYOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a new parameterization for a trajectory-based distillation method. It builds on Pi-Flow to modify the parameterization of the student network and can distill the model via regression. The method removes the need to compute JVP or an additional network. The authors let the student predict the mean velocity across different intervals during sampling, then integrate the prediction to match the teacher’s mean predictions, and can achieve arbitrary NFE during sampling. The paper shows improvement over multiple baselines on t2i and t2v generation.

Critical strengths
  1. The author proposed a new method for trajectory-based distillation. It is based on Pi-Flow but brings in the idea of flow matching and can fine-tune the model with regression loss. The method overall is simple and avoids components such as JVP and additional score predictors.
  2. The block designs and student predictor head support variable NFE selections during sampling.
  3. The experiments are strong. It shows improvement over multiple benchmarks and tasks.
Critical weaknesses
  1. Lacking some CM, flow-map-based results. However, this paper is from a different class. It will be helpful to compare with other distillation model results.

  2. I noticed the method can also support data-free distillation. Are there any results for the data-free setting compared with DMD?

  3. Can you provide some results with NFE = 1, 2?

  4. What are the training and inference efficiencies, especially for inference when predicting multiple heads and integrating? What is the speed compared with other methods?

  5. I noticed that the PDD-Midpoint results are even better than the teacher. Can you explain where you think the gain comes from?

Questions for authors

See the Weakness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.