- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes PACT, an offline flow-map distillation method for accelerating pre-trained image and video diffusion backbones. PACT retains and reuses teacher rollout states to replace differential flow-map supervision with explicit transition regression.The student is initialized from the pre-trained backbone frozen with LoRA adapters and destination-time and guidance conditioning trained with the regression loss. This removes teacher evaluations and derivative-based supervision from student training.
Experiments cover four-step text-to-image generation with SD3.5 Medium and eight-step text-to-video and image-to-video generation with LTX-2.3. Compared with DP-DMD and official LTX distilled checkpoints, PACT generally retains more visual diversity and improves matched-seed teacher correspondence in image and text-to-video generation, with tradeoffs in quality metrics.
Critical strengths
-
Simple and coherent training procedure. Cached teacher transitions provide explicit regression targets, allowing student training without online teacher evaluation, auxiliary score networks, adversarial objectives, or Jacobian-vector products.
-
Robust Pipeline across tasks and model sizes The experiments cover both a 2.5B image model and a 22B audiovisual model, showing that the training recipe is compatible with different architectures and generation tasks.
-
Concrete inference-speed measurements and transfer results. The paper reports detailed latency breakdowns, including encoding and decoding costs, and demonstrates that the video student retains much of the teacher’s performance at spatial resolutions and aspect ratios absent from its training bank.
Critical weaknesses
-
Incomplete accounting of training efficiency. PACT simplifies student training by generating and storing teacher trajectories beforehand. However, constructing the trajectory bank requires substantial teacher computation, particularly for the 22B audiovisual model, while storing intermediate states adds storage and data-loading costs. These costs are amortized through reuse, whereas online distillation incurs teacher computation during training. Without reporting bank-generation cost, storage requirements, and total training GPU-hours under matched quality targets, the overall efficiency advantage remains unclear.
-
Limited methodological novelty. The method adapts existing offline flow-map regression to large pretrained models through trajectory caching, LoRA, and destination-time and guidance conditioning. This provides a useful practical training recipe, but the paper offers limited new theoretical insight into why this approach succeeds at scale. It also lacks a controlled comparison with JVP-based flow-map distillation using the same backbone and adaptation architecture, leaving the relative benefits of cached-transition regression unclear.
-
Weaker compositional accuracy and preference scores on T2I tasks. At four steps, PACT achieves better COCO FID than DP-DMD (20.60 versus 25.45), but lower GenEval accuracy (0.622 versus 0.649), COCO ImageReward (0.753 versus 0.988), and Pick-a-Pic ImageReward (0.921 versus 1.128).
Questions for authors
-
Have the authors compared cached-transition regression with a derivative-based flow-map distillation objective under the same SD3.5 setup? The existing teacher, cached source states, and student architecture could be reused to isolate the supervision objective. Comparing generation quality, teacher correspondence, and training cost would clarify the benefit of the proposed regression scheme over existing flow-map distillation.
-
How were the interval-sampling probabilities chosen? Does this sampling policy improve results compared with uniformly sampling valid state pairs from the same trajectory bank?
4 comments
Thank you for the detailed response and additional experiments. The compute accounting and uniform-sampling ablation largely address my questions, and the JVP comparison provides useful practical evidence within the tested setup. The six-step results also help clarify the trade-off between generation quality and inference cost. I have no further major concerns regarding these points.
My assessment of the methodological novelty remains unchanged, and I will maintain my score.
Thank you very much for taking the time to review our rebuttal and additional experiments, and for following up during the discussion period. We are glad to hear that the compute accounting, uniform-sampling ablation, JVP comparison, and six-step results addressed your empirical questions and clarified the quality–inference cost trade-off.
We really appreciate your thoughtful review and constructive feedback throughout the review process.
Best regards, The Authors
Thank the authors for the response, my concerns on performance side are resolved. However, my overall concern on the novelty remains, I will therefore keep the score
We thank the reviewer for their time and for recognizing the simplicity of our method and the breadth of our experiments. We address their concerns below.
1. Training efficiency and derivative-based comparison.
We appreciate the request for detailed compute and storage accounting. We report the following:
We have not optimized bank size or training duration, and data-loading overhead was not separately measured, but it is highly cluster-relevant and was not a bottleneck for us. These details are now included in the appendix.
For the derivative-based comparison, we implemented a MeanFlow-style JVP objective under the same SD3.5 PACT setup, changing only the supervision objective. Despite stabilization attempts (detached targets, FP32 JVP computation, and FP32 trainable parameters/optimizer states), training diverged in all experiments. Moreover, JVP required ~14× longer per update (3.67 s vs. 0.267 s, same 16 GPUs and global batch size 32). This comparison excludes PACT's one-time bank-generation cost, reported above. While divergence prevents a quality-matched comparison, these results highlight the practical stability and training-speed advantages of PACT in this setup. We have added these findings to the paper.
2. Methodological novelty.
We agree that flow maps are not new. Our contribution is a simple, scalable offline distillation recipe demonstrated on large pretrained image and video models. Compared with the widely used DMD family, PACT avoids complex training components while better preserving teacher correspondence and visual diversity, with competitive generation quality. We believe the new JVP comparison further supports its practical value. Generalization to unseen resolutions and flexibility in inference steps and sigma schedules are additional useful properties.
3. Compositional accuracy and preference scores.
We acknowledge the lower four-step scores, as already discussed in Sec. 4.1 (lines 360–363), alongside the improvements in teacher correspondence and diversity. However, the gaps narrow considerably with just two additional steps:
While this requires six rather than four inference steps, it shows that quality improves substantially with a modest increase in step count, if a usecase requires higher level of quality.
4. Interval-sampling probabilities.
The probabilities (0.3, 0.3, 0.3, 0.1) were chosen to balance jump lengths and emphasize longer transitions needed for few-step inference. Following the reviewer's suggestion, we trained an SD3.5 student with uniform sampling under identical settings. At four inference steps, both models achieve nearly identical results: GenEval 0.622 vs. 0.620, COCO FID 20.60 vs. 20.66, and COCO ImageReward 0.753 vs. 0.749 (PACT vs. uniform). Diversity and teacher correspondence are also nearly unchanged. Predicted-velocity MSE on unseen trajectories is similarly close across all jump buckets, with only a small advantage for our original policy in the third bucket (0.025 vs. 0.027). These results suggest that PACT is robust to the sampling distribution, with no substantial benefit from our weighted policy in this setting. We have added the ablation to the appendix.