- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies how the distribution of training noise levels affects one-step consistency tuning under limited training budgets. It shows that a small average consistency loss need not imply low generation error, develops an error bound along noise levels below the generation start, and introduces a simplified chain model linking training allocation to convergence speed. Motivated by this analysis, the authors propose Uniform Consistency Tuning (UCT), which replaces 25% of ECT’s noise-level samples with uniform draws over a specified interval. Experiments on CIFAR-10 and ImageNet-64 demonstrate improvements over ECT at small training budgets, although the ImageNet-64 advantage disappears with longer training.
Critical strengths
- Simple and inexpensive modification: UCT is easy to incorporate into ECT and adds negligible per-update computational overhead.
- Relevant theoretical question: The paper examines why low average consistency loss may fail to predict generation quality and offers an interpretable account of how training allocation affects convergence.
- Consistent small-budget improvements: UCT improves CIFAR-10 results across six paired seeds, under both data-matched and pure-noise initialization, and shows gains on ImageNet-64 at smaller budgets.
- Useful controlled experiments: The study compares alternative sampling distributions, mixture proportions, and interval endpoints, alongside longer-training runs and an additional DINOv2-based evaluation metric.
Critical weaknesses
Overall, I find UCT’s contribution over prior work limited for the following reasons:
-
Limited theoretical support for the proposed modification: The methodological contribution consists of adding a uniform component to ECT’s noise-level sampling distribution. Although this simple change improves performance at small training budgets, its theoretical justification remains indirect. The convergence analysis assumes independently parameterized outputs across noise levels, and the resistance criterion provides an error bound within this simplified model without establishing improved convergence for shared-parameter neural networks. Moreover, the rate-monotonicity result does not directly establish UCT’s advantage, since the mixture decreases sampling rates at some noise levels. The theory therefore motivates the sampling heuristic but does not establish its superiority over ECT.
-
Diminishing gains with increased training budget: On CIFAR-10, increasing training from 8K to 100K updates reduces UCT’s FID improvement over ECT from 0.41 to 0.13 under data-matched initialization (5.23→4.82 versus 4.13→4.00), and from 1.01 to 0.20 under pure-noise initialization (7.35→6.34 versus 4.50→4.30). The 100K-update results use only one paired seed, leaving the robustness of these smaller gains uncertain. On ImageNet-64, the advantage disappears at 100K updates. Although these findings are consistent with the paper’s focus on limited training budgets, they leave unclear whether UCT provides substantial benefits at budgets needed to achieve stronger generation quality.
-
Limited evidence of scalability and generality: Evaluation is restricted to CIFAR-10 and ImageNet-64, whereas prior work, including ADCM [1] and CMT [2], already reports ECT/ECM baselines on ImageNet-512. A matched higher-resolution comparison would help establish whether UCT retains its efficiency benefits in more demanding settings. Its applicability to text-to-image and video generation also remains an open question: conditioning and temporal consistency introduce additional requirements that the current experiments and theoretical analysis do not address. While evaluating all these tasks may exceed the paper’s scope, the present evidence supports a narrow conclusion about UCT’s practical applicability.
[1] Bai et al. “Adaptive Discretization for Consistency Models.” NeurIPS, 2025.
[2] Hu et al. “CMT: Mid-Training for Efficient Learning of Consistency, Mean Flow, and Flow Map Models.” ICLR, 2026.
Questions for authors
-
How was the mixture weight π = 0.25 selected, and does this choice remain effective across datasets and longer training budgets?
-
Why is the uniform sampling interval fixed to [0.002, 5] when generation starts from pure noise at level 80? How should this interval be selected for a new dataset?
Response to Rebuttal
Thank you for the response.
My main concerns remain, however. The theoretical results are largely adaptations of known arguments, and the lognormal baseline is still not tuned. The new results also suggest that the benefit of UCT is limited to one-step generation at small budgets: it does not improve two-step FID, nor one-step FID from pure noise on ImageNet-64. I will therefore keep my score.
I understand that adding new datasets during the discussion period is difficult, and I hope the following is useful for a future version. The paper is motivated by the cost of training one-step generators, and that cost matters most for high-resolution and latent-space models, where a better training path would have the most practical value; recent consistency model work also routinely reports results at that scale. Evidence in such a setting would make the contribution much more convincing. If the claims are instead meant to cover only pixel-space settings at small budgets, it would help to state this explicitly as a limitation. I encourage the authors to strengthen the evaluation before resubmitting.