- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes a novel framework for non-uniform diffusion training, to accelerate convergence and the final performance. The paper proposes to understand and analyze the principle of non-uniform training from a multi-task learning perspective. It uses task affinity as a proxy to measure the importance of timesteps, which can achieve adaptive and dynamic timetep sampling based on the online task affinity. Experiments show it can improve the convergence and final performance on the reported models and benchmarks.
Critical strengths
Strengths
- The motivation is intriguing and reasonable. Non-uniform training for diffusion models is a long-standing direction. However, as stated in the paper, there is no clear consensus on which timesteps are truly important. This paper proposes to analyze and understanding it from a dynamic multi-task learning perspective.
- The evidence of task affinity supports the motivation, and helps us understand how non-uniform training schemes work and why they can be unstable.
- Finally, based on the task affinity, the paper proposes an effective dynamic timestep sampling strategy. The method can improve the convergence and final performance on the reported models and benchmarks.
Critical weaknesses
Weaknessses
-
Lack of experiments on SoTA models and training paradigms. The training paradigms has shifted to flow-matching, while the paper only tests DDPM-like schedule. Besides, the involved models and datasets are relatively small, and the text-to-image models (at least on MSCOCO) is not used. One of the key contributions claimed in the paper is the generalization and stability of the proposed non-uniform method. Therefore, lack of these experiments may limit the practicability of the proposed method.
-
The paper claims a positive relationship between task affinity and the training convergence. However, it is mainly supported by visual comparison rather than quantitative results. Including rigorous quantitative analysis would strengthen the paper.
-
Although Figure 4 visualizes the task affinity changes during the different training iterations, the paper does not provide corresponding sampling probability curves to highlight the actual sampling difference between the proposed method and existing methods.
Questions for authors
Please see Weaknesses
Dear Reviewers,
Thank you for your time in evaluating this paper. Since there is only one day left for this stage, could you please read the rebuttal and provide a response? This is important for the review process.
AC