- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper investigates the role of the noise schedule and loss weighting in training diffusion models. It shows that a number of popular diffusion and flow model frameworks differ little once these two choices are aligned. It then proposes an adaptive schedule that follows the non-uniform shape of the loss across noise levels during training, arguing that this beats all non-adaptive approaches. It reports improved performance on three datasets over non-adaptive schedules when sufficient training budget is available.
Critical strengths
- Clear premise and execution
- The paper has a clear narrative structure at a high level, and the sections deliver on that structure adequately. The paper explores the premise in detail and with clarity, especially in Section 4.
- Reproducibility
- The paper provides what looks like the entire codebase used for the project, as well as extensive documentation in the appendix.
Critical weaknesses
- Soundness concerns
- The premise of the paper is to investigate adaptive noise schedules/loss weights. Although it mentions several existing approaches in the related work, the ALBIS derivation does not discuss them beyond stating that it builds on Improved DDPM. Furthermore, the results section reports results for MinSNR and EDM2, but a non-adaptive approach beats them in every setting. This apparently contradicts the result that adaptive schedules are preferable and indicates a problem with the experimental setup. The text in this section discusses how and why ALBIS beats the non-adaptive approaches, whereas MinSNR and EDM2 are not mentioned at all, outside of their setup. It seems that EDM2 might have needed more training budget, for the same reason as ALBIS, but it was not included in the “longer training” analysis, again only comparing to the non-adaptive schedules. This leaves me with major concerns for the soundness and therefore significance of the paper, as existing adaptive approaches are the ones to beat.
- Weak originality
- The paper clearly establishes that noise schedules in diffusion models should be adaptive. However, that was already evident from prior literature. As the comparison to existing adaptive schedules is insufficient, the paper effectively does not provide advancements over existing approaches. I'm not convinced that this approach is better than the adaptive loss weighting of EDM2 for SoTA diffusion models.
Questions for authors
Future work:
- Following from my concerns about originality, to me, a more interesting direction would be how existing adaptive approaches differ, which is best, and why.
Clarification:
- The EDM2 row in Table 2 has very poor performance and significantly underperforms EDM. As I understand it, adding adaptive loss weighting to the same backbone should be a competitive approach. Please provide insight into why you think that was not the case in your experiments.
- How do the roles of the noise schedule and loss weight differ? Mathematically, one can achieve the same objective in expectation with different combinations of these, which results in different training dynamics. You mention in Appendix B.7 that the comparison with EDM2 “separates adaptive weighting from adaptive sampling.“. Please clarify what your results tell us about the relationship between these two components.
7 comments
Rebuttal
The EDM2 row in Table 2 has very poor performance and significantly underperforms EDM. As I understand it, adding adaptive loss weighting to the same backbone should be a competitive approach. Please provide insight into why you think that was not the case in your experiments.
We believe the adaptive loss weighting of EDM2 is not a good choice as adaptive sampler. It keeps the noise sampling distribution fixed and learns an additional loss multiplier for each noise level. At the ideal optimum for a fixed denoiser, this multiplier is the inverse of the expected EDM-weighted denoising loss, so it increases as that loss decreases. In our experiments, we observed that the easy noise levels in the middle increasingly dominated training. This contrasts with the shift towards harder noise levels in the loss-based importance sampling experiments. We are therefore not surprised by its poor performance. Could you explain why you expect this weighting to improve performance?
How do the roles of the noise schedule and loss weight differ? Mathematically, one can achieve the same objective in expectation with different combinations of these, which results in different training dynamics. You mention in Appendix B.7 that the comparison with EDM2 “separates adaptive weighting from adaptive sampling.“. Please clarify what your results tell us about the relationship between these two components.
On paper, the sampling and the weighting should cancel out. However, as seen in Appendix C.4 (Tables 10 and 11), we can get very different results even though it is the same objective in expectation. So we believe sampling and weighting should always be analysed as two different levers and not seen as equivalent. In particular, sampling seems to have a much stronger effect than weighting on the final outcome.
Reply to rebuttal
Thank you for the detailed response and for being open about the planned rewrite. I think reframing the paper as a study of how known adaptive techniques improve fixed logit-normal training is the right direction. Regarding the discussion:
-
MinSNR. You are right, MinSNR uses a fixed weighting rule. I should not have grouped it with the adaptive approaches. My concern still applies to EDM2, which is the only adaptive baseline in the paper.
-
EDM2. I expected EDM2's approach to be competitive because its learned weighting is part of the final configuration of a recent state-of-the-art ImageNet model, and when reading the EDM2 paper, it becomes clear that they turned every stone they could to achieve the performance they did. Your explanation is plausible: the inverse-loss weight upweights noise levels whose loss is already low, on top of a sampler that already concentrates on the middle. However, your reproduction differs from the original in several ways that may matter here: no magnitude-preserving layers, a different uncertainty head, a noise distribution tuned for ImageNet-512 latents, and EDM’s native grid at inference. I understand that you did this to preserve comparability between the approaches, but that logic only holds up if the part you test is independent of the rest of the setup. It seems that EDM2's loss weighting approach significantly depends on other parts of EDM2's setup. The comparison therefore isn't fair, and either EDM2 should be removed (and commented on why), or the comparison needs to be adjusted, so that the approach can be properly analysed. That matters especially in the framing of the paper.
-
Sampling vs. weighting. Appendix C.4 shows convincingly that runs with the same objective can reach very different results, which I personally find a very interesting result, as I have played around a lot with this dynamic in the past. However, I don’t think it supports the claim that sampling generally has a stronger effect than weighting. With middle emphasis, weighting wins in all six settings. With tail emphasis, sampling wins in five of six. As you note there, the run with more tail samples is the better one in both cases. The C.4 experiments also use fixed tilts only. Separately, the EDM2 comparison doesn't seem to separate adaptive weighting from adaptive sampling, because the two methods also differ in which direction they shift the emphasis (inverse-loss vs. loss-proportional), in the base distribution, and in the parameterization. The cleanest control would be to apply Albis’s loss-based emphasis as an adaptive loss weight under fixed sampling, with everything else unchanged.
-
Prior work. I agree that adaptive samplers are rarely used in practice and that earlier papers reported modest gains. Showing that they are an easy way of gaining consistent improvements over the current default would be a useful contribution. To support that claim, the paper would need comparisons against the adaptive and curriculum methods it already cites (e.g., Xu et al., 2024; Kim et al., 2025a,b; Sun, 2026; Raya et al., 2026).
I wish you the best with the revised version.
Rebuttal
Thank you for your review. We have decided to rewrite the paper to make our contribution clearer. Based on the feedback, multiple reviewers understood the paper as proposing a new method. Our goal is to show that fixed logit-normal flow-matching, which is currently the standard for any industry diffusion model released this year, can be improved using known techniques from the literature, and to study why these techniques help. We believe our work is a valuable contribution, but our current writing suggests a direction we didn't properly deliver on. We will withdraw the paper at the end of the rebuttal period. We would still like to use the discussion to clarify the experiments and understand your suggestions for improving the paper.
We understand that our writing makes the paper read as an investigation of adaptive noise schedules, but the main message we wanted to convey was: if you are using logit-normal sampling with a fixed noise schedule right now, you can get free performance by adding an adaptive sampler. Loss-based importance sampling works well in our experiments, but that doesn't mean EDM2's adaptive weighting will.
EDM2 was not included in the longer training analysis because its performance was poor, so we didn't consider it worthwhile given our compute constraints. It is included in the ImageNet experiments. If EDM2 needed more training budget, we would have seen it there. MinSNR uses a fixed loss-weighting rule, not an adaptive one.
We don't really agree that prior literature suggests diffusion models should be adaptive. First of all, the iDDPM and Kingma and Gao papers both report only marginal improvements from using their adaptive samplers. Furthermore, we reviewed the major open-source industry papers from this year, and no one is using adaptive samplers.
We agree that comparing existing adaptive approaches would fit well with our paper. We will definitely add that alongside a major rewrite of our paper. Thank you for the suggestion.