acceptodds
Withdrawn from ICLR 2027

Towards Efficient Diffusion Training with Noisy Labels

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Label noise can disrupt the correspondence between images and class labels, degrading conditional alignment and generation quality in diffusion models. Transition-aware weighted Denoising Score Matching (TDSM) addresses this problem by estimating instance- and time-dependent clean-label posterior probabilities using a label transition matrix and a classifier, and using these probabilities to aggregate class-conditional denoising predictions. However, this mechanism requires a separate denoiser evaluation for each class involved in the aggregation, incurring substantial training overhead when the number of classes is large. To reduce this cost, we propose mixed-label conditioning, which combines learnable class embeddings according to the estimated clean-label posterior probabilities and conditions the denoiser on the resulting mixed representation. By encoding label uncertainty directly in the conditioning input, our method requires only a single forward pass for each mixed-label prediction, avoiding explicit class-wise prediction and output aggregation. To further improve training efficiency and effectiveness, we introduce mixed diffusion-model-guided sample selection, which compares the diffusion losses of mixed-label and reference predictions to select training examples that benefit from conditioning at the sampled noise level. Experiments on CIFAR-10 and CIFAR-100 under noisy-label settings evaluate generation quality, conditional alignment, and computational cost. The results demonstrate generation performance comparable to that of TDSM with substantially reduced training overhead.

Primary area:
generative models
Keywords:
diffusion model, noisy label

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_fVzQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the issue that when the number of categories is large in the label noise problem, TDSM incurs significant computational overhead. The authors propose a hybrid label conditioning approach, where the posterior weights are used to linearly combine the learnable category embeddings, and then the mixed conditional prediction is obtained through a forward computation of the denoising network once, thereby reducing the need for individual category calculations. The paper reports the generation quality, category condition consistency, and training time per round under the synthetic label noise settings of CIFAR-10 and CIFAR-100.

Critical strengths
  1. The author addressed the bottleneck of TDSM, which involves multiple forward calculations of category conditions. By changing the output mixture to category embedding mixture, the repetitive calculations of the denoising network can be reduced, without adding additional modules in the generation stage. This significantly reduces the computational bottleneck and has excellent practical application value.
  2. The author's experiment was quite comprehensive. Besides generating indicators, the author also reported category-level indicators and included the additional reference predictions required for sample screening in the efficiency statistics. The evaluation dimensions were quite comprehensive.
Critical weaknesses
  1. By screening samples based on the reconstruction benefits of the current model, there is a possibility of continuously favoring samples that are easier to model, while reducing the training opportunities for difficult samples, rare patterns, or certain noise intervals. The author should further explain how this model avoids the problem of internal imbalance within categories when dealing with rare samples and other issues.
  2. There is insufficient evidence to explain why mixed-condition training can restore the clean distribution of a single category. The statistical interpretation of TDSM relies on the explicit weighted combination of the scores of each clean category. The manuscript replaces this structure with a general nonlinear function: $s_\theta(x_t,v,t) = F_\theta!\left(x_t,e_{\mathrm{time}}(t)+\sum_i v_iE_i\right). $ The authors correctly point out that the two forms are usually not equivalent. However, the subsequent training directly constrains the mixed-condition prediction corresponding to the estimated posterior v=w^(xt,y~,t)v = \hat w(x_t,\tilde y,t), while the category vertices uyu_y are used during generation. During the noisy stage where the label posterior is less clear, there may be significant differences between the two types of conditions.
  3. The article mainly focuses on comparing DSM and TDSM, but has not fully covered the recent methods for generating noise labels. Although the paper discusses related works such as SBDC and SPFM, there is a lack of corresponding comparisons or detailed explanations that are not directly comparable, making it difficult to fully assess the relative advantages of the proposed method.
  4. The experiment was conducted only on the two low-resolution datasets, CIFAR-10 and CIFAR-100, and used artificially constructed category-conditioned label noise. The authors should conduct tests on higher-resolution datasets, such as ImageNet, to make their experimental results more convincing.
Questions for authors
  1. The zero-condition branch has no explicit unconditional training objective. How can it serve as a reliable screening reference?
  2. The author in Equation (12) acknowledges that the embedding mixture and output mixture in nonlinear networks are not equivalent. Furthermore, the training mainly uses soft mixed conditions, while generation uses one-hot conditions. The connection between the two still needs to be explained.
Reviewer_HUMNOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates efficient training of class-conditional diffusion models under noisy labels. Building upon transition-aware weighted denoising score matching (TDSM), the authors propose mixed-label conditioning, which replaces explicit class-wise score aggregation with a weighted combination of class embeddings. This modification allows the denoising network to process label uncertainty through a single mixed-condition prediction, thereby reducing the computational overhead associated with multiple class-conditional evaluations. Additionally, the authors introduce diffusion-model-guided sample selection, which retains training samples when mixed-label conditioning achieves a lower reconstruction loss than a zero-condition reference. Experiments are conducted on CIFAR-10 and CIFAR-100 under synthetic symmetric and asymmetric label noise. The reported results demonstrate reduced training time compared with TDSM while maintaining competitive generation quality in several evaluated settings.

Critical strengths
  1. Practical computational efficiency. The paper identifies a concrete computational bottleneck in TDSM and proposes a straightforward approach to reducing repeated denoiser evaluations. The reported training speedups, particularly on CIFAR-100, demonstrate the practical value of the proposed computational simplification.
  2. Clear presentation and empirical efficiency analysis. The overall framework is clearly presented, and the experiments report both generation quality and computational cost. The comparison between mixed-label conditioning and explicit score aggregation provides useful preliminary evidence regarding the feasibility of the proposed approximation.
Critical weaknesses
  1. Limited Methodological Novelty. The methodological novelty of this work is limited. The proposed mixed-label conditioning essentially replaces the output-level weighted aggregation in TDSM with a weighted combination of class embeddings, which represents a relatively straightforward computational simplification rather than a fundamentally new approach. Additionally, the diffusion-model-guided sample-selection strategy follows a principle similar to existing loss-based selection methods. Overall, the framework primarily combines and adapts existing techniques without introducing sufficiently distinctive methodological innovations.
  2. Insufficient Experimental Validation. The experimental evaluation is relatively limited, relying exclusively on CIFAR-10 and CIFAR-100 with synthetic label noise. Experiments on more challenging datasets, higher-resolution images, and realistic noise settings are absent, while comparisons with alternative noise-robust generative methods remain insufficient. Notably, the CIFAR-100 setting with 40% symmetric noise is excluded from the main results due to substantial performance degradation. Furthermore, the proposed method consistently underperforms TDSM in terms of FID across the reported noisy-label settings on CIFAR-10. These limitations weaken the evidence supporting the method's robustness and generalizability.
  3. Lack of Substantial Theoretical Contributions. The paper lacks sufficient theoretical analysis to support its central methodological modification. Since diffusion denoisers are nonlinear, embedding-level mixing is not mathematically equivalent to output-level score aggregation, as acknowledged in Eq. (12). Although the authors provide empirical prediction-agreement results, no theoretical guarantees, approximation-error bounds, or sufficient conditions are established to justify this approximation. Moreover, it remains unclear whether the proposed objective preserves the clean-label score recovery properties of TDSM. Consequently, the theoretical contribution is limited and does not adequately support the proposed framework.
Questions for authors
  1. Methodological novelty. The proposed mixed-label conditioning replaces output-level score aggregation with embedding-level mixing, while the sample-selection strategy follows a loss-comparison principle similar to existing methods. Could the authors clarify the fundamental methodological novelty of the framework beyond computational simplification and the combination of existing techniques?
  2. Theoretical contribution. Since embedding-level mixing and output-level score aggregation are generally not equivalent for nonlinear denoising networks, can the authors provide theoretical conditions, approximation-error bounds, or other formal analysis supporting the proposed replacement? In particular, is there any guarantee that the proposed objective preserves the clean-label score recovery properties established for TDSM?
  3. Experimental generalization. Why are the experiments restricted to CIFAR-10 and CIFAR-100 with synthetic label noise? Is there evidence that the proposed approach remains effective on more challenging datasets, higher-resolution images, or realistic label corruption?
  4. Robustness under severe label noise. Could the authors provide the complete results for CIFAR-100 under 40% symmetric label noise and explain the substantial performance degradation? How does this limitation affect the claimed robustness of the proposed framework?
  5. Contribution of sample selection. Can the authors provide a more systematic analysis of the sample-selection mechanism, particularly regarding the retention rates of correctly and incorrectly labeled samples across different diffusion noise levels? Such evidence is important for determining whether the selection strategy genuinely improves robustness to label noise.
Reviewer_kgqLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The work addresses the issue of label noise in generative diffusion models wherein mistaken labels degrade the generation quality. Referencing the existing work of TSDM, which solves the problem by constructing weights that estimate the conditional probability per sample per class of the correct class (posterior over the true class with a separate classifier), leading to a weighted score estimate, this work proposes mixed-label conditioning, incorporating learnable class embeddings that side-steps several forward passes to estimate individual weights with fewer. Sample selection during training isolates those that benefit from this conditioning as measured by losses. The resulting method is more efficient at comparable performance and cheaper to train.

Critical strengths
  1. The proposed method does add significant efficiency gains to the network's training. On Cifar-100 the method is 3.84x faster and should grow with larger number of classes.
  2. The gains on Cifar-100 seem significant against DSM and TDSM at 20% symmetric noise. The method seems to do well when a large number of semantically close classes are a part of the dataset.
Critical weaknesses
  1. The work may be incremental over TDSM and SPFM and mixing while matching TDSM's speed fails to match in quality as evinced in Table 3. It is likely that the gains in the Table 1 could be due to the selection step alone which seems to be SPFM's conditional-vs-unconditional loss comparison for a diffusion loss.
  2. The authors could strengthen the contribution upon incorporating SPFM with the TDSM and DSM methods and isolate the contribution of the mixing objectives.
  3. The error bars aren't reported over multiple runs which could fortify the improvements that the method demonstrates.
  4. If selection is done using loss as a proxy for label noise, the authors could isolate confounding factors that could lead to a similar spike in loss such as difficulty of the sample, network incompetence and noise in the initial stages etc. High density scores with parity in coverage may hint at the network focussing over typical samples that pass through the filter and atypical ones filtered out. Incorporating the filter based metrics (retention rate, precision, recall etc.) could help.
Questions for authors
  1. Could the authors add DSM and TDSM + selection as additional baselines to help quantify and isolate the contribution/loss due to mixing?
  2. What if TDSM was augmented with top-k class filter that matches the latency of this work?
  3. Could the authors kindly include some filter stats like the proportion of samples dropped, evolution of its precision/accuracy metrics as training progresses etc.?
  4. More seeds and averaging remove more confounders for the results. Could the authors incorporate those for some tables?
  5. If the density rises to over 95% in C-10 with clean labels, is it likely that the filter is isolating easier typical samples for loss and excluding the atypical ones? Would a warm-up without masking help here?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.