acceptodds
Withdrawn from ICLR 2027

Can We Break the Loop? Active Boundary Restoration for Learning with Noisy Labels

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Most methods for learning with noisy labels identify reliable samples through training loss, predictive confidence, or model agreement. These reliability estimates depend on an evolving representation: as noisy supervision distorts class geometry, hard clean examples and mislabeled examples can become increasingly difficult to distinguish near class boundaries. Robust learning therefore requires maintaining a representation in which sample reliability remains identifiable. We propose HamBR, an active boundary restoration framework that explores low-support inter-class regions and constructs virtual boundary anchors. Constrained spherical Hamiltonian trajectories generate structured candidates along the current feature geometry, while an explicit density-retention rule controls their support relative to inter-class initialization. Geometric regularization then draws reliable features toward their class cores and away from the anchors. Theoretical analysis establishes a relative support-density bound, characterizes optimization of prototype–outlier margins, and provides an anchor-induced boundary-separation guarantee. Our evaluation examines classification performance, representation geometry, and alternative candidate-generation strategies under noisy supervision. Across synthetic- and real-noise benchmarks, the reported results support the benefits of active geometric intervention for robust learning. These findings motivate considering sample selection and representation geometry jointly, with active boundary restoration complementing advances in selection rules.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Learning with noisy labels

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_KQoaOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes HamBR for learning with noisy labels, based on the idea that reliable sample selection requires a sufficiently well-structured latent space. It therefore performs sample selection in the learned representation space while actively maintaining that space through a dynamically filtered support bank and boundary-oriented regularization, pulling reliable samples toward class prototypes and pushing them away from low-density virtual boundary anchors.

Critical strengths
  1. The paper provides a clear representation-centric perspective on noisy-label learning. Rather than only improving the sample-selection rule, the paper highlights that reliable selection itself depends on the quality of the learned feature space, and proposes to actively maintain class geometry during training. This perspective is well motivated and leads to a coherent framework that couples reliability estimation with boundary-oriented regularization.
  2. The empirical evaluation is relatively comprehensive and supports the proposed mechanism. Beyond classification accuracy, the paper analyzes hard-clean sample retention, feature-space separation, alternative candidate-generation strategies, and OOD behavior. These experiments provide useful evidence that HamBR improves both predictive performance and the underlying representation geometry.
Critical weaknesses
  1. The experimental protocol weakens the paper’s central motivation and makes the results difficult to interpret. The paper uses a pre-trained ResNet-18 on CIFAR-10/100, rather than the commonly adopted from-scratch setting in noisy-label learning, but does not specify the source of pre-training. This is particularly concerning because the paper is motivated by the degradation of representation geometry under noisy supervision, whereas a pre-trained backbone already provides a relatively well-structured representation before noisy-label training begins. Moreover, despite this favorable initialization, the paper does not provide controlled comparisons with recent strong LNL methods such as JAL [1] and CA2C [2]. Therefore, it remains unclear whether the reported gains persist under the standard from-scratch setting and how competitive HamBR is against recent methods under a unified protocol.

  2. The writing, figure quality, and table presentation need improvement.

[1] J. Wang et al. Joint Asymmetric Loss for Learning with Noisy Labels. ICCV, 2025. [2] M. Sheng et al. CA2C: A Prior-Knowledge-Free Approach for Robust Label Noise Learning via Asymmetric Co-learning and Co-training. ICCV, 2025.

Questions for authors

Please see weaknesses.

Reviewer_yEfPOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

In this paper, the authors study learning with noisy labels, particularly from the perspective of sample selection. A main motivation is that many existing methods rely on training loss, predictive confidence, model agreement, etc., to identify reliable examples. However, the effectiveness of these criteria depends on the quality of the learned representations, which may themselves be affected by noisy labels. For example, distinguishing hard clean examples from noisy examples can be challenging near class boundaries, where the underlying representation geometry may be distorted. To address this challenge, the authors propose a learning framework that combines a dynamically filtered support bank with geometric regularization. The support bank maintains features of samples considered reliable and is used to estimate class prototypes and feature-space support density. The proposed algorithm then generates virtual outliers through updates guided by a support-density potential. Candidates are retained based on their estimated support density and used in a contrastive objective that encourages reliable features to be closer to their corresponding class prototypes than to the generated virtual outliers.

Critical strengths
  • The high-level idea of addressing noisy-label learning through class-boundary restoration is interesting. The proposed algorithmic design appears sensible.
  • The experimental results are strong on the evaluated benchmarks.
Critical weaknesses
  • I found the technical section difficult to follow. The motivation presented in the introduction is reasonable, but the logical development of the proposed method could be presented more clearly. Even after reading the paper multiple times, I found it challenging to understand how the individual components are combined and contribute to help achieve the main goal. This seems to be partly due to the apparently disconnected presentation of individual components and the introduction of Hamiltonian dynamics, which appears to complicate the presentation without sufficiently clarifying its role in the proposed method. In particular, what specific properties of Hamiltonian dynamics are essential for this purpose? Once understood, the underlying procedure appears relatively simple and sensible. The authors also present a boundary-restoration certificate in Section 3.3, but its significance for the proposed method could be explained more clearly.

  • The reported experiments are promising, but the evaluation is limited to established noisy-label learning benchmarks, including CIFAR-10, CIFAR-100, Animal-10N, and Food-101. Given the practical motivation of the proposed algorithm, it would be useful to demonstrate its effectiveness on more challenging large-scale datasets, such as ImageNet. In addition, it remains unclear how well the proposed boundary-restoration mechanism would work in settings with substantial intrinsic overlap between classes.

  • Some important details are deferred to the appendix. For example, Eq. (10) presents the overall optimization objective, but the definitions of several loss terms are only provided in Appendix D. These definitions are essential to understanding the actual training algorithm and should be included in the main paper to make it more self-contained.

Minor Comments

  • The notation P_\phi does not appear to be explicitly defined when it is introduced in Section 3.
Questions for authors

Please provide a detailed discussion of how the three main results presented in Appendix F support the proposed algorithm and its (claimed) boundary restoration. It would be helpful to clarify the significance of each result and how they contribute to understanding the proposed method.

Reviewer_wP9vOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

HamBR addresses feedback between sample selection and representation learning under label noise. It builds a confidence-filtered feature bank, generates virtual anchors through finite spherical Hamiltonian proposals initialized at prototype midpoints with relative support-based retention, and regularizes selected features toward their class prototypes and away from these anchors. Experiments under a 30-epoch budget report gains in classification, hard-clean sample selection, and OOD detection, supported by propositions on relative support, margin monotonicity, and conditional transfer to the classifier.

Critical strengths
  1. The representation–selection feedback perspective is well motivated, and the regularizer complements existing frameworks (+10.46 points on average when added to DivideMix). This supports the practical relevance of the contribution.

  2. The evaluation includes synthetic and real-world label noise, hard-clean diagnostics, and candidate-generation comparisons (prototype midpoints, random perturbations, random walks, gradient search). These provide evidence of practical benefit beyond reducing the auxiliary loss.

Critical weaknesses
  1. The claimed classifier-boundary restoration mechanism remains insufficiently isolated. HamBR operates in a learned projection space z=Pϕ(h)/∥Pϕ(h)∥z = P_\phi(h)/\|P_\phi(h)\|. Although its gradients reach the shared backbone, improvements in the geometric objective could be accommodated largely by the projector.
  2. Classification gains show that HamBR modifies the backbone, but not that it does so through boundary restoration rather than a generic regularization effect.
  3. The classifier-space margin Dt(h)D_t(h) of Proposition 4 is never measured, although the introduction lists actual classifier margins as an evaluation criterion. The feature space used for Fig. 1, Fig. 3(b), and Table 7 is not stated.
  4. Proposition 4 assumes a fixed projection, directional alignment, and bounded classifier-head drift. As acknowledged in Sec. 5, these conditions are not verified during joint training.
  5. With U=−log⁡ρ^KU = -\log \hat{\rho}_K, the force favors increasing support, while retention constrains a different, all-bank score relative to initialization. This is a valid proposal heuristic, but its advantage over explicitly decreasing support remains unclear.
  6. The Fig. 4 alternatives (random perturbation, random walk, gradient search) are insufficiently specified, including whether they share the midpoint initialization, retention rule, and compute budget.
  7. Reported gains over random spherical perturbations are 1.70, 0.45, and 0.50 percentage points; their robustness is difficult to assess without a described repeated-run protocol, especially as the no-HamBR ablation is non-monotonic (≈80.8% at 20% vs. ≈82.3% at 50% symmetric noise).
  8. All methods use a 30-epoch budget, and baselines fall well below their published results (e.g., UNICON 72.8% vs. standard CE 73.4% at 20% symmetric noise), so it is unclear whether the gains persist under standard protocols.
  9. Prototype construction, updates, and class-pair sampling also need clarification.
Questions for authors
  1. What evidence distinguishes boundary restoration in the backbone/classifier from projector adaptation or generic regularization? Reporting Dt(h)D_t(h) with and without HamBR, the feature space used for Fig. 1, Fig. 3(b), and Table 7, and a frozen-projector control would directly address this concern. Is PϕP_\phi shared with Lcon\mathcal{L}_{con}?
  2. Does the gradient-search baseline explicitly decrease support? Please specify the objective, initialization, retention rule, and computational budget of each Fig. 4 baseline, and report variability across seeds.
  3. Do the gains hold under the baselines' standard training protocols?
  4. How are prototypes constructed and updated, class pairs sampled, bank weights defined, and κ\kappa, γ\gamma, TfilterT_{filter}, and εw\varepsilon_w selected?

Overall, I liked the paper, and I am willing to raise my score if my concerns are addressed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.