acceptodds
Under review as a conference paper at ICLR 2027

Elastic Coding: Robust Maximal Coding Rate Reduction under Label Noise

Abstract

Maximal Coding Rate Reduction (MCR²) exhibits striking empirical robustness to corrupted labels, yet the source and limits of this robustness remain unclear. We show that label corruption leaves its global expansion term unchanged, while sample-wise normalisation limits the influence of individual examples. Under a balanced union-of-orthogonal-subspaces model, uniformly flipped labels disperse corrupted examples across off-class directions, allowing the clean semantic subspace to remain the leading eigenspace when the flip rate is below . Robustness weakens when corruption concentrates in an off-class subspace strong enough to compete with the clean one. Motivated by a decomposition of the MCR² gradient, we propose **Elastic Coding**, which relaxes class compression when it excessively opposes global expansion. Across datasets with structured and real-world label noise, Elastic Coding achieves the best performance among the compared baselines in most evaluated settings.

Primary area:
learning theory
Keywords:
Maximal Coding Rate Reduction, label noise, robust representation learning, Elastic Coding

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_7J7YOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies why MCR2 tolerates dispersed label noise and when concentrated errors weaken its class structure. It proposes Elastic Coding, which reduces class compression when its tangent gradient opposes global expansion. Experiments cover synthetic and real-world label noise.

Critical strengths
  1. The spectral analysis gives a clear explanation of why noise structure matters beyond the corruption rate.
  2. The paper carefully separates fixed-feature results from training guarantees, and representation quality from class-name prediction.
  3. Elastic Coding improves over MCR2 in the real-noise experiments. The common centroid readout and linear probes make the comparison more informative.
Critical weaknesses
  1. Does the adaptive rule improve over simply reducing compression?
  2. What links the opposing-gradient ratio to the clean-dominance gap?
  3. The numerical values of tau, beta, lambda_min, delta and coding precision epsilon are not specified, including in Appendix D. How were these selected without clean labels?
  4. Some baselines also perform unusually poorly: AEL reaches 12.35% on CIFAR-20 at 20% noise, versus 65.16% for CE. Please explain the baseline tuning protocol and report mean/std over multiple model and corruption seeds before drawing firm ranking conclusions.
Questions for authors
  1. Does a fixed compression weight of 0.5 give similar results?
  2. Does gradient conflict track loss of clean-subspace dominance?
  3. How are the threshold and compression floor selected?
  4. Are the gains stable across model and noise seeds?
3 comments
AuthorsOpenReview ↗

Response to Reviewer 7J7Y, Part 1

(We apologise for the relatively late response, as the additional experiments required to address the reviewers’ concerns, particularly the multiple-seed evaluations for Table 2, were computationally expensive and time-consuming. We have only just completed these experiments.)

We sincerely thank you for your careful review and positive assessment of our work. We particularly appreciate your recognition of the spectral explanation of noise structure, our distinction between fixed-feature results and training guarantees, and the improvements over MCR² under real-world label noise. We address each of your concerns in turn, with minor rewording of your comments to ensure a clear and logical flow while preserving their original meaning.

Weakness 1 & Question 1: It is unclear whether the adaptive rule improves over simply reducing the compression weight, particularly to a fixed value of 0.50.5.

Thank you for suggesting this comparison. This concern is also addressed in our responses to Reviewer eGrH's Weakness 1 and Reviewer btWb's Weakness 2, to which we kindly refer you for further details. Below, we briefly summarise the key experimental results and findings.

We have added a fixed-λ=0.5\lambda=0.5 baseline across all nine CIFAR-10 span-2 noise rates. Revised Table 1 shows that Elastic Coding achieves higher observed-label accuracy at all six noise rates from 10% to 60%, and higher clean-label probe accuracy at all nine rates. At 50% noise, Elastic Coding achieves 82.78% versus 58.96% observed-label accuracy and 83.49% versus 62.91% clean-label probe accuracy. At higher noise rates (70%–90%), fixed λ=0.5\lambda=0.5 achieves higher observed-label accuracy, although Elastic Coding maintains better clean-label probe accuracy.

The complete weight trajectories in Appendix F, Figure 7 and Table 8 show that Elastic Coding starts near λ=1\lambda=1 and finishes near 0.50.5, demonstrating that the final weight does not reflect the compression applied throughout training. These results show that a fixed weight of 0.50.5 does not reproduce Elastic Coding's performance under the tested protocol.

Weakness 2 & Question 2: The relationship between the opposing-gradient ratio and the clean-dominance gap is unclear.

This concern overlaps with Reviewer eGrH’s Weakness 2 and Reviewer btWb’s Weakness 1; we invite you to consult our responses there for the detailed analysis and experiments.

Specifically, we have clarified that the opposing-gradient ratio is not an estimator of the clean-dominance gap. Under exact orthogonal isotropy, both tangent gradients vanish even when the gap is negative, so losing clean dominance need not activate the controller.

To investigate the relationship beyond this idealised setting, we added 36 fixed-feature configurations and six CIFAR-10 training controls, including clean labels and matched 50%50\% dispersed and concentrated noise. The association between conflict and covariance-contribution gap proxies varies with geometry, noise structure, method, and proxy rank. Clean-label training also activates Elastic, showing that the signal is not specific to label corruption.

Accordingly, Sections 2.3 and 3.3, and Appendix G now distinguish the two roles: the spectral analysis describes fixed-feature subspace competition, while Elastic uses an optimisation-conflict diagnostic to limit compression opposing global expansion. The analysis motivates examining compression, but, as expected, does not guarantee that the controller detects or repairs negative clean-dominance gaps.

AuthorsOpenReview ↗

Response to Reviewer 7J7Y, Part 2

Weakness 3 & Question 3: The numerical values and selection procedure for τ\tau, β\beta, λmin⁡\lambda_{\min}, δ\delta, and coding precision ϵ\epsilon are missing. How were these selected without clean labels?

Thank you for highlighting these missing details. This concern overlaps with Reviewer btWb's Weakness 3 and Reviewer eGrH's Weakness 3, to which we kindly refer you for further implementation details.

Specifically, we have now explicitly documented all settings in Section 2.3 and Appendix E.1:

where d=512d=512 and KK is the number of classes. These settings are shared across all datasets and noise rates, without dataset-specific or noise-rate-specific tuning. The EMA is initialised with the first measured conflict ratio, and the compression weight is clipped to [0.05,1][0.05,1] and held fixed during each encoder update.

Importantly, Elastic Coding does not require clean labels to compute the conflict statistic or update the compression weight. Clean labels are used only for explicitly identified representation diagnostics. However, this does not establish how the original controller settings were selected.

Regarding hyperparameter selection, we generally adopted the settings recommended in the original papers for baseline methods, with exact variants and coefficients documented in Appendix E.3. All baseline and Elastic Coding settings were kept fixed throughout the additional multi-seed experiments.

Weakness 4 & Question 4: Some baselines perform unusually poorly, particularly AEL. The baseline tuning protocol should be clarified, and results across multiple model and corruption seeds are needed before drawing firm ranking conclusions.

Thank you for raising this concern. It overlaps with Reviewer eGrH’s Weakness 3, and we invite you to consult our response there for the detailed protocol and implementation checks.

To be specific, for the baselines, we generally adopted the hyperparameters recommended in their original papers. Appendix E.3 now specifies the exact loss variants and coefficients, which were held fixed for the additional seeds. We also checked loss values and gradients against pinned reference implementations on 24 fixed-input configurations; no discrepancies were found at the tested precision. These checks do not establish optimal tuning or reproduce each method’s complete original training recipe.

Revised Table 2 reports means and sample standard deviations over five seeds (42–46) for every method–setting combination. Synthetic experiments vary paired model and corruption seeds; real-noise experiments vary training randomness while retaining the observed labels and fixed data subsets.

AEL’s poor performance persists across seeds: on CIFAR-20 at 2020% noise, it achieves 12.22±1.68%12.22\pm1.68\%, compared with CE’s 65.52±0.39%65.52\pm0.39\%. Its cause remains unresolved, and we explicitly acknowledge that the disclosed training configuration may not suit every baseline equally well.

Across the revised comparisons, Elastic exceeds MCR² in mean accuracy in all six settings and achieves the highest mean in four of six; CE leads in both Imagenette settings. We therefore frame these results as comparisons under the disclosed configurations, without claiming uniform superiority, or reproduction of each baseline’s best published performance.

Reviewer_7J7YOpenReview ↗

Response

Thank you for adding the fixed-weight comparison and multi-seed experiments. These strengthen the empirical evidence, though the advantage over a simple weight schedule, the connection between gradient conflict and the spectral analysis, and the reasons for the unusually weak baseline results remain unclear. I appreciate the authors’ acknowledgement of these limitations and maintain my weak acceptance rating.

Reviewer_btWbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper examines how the structure of label noise affects MCR². It derives a spectral condition for preserving the clean class subspace and introduces Elastic Coding, which weakens class compression when its gradient opposes global expansion. Experiments cover synthetic and real label noise.

Critical strengths
  1. The distinction between dispersed and concentrated label errors offers a useful explanation of MCR^2 robustness.
  2. Elastic Coding is simple and improves over MCR^2 across several settings.
Critical weaknesses
  1. The theory does not yet justify the conflict signal. Under the exact isotropic, orthogonal subspace population model, both coding gradients are radial, so their tangent projections vanish even when the clean dominance gap is negative. Losing clean dominance therefore need not activate the proposed correction. This does not refute the propositions, but Sections 2.2 and 2.3 need evidence connecting the spectral criterion to the conflict signal during training.

  2. The benefit of adaptation is unclear. Table 1 reports final compression weights between 0.483 and 0.543. A fixed weight near 0.5 is an important missing baseline. Comparing it with the adaptive rule under matched tuning conditions would help establish whether the gains come from adaptation or simply weaker compression.

  3. Key settings are missing. The paper does not report numerical values for the conflict threshold, smoothing coefficient, compression floor, numerical stabilizer or coding precision. It also leaves their selection procedure unclear. These details are needed to reproduce the results and assess how much tuning the method requires.

Questions for authors

Please refer to the weaknesses.

2 comments
AuthorsOpenReview ↗

Response to Reviewer btWb, Part 1

(We apologise for the relatively late response, as the additional experiments required to address the reviewers’ concerns, particularly the multiple-seed evaluations for Table 2, were computationally expensive and time-consuming. We have only just completed these experiments.)

We sincerely thank you for carefully evaluating our work and recognising the importance of distinguishing dispersed from concentrated label errors, as well as the simplicity of Elastic Coding and its improvements over MCR². Below, we address your concerns point by point. For clarity and readability, we have slightly rephrased your comments while preserving their original meaning.

Weakness 1: The theory does not justify the conflict signal. Under exact orthogonal isotropy, tangent gradients vanish even when the clean-dominance gap is negative, so losing clean dominance need not activate the controller. Evidence connecting the spectral criterion to training-time conflict is needed.

Thank you for this precise observation. This concern also overlaps with Reviewer eGrH's Weakness 2, and we kindly refer you to our response there for the detailed experiments and discussion.

Briefly, we conducted 36 fixed-feature experiments covering four representation geometries and six additional CIFAR-10 training controls comparing MCR² and Elastic Coding under clean labels, 50% dispersed noise, and 50% concentrated noise. We track gradient conflict, compression weights, and covariance-contribution gap proxies throughout training.

The results show that gradient conflict does not universally track spectral deterioration. Its relationship with the gap proxy varies across geometries, noise structures, methods, and proxy ranks. Interestingly, gradient conflict also occurs under clean labels, with Elastic Coding exhibiting a final-epoch mean conflict of 1.8711.871 and a mean compression weight of 0.5360.536.

We therefore conclude that ** Elastic Coding is not designed to detect corrupted labels or negative clean-dominance gaps. Instead, it acts as an optimisation-conflict controller that effectively mitigates the robustness failures of MCR².**

In other words, while our spectral analysis motivates examining class compression, the controller measures compression opposing global expansion and does not exactly guarantee the detection or correction of negative spectral gaps. We have clarified this distinction in Sections 2.3 and 3.3, and Appendix G.

Weakness 2: The benefit of adaptation is unclear because the final compression weights are close to 0.50.5. A fixed-weight baseline is needed to determine whether the gains come from adaptation or simply weaker compression.

This concern overlaps with Reviewer eGrH's Weakness 1, and we kindly refer you to our response there for the detailed comparison. We summarise the main experimental results and findings below.

We have added a fixed-λ=0.5\lambda=0.5 baseline across all nine CIFAR-10 span-2 noise rates, using the same architecture, model and corruption seeds, initial learning rate, and scheduling and stopping rules. Revised Table 1 shows that Elastic Coding achieves higher observed-label accuracy at all six noise rates from 10% to 60%, and higher clean-label probe accuracy at all nine rates. At 50% noise, Elastic Coding achieves 82.78% versus 58.96% observed-label accuracy and 83.49% versus 62.91% clean-label probe accuracy. At higher noise rates (70%–90%), fixed λ=0.5\lambda=0.5 performs better in observed-label accuracy, although Elastic Coding maintains higher clean-label probe accuracy.

We also provide the complete compression-weight trajectories in Appendix F, Figure 7 and Table 8. Elastic Coding starts near λ=1\lambda=1 and finishes between 0.4830.483 and 0.5420.542, showing that the final weight does not reflect the compression applied throughout training. We hence summarise that these results demonstrate that simply fixing λ=0.5\lambda=0.5 does not reproduce Elastic Coding's performance under the tested protocol.

AuthorsOpenReview ↗

Response to Reviewer btWb, Part 2

Weakness 3: Key reproducibility settings are missing, including the conflict threshold, smoothing coefficient, compression floor, numerical stabiliser, and coding precision. Their selection procedure is also unclear.

Thank you for pointing out these omissions. This concern overlaps with Reviewer eGrH's Weakness 3 and Q3, and we kindly refer you to our responses there for further details. We have now explicitly documented all settings in Section 2.3 and Appendix E.1:

  • Conflict threshold: τ=1\tau=1.
  • EMA smoothing coefficient: β=0.9\beta=0.9.
  • Compression floor: λmin⁡=0.05\lambda_{\min}=0.05.
  • Numerical stabiliser: δ=10−12\delta=10^{-12}.
  • Coding precision: ϵ2=d/(3K)\epsilon^2=d/(3\sqrt K), with d=512d=512 and KK classes.

These settings are identical across all datasets and noise rates. We also clarify that the EMA is initialised using the first measured conflict ratio, while the compression weight is clipped to [0.05,1][0.05,1] and held fixed during each encoder update.

Regarding hyperparameter selection, we generally adopted the default settings recommended in the original papers for baseline methods, with the exact variants and coefficients now documented in Appendix E.3. For Elastic Coding, we used the same controller settings across all datasets and noise rates, without dataset-specific or noise-rate-specific tuning. All hyperparameters were kept fixed throughout the additional multi-seed experiments.

Reviewer_eGrHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the robustness of Maximum Coding Rate Reduction (MCR²) under label noise. The authors explain the sources of its robustness from three aspects: global expansion, sample normalization, and class subspace competition. Under an idealized orthogonal subspace model, they use the clean-dominance gap to characterize the impact of noise structure on class-dominant directions. On this basis, the authors propose Elastic Coding, which adaptively reduces the intra-class compression weight according to the tangential gradient conflict between the expansion and compression terms. Experiments show that the method outperforms the original MCR² under multiple settings and surpasses the compared classification loss baselines on some synthetic and real-world noisy datasets.

Critical strengths
  1. The research question is clear, and the analysis is somewhat instructive. The paper emphasizes that the impact of label noise depends not only on the noise rate but also on how mislabeled samples are distributed across different classes. This perspective helps explain why MCR² may exhibit varying degrees of robustness under the same noise rate.

  2. Compared with the original MCR², the method achieves practically meaningful improvements. Results under different readout methods also provide supplementary evidence for improved representation quality. In addition, the authors distinguish relatively clearly between representation diagnostics and actual classification performance.

Critical weaknesses
  1. The independent contribution of the "adaptive mechanism" proposed in the paper compared with "simple weight adjustment" has not yet been demonstrated. The new designs of Elastic Coding mainly include tangential gradient decomposition, conflict statistics, EMA smoothing, and compression weight adjustment. However, the final weights corresponding to each noise rate in Table 1 are all close to 0.5, yet the paper does not provide fixed compression weight experiments, especially a control with fixed λ=0.5\lambda=0.5. Therefore, the existing experiments cannot distinguish whether the performance improvement comes from the proposed adaptive conflict control or merely from reducing the compression strength of the original MCR².
  2. There is a lack of direct mechanistic evidence between some of the paper's theoretical analysis and the actual algorithm. Propositions 2–3 analyze fixed, orthogonal, and isotropic class subspaces, whereas the algorithm uses the conflict between expansion and compression gradients during training as the adjustment signal. The paper does not establish a clear relationship between this conflict indicator and the deterioration of the clean-dominance gap.
  3. The experimental comparisons and disclosure of their protocols in the paper are still insufficient to establish a stable advantage over contemporaneous methods. Although the paper has included recent baselines such as JAL and VBL, it does not fully specify the specific loss variants, key coefficients, and hyperparameter selection protocol. The main experiments use only one random seed, and multi-random-seed experiments need to be further supplemented. In addition, the code provided by the authors shows that the true-label centroid probe in Table 1 uses randomly augmented training features, while the observed-label centroid uses deterministically transformed training features; the main text does not disclose this difference, so the gap between the two readouts cannot be fully attributed to the label grouping method.
Questions for authors

Q1. Under the same training and hyperparameter selection protocol, is Elastic Coding consistently superior to fixed (λ=0.5), other reasonable fixed weights, and a simple preset weight schedule? Please provide experiments on the trajectory of the weight during training.

Q2. What is the relationship between the conflict indicator during training and the clean-dominance gap? Can you, in experiments that match the noise rate and vary the noise structure, track both simultaneously and add a clean-data control to demonstrate that this signal mainly responds to harmful competition related to label noise?

Q3. Which specific JAL/VBL variants and hyperparameters were used in Table 2? Were the same Elastic Coding weight settings used across different datasets? In addition, please supplement independent repeated runs (multi-random-seed experiments) of the key results, and use the same training feature cache to re-compare the two centroid readouts in Table 1, so as to rule out possible effects caused by differences in the feature extraction pipeline.

3 comments
AuthorsOpenReview ↗

Response to Reviewer eGrH, Part 1

(We apologise for the relatively late response, as the additional experiments required to address the reviewers’ concerns, particularly the multiple-seed evaluations for Table 2, were computationally expensive and time-consuming. We have only just completed these experiments.)

We sincerely thank you for carefully evaluating our work, giving very constructive comments and recognising its clear research question and informative analysis of label-noise structure. We also appreciate your acknowledgement of the practical improvements over MCR² and our clear distinction between representation quality and classification performance. Below, we address your concerns point by point. For clarity and readability, we have slightly rephrased your comments while preserving their original meaning.

Weakness 1 & Question 1: The benefit of adaptation over simply reducing the compression weight is unclear, particularly because the final weights are close to 0.50.5. A fixed-weight control and training trajectories are needed.

We have added a fixed-λ=0.5\lambda=0.5 baseline across all nine CIFAR-10 span-2 noise rates, together with the complete compression-weight trajectories. The results are reported in Section 3.2, Table 1, and Appendix F.

For a fair comparison, all methods use the same architecture, model and corruption seeds, data augmentation, initial learning rate, and learning-rate scheduling and stopping rules. All runs complete 100 epochs. Both centroid-based evaluations use the same deterministic training-feature cache at each checkpoint. Each configuration is evaluated in a single run.

Elastic Coding outperforms fixed λ=0.5\lambda=0.5 in observed-label centroid accuracy at all six noise rates from 10% to 60%, and in clean-label centroid probe accuracy at all nine noise rates. For example, at 50% label noise, we obtain the following results:

MethodObserved-label accuracyClean-label probe
MCR²70.34%70.75%
Fixed λ=0.5\lambda=0.558.96%62.91%
Elastic Coding82.78%83.49%

Note that the clean-label probe is a representation diagnostic that requires access to clean training labels. At higher noise rates (70%–90%), fixed λ=0.5\lambda=0.5 achieves higher observed-label accuracy than Elastic Coding, although Elastic Coding consistently achieves better clean-label probe accuracy. We explicitly distinguish these two evaluation settings in the revised manuscript.

Furthermore, the training trajectories show that Elastic Coding starts with a compression weight near λ=1\lambda=1 and gradually reaches final values between 0.4830.483 and 0.5420.542 (Appendix F, Figure 7 and Table 8). This demonstrates that a final weight close to 0.50.5 does not reflect the compression strength applied throughout training.

We therefore conclude that these results demonstrate that simply replacing Elastic Coding with a fixed compression weight of 0.50.5 does not reproduce its performance under the tested protocol.

AuthorsOpenReview ↗

Response to Reviewer eGrH, Part 2

Weakness 2 & Question 2: The relationship between the fixed-feature spectral analysis and the gradient-conflict signal used by Elastic Coding is insufficiently established.

We have revised Section 2.3 and added further theoretical and empirical investigations in Section 3.3 and Appendix G.

First, we identify a counterexample showing that loss of clean dominance does not necessarily trigger gradient conflict. Under exact orthogonal isotropy, both coding gradients are radial, so their tangent projections vanish even when the clean-dominance gap is negative. We further examine 36 fixed-feature configurations covering different representation geometries, confirming that the relationship is geometry-dependent rather than universal.

Second, following your suggestion, we conducted six additional CIFAR-10 experiments, comparing MCR² and Elastic Coding under clean labels, 50% dispersed noise, and 50% concentrated noise. We track gradient conflict, compression weights, and covariance-contribution gap proxies throughout training (seed 42). Results are reported in Appendix G, Figures 9–10 and Tables 14–15.

These experiments reveal both the usefulness and limitations of our interpretation. At 50% noise, concentrated corruption produces a near-zero rank-1 gap proxy, whereas dispersed corruption maintains a positive one. Interestingly, gradient conflict also occurs under clean labels, with a final-epoch mean conflict of 1.8711.871 and compression weight of 0.5360.536 for Elastic Coding. Moreover, the relationship between conflict and the gap proxy varies across methods, noise structures, and proxy ranks.

We therefore conclude that Elastic Coding is not designed to detect corrupted labels or negative clean-dominance gaps. Instead, it acts as an optimisation-conflict controller that effectively mitigates the robustness failures of MCR².

AuthorsOpenReview ↗

Response to Reviewer eGrH, Part 3

Weakness 3 & Question 3: The paper should specify baseline variants, coefficients, and hyperparameter selection, report multi-seed results, and eliminate feature-extraction differences between the two centroid readouts.

We have expanded the experimental protocols in Appendix E, updated Table 2 with five-seed results, and re-evaluated both centroid readouts using consistent feature extraction. Specifically, we have done:

1. Baseline variants and hyperparameters.

Appendix E.3 now specifies the exact loss formulations, coefficients, numerical stabilisers, and reference implementation versions. Elastic Coding uses identical controller settings across all datasets and noise rates: τ=1\tau=1, β=0.9\beta=0.9, λmin⁡=0.05\lambda_{\min}=0.05, δ=10−12\delta=10^{-12}, and ϵ2=d/(3K)\epsilon^2=d/(3\sqrt K) with d=512d=512. All settings were fixed across the additional seeds, and clean labels were not used for training.

2. Five-seed evaluation.

We have updated Table 2 with five-seed results (42–46) for all ten methods across six settings, comprising 300 results. All runs complete 100 epochs. Synthetic experiments vary model and corruption seeds, while real-noise experiments vary training randomness with fixed observed labels and data subsets.

Elastic Coding outperforms MCR² in all six settings and achieves the highest mean accuracy among the compared methods in four of six settings. The results below report mean ± sample standard deviation (%).

SettingMCR²Elastic Coding
CIFAR-20, 10%69.57 ± 0.3974.39 ± 0.78
CIFAR-20, 20%68.24 ± 0.6871.03 ± 0.74
Imagenette, 10%76.66 ± 0.7779.26 ± 0.35
Imagenette, 20%75.08 ± 0.3076.56 ± 0.81
ANIMAL-10N78.18 ± 0.3781.69 ± 0.36
Clothing1M-100K63.55 ± 0.8665.33 ± 0.76

CE retains the highest mean accuracy in both Imagenette settings. We therefore report descriptive comparisons without claiming uniform superiority.

3. Consistent feature extraction.

We have recomputed Table 1 using the same deterministic training-feature cache and test features for both centroid readouts. Only the labels used to construct centroids differ. The clean-label probe is used solely to assess representation quality, not for training.

4. Verification of weak baselines.

With the assistance of GPT-6 Astra, we verified baseline loss values and gradients against pinned reference implementations across 24 configurations, covering eight losses and three class counts, with no discrepancies found at the tested numerical precision. However, some baselines remain very weak. For example, AEL achieves 12.22 ± 1.68% on CIFAR-20 at 20% noise, compared with 65.52 ± 0.39% for CE.

Since we used the default hyperparameters recommended in the original papers, it is reasonable to expect that more extensive hyperparameter tuning could improve the performance of these baselines. However, exhaustive hyperparameter tuning would be computationally prohibitive, particularly given the extensive five-seed experiments we have conducted. We therefore conjecture that some baseline methods may not generalise well to the experimental settings considered in our paper without additional hyperparameter tuning.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Responses and Revisions

We sincerely thank all reviewers for their detailed and constructive feedback, which has helped us clarify our contributions and strengthen the paper. We appreciate their positive comments on:

  • Clear research question and informative analysis
  • Useful distinction between dispersed and concentrated label noise
  • Simplicity of Elastic Coding and improvements over MCR²
  • Careful separation of representation quality from classification performance
  • Explicit distinction between fixed-feature theory and training guarantees

Below, we summarise our main clarifications and revisions. The added analyses and experiments are highlighted in light red in the revised paper.

  • Concern 1: The benefit of adaptation over simply reducing compression is unclear. (Reviewers eGrH, btWb, and 7J7Y)
    We added a fixed-λ=0.5\lambda=0.5 baseline across nine noise rates and complete weight trajectories. Elastic improves observed-label accuracy at six rates and clean-label probe accuracy at all nine (Section 3.2; Appendix F).

  • Concern 2: The spectral analysis does not establish the gradient-conflict signal. (Reviewers eGrH, btWb, and 7J7Y)
    We added 36 fixed-feature configurations and six training controls, including clean-data controls, and explicitly acknowledge the orthogonal-isotropic counterexample (Sections 2.3 and 3.3; Appendix G).

  • Concern 3: Key settings and implementation details are missing. (Reviewers eGrH, btWb, and 7J7Y)
    We now document controller settings, coding precision, baseline variants, coefficients, and training protocols. Elastic’s controller settings are shared across datasets and noise rates (Section 2.3; Appendix E).

  • Concern 4: Multi-seed evidence and stronger baseline verification are needed. (Reviewers eGrH and 7J7Y)
    We expanded every method–setting combination in Table 2 to five seeds, comprising 300 results. We also added reference implementation checks, while acknowledging that these do not establish optimal baseline tuning (Table 2; Appendix E).

  • Concern 5: Some baselines perform unusually poorly, raising concerns about implementation correctness. (Reviewers eGrH and 7J7Y)
    We checked baseline loss values and gradients against reference implementations and found no discrepancies on the tested inputs. We have also included the baseline implementation code in the supplementary material for inspection and reproducibility (Appendix E.3).

  • Concern 6: Feature-extraction differences may affect the centroid comparisons. (Reviewer eGrH)
    We recomputed both centroid readouts using the same deterministic feature cache per checkpoint, eliminating the previous extraction discrepancy while retaining the distinction between classification performance and representation diagnostics (Section 3.1; Table 1).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.