acceptodds
Under review as a conference paper at ICLR 2027

Weak Learning Failure Alone is not Gradient Starvation

Abstract

Gradient starvation describes competition in which learning one predictive feature suppresses learning another. Yet a weak response alone cannot establish suppression: the same feature may remain poorly learned without the competing cue. We develop a controlled approach using matched training runs with an informative cue, an uninformative cue, and no cue. Our criterion requires both reference runs to reach a learning target within a fixed budget and measures the informative-cue deficit at each reference's first target crossing and across training. Across four model families, comparisons with uninformative cues show why negative differences alone are insufficient evidence. We derive an exact gradient-flow decomposition that tracks how differences in output loss gradients and response sensitivities, together with weight decay, change weak-feature learning. In a noiseless two-layer linear model under unregularized logistic gradient flow, we prove that a perfectly reliable cue can sustain a relative learning deficit indefinitely, even as both runs continue learning. We also rigorously verify finite-time suppression bounds against cue-free training in small networks through validated numerical integration. In CIFAR-10, removing or randomizing the cue during training improves cue-free accuracy by about 7.6 percentage points relative to retaining it, measured using a fresh linear classifier fitted to frozen features. On pretrained Waterbirds models, poor worst-group accuracy coexists with small measured core-feature deficits. Together, these results distinguish cue-induced suppression from weak learning shared across training conditions and show why poor robustness need not imply a substantial feature-learning deficit. Code is available at https://anonymous.4open.science/r/gradient_starvation-C646/.

Primary area:
optimization
Keywords:
Gradient Starvation, Causal Diagnosis, Matched Counterfactual, Feature Learning, Shortcut Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_j9KJOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The manuscript studies classification problems where the input data contain "weak features" with strong predictive power and "cues" that are easy to learn and significantly reduce the training loss but do not necessarily have strong predictive power (e.g., background in birds classification dataset). The main finding of this work is that the poor learning of the weak features is not necessarily due to gradient starvation (the poor learning of the weak features is due to competition with the cue) but could result from the intrinsic difficulty of learning the weak features. To show that this phenomenon occurs, the authors propose a metric M(θ)M(\theta) measuring how the model learns the weak feature. Then, they compare this metric (or the accumulated version) between different learning scenarios (informative/uninformative cue and no cue). When MBM_B (informative cue run) and MWM_W (cue free run) are both small, they conclude that the weak features are hard to learn. Conversely, if MWM_W and MNM_N (uninformative cue run) are both relatively large while MBM_B remains small, they consider that as evidence of gradient starvation. Finally, they illustrate their tool on several datasets.

Critical strengths

The interaction between easy-to-learn cues and weaker predictive features, and its consequences for generalization, is an interesting problem.

Critical weaknesses
  1. The proposed metric criterion M(θ)M(\theta) is used to determine whether we are in a gradient starvation situation or if the weak features are intrinsically hard to learn, is arbitrary, could change depending on the dataset, and requires identifying the weak features (or generating them, as in the fashion MNIST experience). But in practice, the weak features and the cues are not known. This considerably limits the applicability of this diagnostic tool.

  2. The proposed solution essentially amounts to a matched controlled experiment in which the cue is varied, while the other parameters are held fixed. I don't see particular conceptual novelty here.

  3. The advertised mathematical contribution seems also very limited, none of the mentionned theorem is stated in the main text.

Questions for authors

The proposed method distinguishes gradient starvation from hard weak-feature learnability only a posteriori. Are there identifiable structural properties of the data/model that predict which regime occurs?

2 comments
AuthorsOpenReview ↗

Response to Reviewer j9KJ

We thank the reviewer for the thoughtful assessment. We have uploaded the revised version. B, N and W denote informative-cue, label-independent-cue and cue-free runs.

Addressing Weakness 1: choices and applicability

The response, target, budget and thresholds are fixed before confirmation (App. D). Each family's α* is the smallest grid value at which at most 5% of calibration controls meet the minimum-deficit condition. This prevents tuning to confirmation outcomes, but does not make the choices unique or guarantee a 5% rate on new runs. Three families miss that target. We describe a controlled operational test without established diagnostic error rates.

Threshold stability is limited. Post hoc increases from α=0.25 to 0.40 in F-nl and 0.05 to 0.15 in F-e2r preserve 318/319 and 96/98 informative-run decisions; CIFAR retains 8/8 through α=0.50. F-lin falls from 192/288 to 1/288 at α=1.00 versus 0.75. These conditional frequencies use W-target-reaching pools, additionally restricted to ρ≤4, lag≤4 for F-lin. They do not estimate sensitivity or independently validate new thresholds (Tables 4–5).

New datasets require justified measurements and controls. The method tests a specified cue hypothesis under an available intervention; it does not discover unknown cues (Sections 1 and 6). In CIFAR the weak feature is natural image content; in Waterbirds both bird and background are natural. With known group labels, a reweighted reference agrees with the pixel intervention in 8/8 Waterbirds pairwise decisions, all negative. This does not establish general equivalence.

Responding to Weakness 2: novelty

Cue-present/cue-free comparisons are established, including the shortcut-versus-XOR analysis of LaBonte and Muthukumar [1]. Our contribution combines reference progress, a relative deficit at first target crossing and a signed accumulated deficit against both cue-free and label-independent references, with analysis of these comparisons.

N separates cue-label information from the optimization effect of cue exposure. Under Corollary 1's assumptions, sign-symmetric initialization and a nonzero initial rate give a negative initial gap rate in half the initializations even for an uninformative cue. N excludes 3/617 positive B-versus-W cases in the reported eligible pool, but 29/32 at ρ=16, lag=4, outside F-lin's subset. That cell has 32/32 positive B-versus-W and N-versus-W comparisons but only 3/32 B-versus-N comparisons.

Reference progress prevents attributing failure to the cue when W also fails. At injected amplitude 0.001, W misses its target in all 64 records at tested budgets through H=5,000. The comparison is inconclusive, not evidence of cue-induced suppression or intrinsic unlearnability.

On Weakness 3: theory

Section 4 (pp. 4–6) now states the principal results and assumptions; App. B contains the proofs. Theorem 2(a) considers a noiseless, zero-lag two-layer linear model under unweighted, unregularized logistic gradient flow, weak-feature strength one, a perfectly reliable cue of strength ρ, and nonzero growing modes in both runs. It proves

MwB(t)/MwW(t)→1/(1+ρ2)M_w^B(t)/M_w^W(t)\to 1/(1+\rho^2) as t→∞t\to\infty.

Both responses grow while their relative deficit persists. Other results give an exact response-drift decomposition, an initial-rate formula and sign symmetry, and an output-cue analysis. Validated integration certifies finite-time B-versus-W bounds and W's target attainment for 41–51% of 1,000 sampled initializations per configuration across nine small-network configurations. These results do not establish the full B/N/W criterion.

Answering the Question: structural predictors

Cue strength and reliability yield model-specific predictions: Theorem 2(a) gives the limiting relative deficit ρ²/(1+ρ²). In the same model, with an imperfect cue, agreeing and disagreeing examples, and loss below log 2 at some time, Theorem 2(b) proves directional convergence to the weak coordinate; convergence of the B/W response ratio to one remains a conjecture. Weak-signal strength and budget matter through reference progress. Under Proposition 3's assumptions, an output cue can slow progress along the same weak-learning path.

We do not provide a general predictor of the full criterion from initialization. In C1's linear RNN, −d₀ gives AUROC 0.617 [0.604, 0.630] for the B-versus-W condition at H=5, α=0.10. The separate A2-P2 test for any negative N-versus-W gap fails its endpoint: AUROC 0.306 [0.157, 0.471], with frozen score orientation. These are 95% intervals. The operational test uses observed trajectories (App. E.4; Table 13).

We thank the reviewer once again for their time and thoughtful suggestions, which have helped us improve the clarity and precision of the paper.

Reference

[1] LaBonte and Muthukumar. SGD Provably Prioritizes a Shortcut Spurious Feature in the XOR Model. NeurIPS 2026. https://arxiv.org/abs/2606.30444

I thank the authors for their rebuttal and for improving the presentation of the theoretical results. However, their response regarding novelty essentially confirms my original concern: the proposed approach consists of controlled experiments combined with heuristic criteria for measuring "learning deficits". While this may be useful in some situations, I remain unconvinced that this constitutes a substantial conceptual contribution, and therefore maintain my negative assessment.

Reviewer_w5zSOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates a fundamental question in feature learning: when a neural network learns a predictive cue but responds weakly to another useful feature, how can we establish that the cue has actually suppressed learning of the weaker feature, rather than merely observing that the weak feature is difficult to learn under the given training procedure? The authors propose a matched three-run comparison: an informative-cue run (B), an uninformative-cue run (N), and a cue-free run (W). All runs share their initialization and, where applicable, data order and other training choices. The weak-feature response is measured on inputs with the competing cue removed. Their suppression criterion requires both reference runs to reach a specified response target within a fixed training budget, a relative deficit at the reference’s first target crossing, and a sufficiently large normalized signed integrated deficit. The informative-cue run must satisfy the criterion against both references. The paper also calibrates the integrated-deficit threshold using uninformative-cue controls and explicitly distinguishes inconclusive results from evidence of suppression.

The theoretical contributions include an exact gradient-flow decomposition of changes in the weak-feature response into differences in output-loss gradients, differences in response sensitivities, and weight-decay effects. The authors derive an initial-rate result showing that an uninformative cue can alter early learning dynamics, prove a persistent relative weak-feature deficit for a perfectly reliable cue in a noiseless two-layer linear model under unregularized logistic gradient flow, and verify finite-time suppression bounds in small networks using validated numerical integration. Empirical studies cover linear and nonlinear recurrent networks, injected-pattern experiments on MNIST and Fashion-MNIST, a synthetic ribbon cue on CIFAR-10, and pretrained Waterbirds models. On CIFAR-10, removing or randomizing the cue partway through training improves fresh-linear-head cue-free accuracy by approximately 7.6 percentage points relative to continued training with the informative cue. On Waterbirds, substantial differences in worst-group accuracy coexist with relatively small measured core-feature response differences. The overall message is that causal suppression, weak learning, and reliance on a learned feature for prediction are related but distinct phenomena.

Critical strengths
  1. A meaningful and carefully framed problem formulation. The three-run comparison is the most compelling contribution. Comparing an informative cue against both a cue-free reference and an uninformative-cue reference addresses two different confounds: insufficient learning under the available training budget and optimization changes caused merely by introducing an additional cue. Requiring sufficient reference progress and a substantial deficit at the target crossing and across training is a sensible improvement over declaring starvation whenever a response difference becomes negative. Importantly, the authors do not present their operational criterion as a universal definition.

  2. Substantial theoretical work and unusually transparent validation. The exact response-drift decomposition is mathematically useful, and the long-time result for the noiseless linear model offers a concrete explanation of how both runs can continue learning while a relative deficit persists. Validated numerical integration is a valuable addition to ordinary sampled training curves. The preregistration details, explicit reporting of failed endpoints, and acknowledgement of dependent observations and limited sample sizes are also significant strengths. These choices make the work easier to audit and reduce the risk that its conclusions are interpreted more broadly than its evidence warrants.

Critical weaknesses
  1. The empirical operating characteristics of the proposed suppression criterion remain insufficiently established.

The threshold calibration is central to the proposed method, yet the confirmation results do not consistently satisfy the intended calibration target. For the uninformative-cue-versus-cue-free minimum-deficit condition, the reported frequencies are 5.3% (34/640) for F-lin, 6.9% (22/320) for F-nl, and 5.3% (17/320) for F-e2r, all above the intended 5% target. For CIFAR-10, zero of eight controls meet the condition, but the reported nominal upper confidence bound is 36.9%, leaving the control rate very uncertain. The paper correctly acknowledges that record-level intervals do not account for dependence among reused seeds and conditions; nevertheless, that dependence makes it difficult to quantify how reliably the criterion generalizes to new configurations.

Moreover, the paper explicitly states that no ground-truth set of starved runs is available. Consequently, the reported frequency with which the full criterion is met among informative-cue runs is a conditional frequency, not a sensitivity estimate or an independent validation of diagnostic accuracy. This is not an argument that every proposed operational definition must function as a binary classifier. Rather, it means that the practical reliability of this particular thresholded criterion remains less established than the central methodological emphasis might suggest.

Actionable improvement: Provide a calibration and confirmation analysis that respects the seed/configuration hierarchy, and clarify how the criterion behaves in settings where suppression is analytically known to be present or absent. Alternatively, narrow the practical claims so that the criterion is explicitly presented as a controlled operational test rather than an empirically calibrated diagnostic with established error rates.

  1. The conclusions about feature learning depend materially on the response measurement, limiting some empirical interpretations.

The authors report that four CIFAR-10 response measures agree on the relevant assessment in only 5 of 8 seeds (Appendix E.2; the corresponding preregistered measurement endpoint is not met). This is important because the paper's central conclusions depend on a chosen scalar response, a target, and a training budget. The authors appropriately acknowledge that the response need not capture every useful feature, but the observed measurement sensitivity makes the interpretation of some image experiments less decisive.

There is also a distinction between measuring cue-free predictive performance using the original prediction head and measuring information accessible to a newly trained linear head. The primary suppression criterion uses class-balanced Brier skill with the original head on cue-removed images, whereas fresh-head accuracy is reported separately. The former is a meaningful functional response, but it does not isolate representation quality from the readout used to turn the representation into predictions. The fresh-head experiments partly address this distinction, but they do not directly establish that the primary criterion is robust to this confound.

This matters particularly when interpreting Waterbirds: the large worst-group accuracy difference alongside small measured core-feature deficits is a useful observation, but it is based on one pretrained ResNet-50 setting with eight seeds. The result supports the paper's qualified claim that poor robustness need not imply a substantial deficit under the chosen measure; it does not establish that conclusion for other architectures, response measures, or training regimes.

Actionable improvement: Report per-seed suppression decisions across the alternative response measures, and distinguish consistently between weak cue-free predictive utility, information accessible to a linear probe, and representation-level feature learning. Broader claims about feature learning should be restricted to what the selected response can identify.

Questions for authors
  1. Calibration and interpretation: How should readers interpret the uninformative-cue control rates that exceed the prespecified 5% target in three families? Can you provide uncertainty estimates that account for the reuse of model seeds and experimental conditions, or demonstrate whether a more conservative frozen threshold changes the principal conclusions? This would materially affect my confidence in the criterion's practical reliability.
  2. Response-measure sensitivity: Given that four CIFAR-10 response measures agree in only 5/8 seeds, can you report the per-seed outcomes under each measure and show whether the primary suppression conclusions persist when the criterion is evaluated with an alternative, representation-oriented probe? If the decisions change, which interpretation of the measured deficit should be considered the primary one?
  3. Waterbirds and the feature-learning claim: In the pretrained Waterbirds experiment, can you clarify how the primary Brier-skill response relates to the fresh-linear-head accuracy results and to worst-group accuracy? In particular, does the conclusion about a small core-feature deficit remain stable under alternative cue-free probes, or is it specific to the chosen response and pretrained evaluation setting?
1 comment
AuthorsOpenReview ↗

Response to Reviewer w5zS

We thank the reviewer for the constructive assessment. We have reanalysed the existing records and uploaded the revised version. B, N and W denote informative-cue, label-independent-cue and cue-free runs.

On calibration and practical reliability (Weakness 1; Question 1)

The 5% confirmation target was missed in three families. We now report intervals that preserve seed reuse: 20,000 bootstrap draws over complete model-seed blocks for F-lin/F-nl and crossed data/model-seed factors for F-e2r. CIFAR uses an exact binomial interval.

FamilyControl countRate (95% interval)
F-lin34/6405.31% [5.00%, 5.78%]
F-nl22/3206.88% [4.06%, 9.69%]
F-e2r17/3205.31% [2.81%, 7.50%]
CIFAR-100/80% [0%, 36.94%]

These intervals condition on the tested configurations and frozen thresholds; they do not capture threshold-selection uncertainty or reliability on new configurations. F-e2r has only four data seeds, so its interval is exploratory. The denominators include runs where W misses its target. Among target-reaching runs, the counts are 34/480, 22/319, 17/98 and 0/8.

Configuration effects matter: 32/34 F-lin positives come from ρ=16, lag=4, outside its prespecified informative-run reporting subset (ρ≤4, lag≤4), which has 0/288 positive controls. Within the other families' reporting pools, two GRU cells account for 21/22 F-nl positives and MNIST at amplitude 0.02 for 15/17 F-e2r positives. These concentrations do not resolve the failed endpoint.

The post hoc threshold sweep keeps other conditions fixed:

Familyα changeControlsInformative-run full criterion
F-lin0.75 → 1.0034/640 → 0/640192/288 → 1/288
F-nl0.25 → 0.4022/320 → 5/320318/319 → 318/319
F-e2r0.05 → 0.1517/320 → 7/32096/98 → 96/98
CIFAR-100.15 → 0.500/8 → 0/88/8 → 8/8

Informative-run pools require W to reach its target; F-lin also uses the restriction above. These are conditional frequencies, not sensitivity. At α=0.75, the last three full-criterion counts fall to 284/319, 64/98 and 1/8. Thus stability is limited to the stated ranges. Reusing confirmation data does not validate new thresholds (Tables 4–5, p. 25).

The analytic guarantees concern B versus W, not the full rule. Reversed feature roles give 0/96 positives (0/72 target-reaching); a reference that cannot learn gives an inconclusive comparison. Neither supplies a ground-truth validation set. We now explicitly present a controlled operational test without established diagnostic error rates.

On response sensitivity (Weakness 2; Question 2)

The full CIFAR criterion holds in 8/8 seeds under each Brier response: original-head neutral input, fresh linear-head neutral input, and fresh-head randomized cue. Fresh-head neutral accuracy gives 5/8. Per-seed outcomes are:

SeedsEach of the three Brier measuresFresh-head accuracy
43001–43005MetMet
43006–43008MetNot met / inconclusive

All use H=4,710 and α=0.15, selected on the primary response and reused. Brier uses β=0.5; accuracy is normalized as (balanced accuracy−0.1)/0.9 with β=0.7. Thus both response and target change. The archived decisions do not identify whether the three accuracy failures concern progress, crossing deficit or accumulated deficit (Table 7, p. 27).

The primary interpretation remains cue-free utility under the original head. Fresh-head Brier supports the conclusion under a separate readout; the failed four-measure agreement endpoint prevents a probe-independent representation claim.

On Waterbirds (Question 3)

WB, WR and WG correspond to B, N and W. Original-head Brier measures gray-background predictive utility; fresh-head gray accuracy measures information accessible to a linear head over frozen representations; worst-group accuracy measures the original predictor on standard groups.

For WB versus WR (B versus N), normalized signed Brier deficits average 0.0010, ranging from −0.0281 to 0.0238. All fall below α=0.05, so the accumulated-deficit condition fails in every seed; the full criterion is met in 0/8 while W reaches its target in 8/8. A small signed integral does not imply a uniformly small gap.

Fresh-head accuracy differences are −0.51 pp for WB−WR (95% paired interval [−0.75, −0.27]) and −1.28 pp for WB−WG [−1.39, −1.18]. The worst-group WB−WR difference is −16.78 pp [−18.22, −15.34]. These are distinct outcomes in one pretrained setting (Section 5.4; Tables 8–9, p. 28).

The original-head areas against WG remain unavailable, and the full fresh-head time-course criterion is unestablished. We therefore do not claim probe-invariant absence of suppression or unaffected feature acquisition from scratch.

We thank the reviewer once again for their time and thoughtful suggestions, which have helped us improve the clarity and precision of the paper.

Reviewer_HCBqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

The paper studies the problem of gradient starvation, i.e., the fact that one feature can suppress the learning of another one, if it is predictive for the same class. To disentangle the suppression of features due to the presence of the aforementioned cues from the general difficulty to learn strong responses to such features, the authors develop a controlled experimental setup which includes one run with informative cues, one run with uninformative cues and one run without any cues. In addition, the authors analyse weak feature learning in a time-continuous gradient flow formulation.

Critical strengths

The paper analyzes an important problem in two complementary ways: 1) Based on theoretical analysis in an idealized, time-continuous setting. 2) Based on numerical experiments trying to separate the effect of (possibly response suppressing) cues from the difficulty to learn certain features. Thereby, this work establishes a methodology to analyze the effects of cues that can lead to gradient starvation. It can therefore provide valuable insights for the community.

Critical weaknesses

For me as a non-expert, the paper was difficult to read. It neither provides very intuitive explanations or simple examples (e.g. visually on CIFAR) for cues and weak features, nor makes the involved quantities mathematically precise in the main paper. In particular, all theoretical results are mere intuitively explained and not stated formally. The supplementary material is extensive and contains all theorems and propositions, all proofs, and many details about the numerical experiments. Yet, the main paper is not self-contained and difficult to understand without the appendix. Considering that the work does not fully use the 9 pages, a bigger picture and more detailed explanations would have been desirable.

Minor points: Beta, H and M are not defined by the time one first looks at Fig. 1. The cue-likelihood reference in Fig. 2 is almost invisible.

Questions for authors

Besides the recommendation to consider improving the clarity, I am wondering about the cited references. There is no work from 2026, no work from 2025 and only one work from 2024. Is the proposed analysis still timely and relevant? If so, why has noone worked on the problem for the past 2.5 years?

1 comment
AuthorsOpenReview ↗

Response to Reviewer HCBq

We thank the reviewer for the thoughtful comments and for recognizing the value of combining theory with controlled experiments. We agree that the main text needed a clearer explanation and formal statements. We have uploaded the revised version.

Addressing the Weakness

  1. Figure 1 now illustrates the CIFAR-10 example: the weak feature is the image content, and the competing cue is an appended ribbon. B uses a class-informative ribbon, N a label-independent ribbon, and W a blank ribbon. Poor image-only performance in B does not establish suppression: W asks whether the same procedure can learn without the cue, while N controls for the optimization effect of cue exposure without label information.

  2. M is the chosen weak-feature response with the cue removed; β is a prespecified target above the initial reference response, and H is the training budget. For CIFAR-10, M is class-balanced Brier skill under the original head, with 0 for uniform predictions and 1 for perfect predictions. Both references must reach β within H, and B must have sufficient deficits at each reference's first crossing and over training. A reference that never reaches β makes the comparison inconclusive. These definitions now precede Figure 1 and appear in its caption (p. 2); Section 3 gives the full criterion.

  3. Section 4 (pp. 4–6) now states the principal formal results and assumptions, with proofs in App. B. For example, Theorem 2(a) considers a noiseless, zero-lag two-layer linear model under unweighted, unregularized logistic gradient flow, with weak-feature strength one, a perfectly reliable cue of strength ρ, and nonzero growing modes in both runs. It proves

MwB(t)/MwW(t)→1/(1+ρ2)M_w^B(t)/M_w^W(t)\to 1/(1+\rho^2) as t→∞t\to\infty.

Both weak-coordinate responses grow, but their relative deficit persists. This is a B-versus-W result under those assumptions, not a theorem about the full three-run criterion. Section 4 also states the response-drift decomposition, initial-rate result, output-cue result and validated finite-time bounds.

  1. The distinction matters for intervention. If W also fails to learn within the budget, cue removal need not solve the problem. In CIFAR-10, removing the cue at 25% of training improves fresh-head cue-free accuracy by 7.6 percentage points after 471 further updates, relative to retaining it (95% interval [6.6, 8.7]). Waterbirds illustrates a separate distinction between original-head performance and information accessible to a fresh head; it does not establish unaffected feature learning. The revised main text uses the full nine pages to explain these distinctions before the detailed analyses.

On the Minor Points

M, β and H are now defined before Figure 1. Figure 2(c)'s cue-likelihood reference now has a larger purple star, black outline, direct label and arrow (p. 7).

Answering the Question

The problem remains active; the gap was in our reference list. Section 2 (pp. 2–3) now discusses recent work on feature complexity [1], early simplicity bias [2], memorization [3], data pruning [4], optimizer dependence [5], and shortcut-versus-XOR learning [6].

LaBonte and Muthukumar [6] provide a particularly relevant precedent: a shortcut can dominate at a budget where XOR would be learned without it. We now acknowledge that overlap explicitly. Our contribution is the specified combination of reference progress, deficits at crossing and over training, and both cue-free and label-independent comparisons, together with their theoretical analysis. We do not claim that cue-free comparisons themselves are new.

We thank the reviewer once again for their time and thoughtful suggestions, which have helped us improve the clarity and precision of the paper.

References

[1] Qiu et al. Complexity Matters: Feature Learning in the Presence of Spurious Correlations. ICML 2024. https://proceedings.mlr.press/v235/qiu24e.html

[2] Yang et al. Identifying Spurious Biases Early in Training through the Lens of Simplicity Bias. AISTATS 2024. https://proceedings.mlr.press/v238/yang24c.html

[3] Bayat et al. The Pitfalls of Memorization: When Memorization Hurts Generalization. ICLR 2025. https://proceedings.iclr.cc/paper_files/paper/2025/file/d5ae1c5167be330b90f4c0a1eed7f8f0-Paper-Conference.pdf

[4] Mulchandani and Kim. Severing Spurious Correlations with Data Pruning. ICLR 2025. https://arxiv.org/abs/2503.18258

[5] Vasudeva et al. The Rich and the Simple: On the Implicit Bias of Adam and SGD. NeurIPS 2025. https://proceedings.neurips.cc/paper_files/paper/2025/hash/ec9b2a6ad5444caeff75efaa6176b3e4-Abstract-Conference.html

[6] LaBonte and Muthukumar. SGD Provably Prioritizes a Shortcut Spurious Feature in the XOR Model. NeurIPS 2026. https://arxiv.org/abs/2606.30444

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Update and Request for Final Feedback

Dear Reviewers and Area Chair,

Thank you for your time and constructive feedback. We submitted our detailed responses and revised manuscript on 15 November, addressing the concerns raised during the review process.

As the discussion period concludes today, we would greatly appreciate any final feedback on whether outstanding concerns remain. If our revisions and clarifications have satisfactorily addressed your concerns, we would be grateful if you could consider updating your assessments accordingly.

We remain happy to provide any further clarifications.

Thank you again for your consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.