- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how rare counterexamples change subsequent learning from majority samples through their effects on feature margins and gradients. It derives correction time and counterexample count bounds in a model with two fixed features, tests the mechanism through controlled interventions and small neural networks, and introduces Shortcut to Knowledge (SK) to predict learning progress using fixed representations.
Critical strengths
- Theorem 4.1 and Appendix A establish that matching feature marginals and covariance does not determine correction times. The analysis also considers endpoint error and the persistence of correction, which strengthens the contribution beyond the initial crossing criterion.
- The intervention in Figure 4(a) provides direct evidence that majority robust feature updates contribute to the correction time gap in the proposed model. The appendices also provide detailed numerical checks and additional controls.
Critical weaknesses
- The theoretical model has limited generality, and the neural network validation is not yet sufficiently convincing. The main theory assumes predefined robust and shortcut features, a perfectly label-aligned robust signal, and balanced diagonal factorization. The correction time analysis depends on the resulting multiplicative weight dynamics, with no corresponding guarantee for jointly learned representations. Moreover, the additional ordinary CNN control reports a mean matched-norm, time-averaged worst-group accuracy gap of only 0.413 percentage points across four settings, including one negative result (Appendix B.3). Although these experiments differ in setup and evaluation from Figure 4(c), they raise concerns about the consistency and generality of the effect in neural networks. Given existing analyses of gradient competition and spurious feature learning [1, 2], the new feature pairing mechanism would benefit from stronger validation beyond this restricted setting.
- The SK evaluation provides limited evidence for its broader practical significance. SK predicts a local quadratic approximation to the training of two readout coefficients with fixed representations. Prediction and validation appropriately use the same initialization, context candidates, training budget, and scoring rule (Appendix C.1). The reported dataset-level Spearman correlation of 0.991 therefore supports this specific forecasting task, but does not directly validate the proposed majority learning mechanism or robust representation learning. Appendix C.3 also reports positive SK improvement for GOOD-HIV despite zero endpoint all-context accuracy, highlighting that improved loss does not necessarily translate into robust classification. Finally, no counterexample selection procedure is evaluated against existing methods. The paper identifies an interesting mechanism, but its broader practical consequences remain insufficiently demonstrated.
References
[1] Gradient Starvation: A Learning Proclivity in Neural Networks
[2] Complexity Matters: Feature Learning in the Presence of Spurious Correlations
Questions for authors
- What evidence establishes that the proposed indirect majority learning mechanism improves learned representations, beyond increasing gradients on existing features? Appendix B.2 acknowledges that pretraining and update norm matching prevent the channel interventions from isolating cross-step feedback alone. Could additional controls distinguish representation improvement from changes in readout weights or update magnitudes?
- How do you explain the small and inconsistent ordinary CNN results in Appendix B.3 compared with the larger effects in Figure 4(c), accounting for differences in experimental settings and evaluation metrics? Would the conclusions remain consistent across additional initializations in networks that learn robust and shortcut features jointly?
- Does SK outperform a simpler first-order predictor, and does its predictive accuracy extend beyond the specified two-coefficient readout? Results connecting SK to endpoint robustness would be especially important.
Thank you for reading the appendices so carefully and for recognizing the intervention in Fig. 4(a). Your observations about the CNN results and GOOD-HIV brought out limits we should have discussed in the main text. We ran the following controls to address your questions.
Q1. Does feedback from ordinary examples improve the learned representation?
A1. We agree that a stronger gradient alone leaves this question open. Our new control found no consistent gain in how well a fresh classifier could read digit identity from the hidden features. In a shared tanh network trained from scratch on 3s and 8s, an ordinary-image update increased held-out digit margins by 0.00293 on average, or 0.00765 after a counterexample step (step size 0.05). To test the features separately, we froze the output layer and took a hidden-layer counterexample step. From that same state, two copies took equal-length updates on ordinary images, using the gradient computed before or after the counterexample step.
We fit fresh linear probes with balanced synthetic backgrounds and five-fold cross-validation on 77 held-out digit identities. Across six checkpoints for each of ten seeds, mean squared probe error was 0.239138 with the original gradient and 0.239115 with the recomputed gradient. Only 5/10 seeds favored recomputation on average. In a 200-cycle control, the two probes had identical worst-group accuracy within every seed (95.10% mean in both arms). We will narrow our claim to how counterexamples change later updates and explain how this relates to the work you cited on gradient competition and feature complexity. Pezeshki et al. and Qiu et al..
Q2. Why is the ordinary CNN gap smaller than in Fig. 4(c), and does it persist across initializations?
A2. We should have discussed the small average CNN gap and the negative result alongside Fig. 4(c). That figure reports the digit-network gap at step 500 (3.52–19.07 percentage points across three seeds). Appendix B.3 reports a 6,000-step average on Fashion-MNIST (+0.413 points across four settings, one negative). We measured both on the same digit trajectories, matching update norms. The table gives mean worst-group accuracy gaps (Q minus P, percentage points) over eight new seeds per architecture.
For all eight seeds in each architecture, Q was ahead at step 500 and behind at step 6000. This shows how an early advantage can coexist with a near-zero time average. The original experiments also changed data and architecture, so their individual contributions remain open. We will report both time scales together.
Q3. Does SK improve on a first-order forecast? Does it extend to wider readouts or endpoint accuracy?
A3. The first-order baseline you suggest was missing from our evaluation. We tested c(t)=c(0)−t∇L(c(0)) on the original 142 tasks and 426 folds with identical starting logits, contexts, budgets and scoring. At B=8, mean absolute error against direct training across the 20 dataset scores was 3.603 points for first order and 0.784 for SK (Spearman 0.962 and 0.991). We repeated the comparison with dense affine heads (10–105 parameters) on fixed logits from Waterbirds, PACS, UCI HAR and GOOD-HIV. The errors were 5.54 and 2.62. The largest quadratic error was 6.07 points on GOOD-HIV.
Your GOOD-HIV example illustrates why both outcomes matter. The fitted loss-progress score was +5.300, yet no query was classified correctly under every candidate context. Across 20 datasets, the SK score had Spearman correlation 0.649 with endpoint all-context accuracy. Computing accuracy directly from the predicted final logits gave a mean absolute error of 0.59 percentage points. We will report loss and accuracy together and state explicitly that these forecasts concern a classifier on fixed features.
Your question also prompted a selection test. On digits, we selected 8 of 72 counterexamples by how much they improved the next ordinary-image step on a separate calibration set. Over ten paired runs, held-out all-context accuracy was 15.7%, versus 23.0% for uniform sampling, 11.0% for highest training loss, and 19.1% for immediate calibration improvement. Given this result, we will present counterexample selection as an open question.