acceptodds
Under review as a conference paper at ICLR 2027

Knowing Less, Teaching More: Weak-to-Strong Generalization from Partial Knowledge

Abstract

Weak-to-strong generalization (W2S) occurs when a strong student trained on a weak teacher’s labels outperforms the teacher on unseen samples. We study how a strong student can exploit supervision from a weak teacher that reliably recognizes an easy feature but has only partial knowledge of a hard feature, producing stochastic label noise on hard-only samples. For a two-layer ReLU network trained by gradient descent, we show that hard-feature learning approaches a saturation level determined by the feature-strength-weighted balance between correctly and incorrectly labeled hard-only samples. Overlap samples containing both features initially promote hard-feature learning, but their contribution vanishes as the easy feature is learned. Nevertheless, the student retains a positive fraction of the saturation level while fitting incorrect labels through sample-specific noise, which has little effect on unseen samples. Consequently, its test error converges to zero during a prescribed late-training interval, even after fitting every weak label. Experiments with ReLU networks and language models show that, at fixed label accuracy, assigning correct labels to samples with stronger hard features improves feature learning and student accuracy. A teacher-selection study further shows that hard-feature knowledge can be a better criterion for assessing weak-supervision quality than validation accuracy.

Primary area:
learning theory
Keywords:
Weak-to-strong generalization, Feature learning, Noisy labels, Memorization, Teacher selection

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_WCbYOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies weak-to-strong generalization when the weak teacher knows a hard feature only partially, instead of not knowing it at all. In their data model, samples contain an easy feature, a hard feature, or both. The teacher always labels samples with the easy feature correctly. On samples with only the hard feature, it is right more often when the feature is stronger (larger α\alpha). For a two-layer ReLU student trained by gradient descent, the authors show three things. Early in training, the student's response to the hard feature approaches a level set by a strength-weighted ratio RR between correctly and wrongly labeled hard-only samples. Later, the student memorizes the wrong labels through each sample's own noise, while keeping part of the hard feature. As a result, the test error still goes to zero after the student fits all the weak labels.

Critical strengths
  1. A natural question: real weak teachers usually know hard features partially, not completely or not at all.
  2. The theory covers the whole training process, and synthetic experiments match it closely.
Critical weaknesses
  1. The hard feature is only a weaker linear direction, and teacher errors are random noise, unlike real hard features.
  2. Novelty is limited. With random flips below 1/2, two-layer ReLU networks learning the signal while memorizing wrong labels through sample-specific noise was already shown in [1]. The setup of easy, hard and overlap features also follows [2]. The new part is mainly that the flip probability depends on α\alpha.

[1] Y. Kou, Z. Chen, Y. Chen, and Q. Gu. Benign Overfitting in Two-layer ReLU Convolutional Neural Networks. In ICML, PMLR 202:17615–17659, 2023.

[2] J. Oh, J. Song, and C. Yun. From Linear to Nonlinear: Provable Weak-to-Strong Generalization through Feature Learning. In NeurIPS, 2025.

Questions for authors

Which steps in Appx. C are new compared with [1] and [2]? Are Lemmas C14 and C15 new?

[1] Y. Kou, Z. Chen, Y. Chen, and Q. Gu. Benign Overfitting in Two-layer ReLU Convolutional Neural Networks. In ICML, PMLR 202:17615–17659, 2023.

[2] J. Oh, J. Song, and C. Yun. From Linear to Nonlinear: Provable Weak-to-Strong Generalization through Feature Learning. In NeurIPS, 2025.

2 comments
AuthorsOpenReview ↗

Response to Reviewer WCbY

We sincerely thank the reviewer for the constructive feedback. Please see our responses to the reviewer’s questions.

W1. The hard feature is only a weaker linear direction, and teacher errors are random noise, unlike real hard features.

We represent the hard feature by a single direction so that we can track how much of it the two-layer ReLU student learns and retains. A feature is hard in our model because the teacher has learned it only partly. Limited model capacity or differences between the teacher’s and student’s training distributions can leave the teacher with a small response bhb_h along that direction, even when the feature is clearly expressed in some samples.

Noise memorized during teacher training can also affect predictions on new samples. This motivates the output-noise term ϵ\epsilon, which we model as independent across samples. On a hard-only sample, the teacher’s score aligned with the true label is 2αbh+ϵ2\alpha b_h + \epsilon. When the feature strength α\alpha is smaller, the same noise distribution is more likely to overwhelm the hard-feature contribution and flip the label. The probability of an error therefore depends on the sample’s feature strength.

Table 1 in Common Response C1 shows that trained Qwen teachers are less accurate on hard-only samples than on easy-only and overlap samples, while their average hard-only accuracy remains above chance. Table 2 also shows that average teacher accuracy is higher in the highest-α\alpha quintile than in the lowest on each dataset. These results support the kind of partial knowledge our model captures. Modeling shared misconceptions would require accounting for how teacher errors are related across samples.

W2. With random flips below 1/2, two-layer ReLU networks learning the signal while memorizing wrong labels through sample-specific noise was already shown in [1]. The setup of easy, hard and overlap features also follows [2]. The new part is mainly that the flip probability depends on α\alpha.

Q1. Which steps in Appendix C are new compared with prior work? Are Lemmas C14 and C15 new?

Our contribution goes beyond making the flip probability depend on α\alpha: we quantitatively connect early hard-feature saturation to later preservation when clean and flipped feature updates compete. Lemmas C14 and C15 provide the key preservation arguments for this regime.

As the reviewer notes, [1] establishes benign overfitting in two-layer ReLU networks: the network learns a shared signal while fitting incorrect labels through sample-specific noise. Its data model assumes random label flips with probability below 1/2, while its theorem requires p≤1/Cp\le 1/C for a sufficiently large constant CC (Condition 4.1). The signal coefficient increases throughout the analyzed interval (Proposition 5.2). Our model instead represents partial teacher knowledge through label correctness that depends on hard-feature strength. In our regime, supporting and opposing hard-feature updates can be comparable, leading to early saturation and a feature response that may subsequently decrease during noise memorization. Thus, the continued feature growth established in [1] does not provide the preservation guarantee needed here.

We adopt an easy/hard/overlap construction similar to [2], but study a different source of supervision. In [2], a linear teacher cannot recognize the nonlinear hard-feature patterns, and correctly labeled overlap samples supply the hard-feature learning signal. Its data-scarce proof relies on clean hard-feature updates dominating, while its data-abundant result guarantees generalization at an early stopping time before substantial sample-specific noise memorization. In our setting, partial teacher knowledge makes hard-only labels informative even without overlap samples. We ask how much knowledge the student extracts from these competing labels and retains during memorization.

Lemmas C14 and C15 supply the two technical steps needed to address this new question. Lemma C14 establishes a noise-response ordering: the smallest accumulated noise response among flipped hard-only samples is at least as large as the largest among clean hard-only samples. This comparison matters because noise memorization weakens both the opposing updates from flipped samples and the supporting updates from clean samples. Lemma C15 combines this ordering with R>1R>1 to control both effects and establish a positive lower bound on the hard-feature response during memorization. Under our assumptions, this bound and the early saturation level both scale with log⁡R\log R. This allows us to quantify the fraction of early hard-feature knowledge retained as the student fits the weak labels.

[1] Y. Kou, Z. Chen, Y. Chen, and Q. Gu. Benign Overfitting in Two-layer ReLU Convolutional Neural Networks. ICML, 2023.

[2] J. Oh, J. Song, and C. Yun. From Linear to Nonlinear: Provable Weak-to-Strong Generalization through Feature Learning. NeurIPS, 2025.

We sincerely thank all reviewers for their valuable comments and constructive feedback on our paper. As the reviewer discussion period is coming to an end, we would greatly appreciate it if you could take another look at our responses and share any further feedback or concerns.

Reviewer_xRMBOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies weak-to-strong generalization when a teacher reliably recognizes easy features but has noisy, partial knowledge of hard features. For a two-layer ReLU network trained by gradient descent, it characterizes an early hard-feature saturation point and proves that a positive fraction of this response survives subsequent noise memorization. Under the stated assumptions, the student fits all weak labels while achieving vanishing test error, even without overlap samples. Synthetic and sentiment-classification experiments examine how the allocation of correct labels across feature strengths affects learning and teacher selection.

Critical strengths
  1. The preservation argument is the most compelling part for me. Connecting faster noise memorization on mislabeled examples to a quantitative lower bound on retained feature knowledge gives the mechanism useful substance.

  2. The feature-strength-weighted balance explains why identical label accuracy can produce different learned responses, and the synthetic experiments test this directly.

  3. Holding inputs and the number of correct labels fixed makes the effect of which examples are labeled correctly particularly clear.

Critical weaknesses
  1. The conceptual novelty over prior feature-learning analyses appears limited. I find the connection between early saturation and later feature preservation interesting, but the broader novelty claim needs sharper justification. The paper closely follows [1] in its easy/hard/overlap patch construction, two-layer ReLU student, gradient-descent training, and signal–noise decomposition. Meanwhile, the changes simplify an important aspect of the original problem: replacing the randomly signed hard features with positive multiples makes the true-label task linearly separable, while the teacher’s weakness is imposed through an assumed response and independent output noise. The hard-only labels therefore already contain useful supervision, making generalization without overlap less surprising. This changes the source of supervisory information rather than resolving the architectural capability gap studied in [1]. Furthermore, [1, Theorem 3.4] already establishes W2SG through benign overfitting in its data-scarce regime, so successful generalization after fitting weak labels is not itself new. The clearest contribution seems to be the quantitative saturation-to-preservation analysis under the modified assumptions. I would appreciate a more explicit comparison explaining which technical obstacle this analysis resolves beyond existing results and why it yields a substantially new understanding of W2SG.

  2. The teacher model limits what the result explains about W2SG. The teacher's weakness is imposed through its response and independent output noise, rather than derived from training a less expressive model. Moreover, the true-label task in Definition 1 is linearly separable: a classifier along μ++ν+−μ−−ν−\mu_+ + \nu_+ - \mu_- - \nu_- applied to the sum of patches separates all three sample types. I am comfortable with a stylized model, but this particular choice makes the result primarily an explanation of learning from informative noisy labels. Its implications for capability gaps or eliciting pretrained knowledge are less clear, especially when real teacher errors may reflect shared, systematic misconceptions.

  3. The LLM evidence does not yet establish the full mechanism or a strong practical advantage. The hard-feature strength is estimated using reference models trained with true labels, and the learned response is a regression proxy based on the same strength variable. Their correlation is encouraging, but does not directly demonstrate late-stage feature preservation during error memorization. The teacher-selection result is also modest: 56 of 64 selections agree, and the overall improvement is only 0.131 percentage points.

[1] From Linear to Nonlinear: Provable Weak-to-Strong Generalization through Feature Learning. NeurIPS 2025.

Questions for authors
  1. Which part of the preservation proof requires a substantially different argument from existing signal/noise analyses, particularly [1]?
  2. Do the LLM training trajectories show retained hard-feature responses as weak-label training accuracy approaches interpolation?
  3. How stable is the teacher-selection gain across feature-estimation seeds, and how does it compare with confidence-weighted validation accuracy?
3 comments
AuthorsOpenReview ↗

Response to Reviewer xRMB (1/2)

We sincerely thank the reviewer for the constructive feedback. Please see our responses to the reviewer’s questions.

W1. What does the saturation-to-preservation analysis explain beyond prior work?

Q1. Which part of the preservation proof requires a different argument from existing signal/noise analyses, particularly [1]?

[1] studies weak-to-strong generalization from a linear CNN teacher to a two-layer ReLU CNN student. In their data distribution, the easy feature μy\mu_y has a fixed sign whenever it appears, while the hard feature appears with either sign, +νy+\nu_y or −νy-\nu_y, with equal probability. This sign symmetry prevents their linear teacher from classifying unseen hard-only samples above chance. The strong student learns the hard feature from correctly labeled overlap samples, where the easy feature lets the teacher give the correct label.

Our model allows the weak teacher to have partial knowledge of the hard feature. On a hard-only sample, the teacher’s output aligned with the true label is 2αbh+ϵ2\alpha b_h+\epsilon, where α\alpha is the sample’s hard-feature strength, bh is the teacher’s response along that direction, and ϵ\epsilon is output noise. When bh=0b_h=0, the teacher is at chance on hard-only samples, as in [1]. When bh>0b_h>0, the teacher can provide informative labels on hard-only samples even when the easy feature is absent.

In both settings, the easy feature helps the teacher label overlap samples correctly. We show that the student’s easy-feature response grows logarithmically before substantial noise memorization. As the student fits overlap samples using the easy feature, their gradients shrink, leaving clean and flipped hard-only samples to drive hard-feature learning. Learning the hard feature makes clean labels easier to fit, so their gradient weights decrease. It makes flipped labels harder to fit, so their gradient weights increase.

In the data-scarce proof of [1], the aggregate hard-feature update from clean samples outweighs the opposing update from flipped samples throughout the analyzed training interval. In Eq. (13), a clean-to-flipped count ratio above 123212^{32} is sufficient for this dominance for the hard-feature occurrences considered in the proof. Our analysis studies what happens when the two groups contribute comparable updates. The teacher’s partial knowledge supplies enough signal on hard-only samples for these updates to balance at a positive level b^\widehat b. We define RR as the ratio of the total hard-feature strength of clean hard-only samples to that of flipped hard-only samples. For R>1R>1, the hard feature saturation point b^\widehat b satisfies

\le \widehat b \le \frac{\log R}{2\alpha_{\min}}.$$ At this point, the updates to the hard feature balance, but individual samples still have gradients that can drive noise memorization. On clean hard-only samples, learning the hard feature moves the student’s prediction closer to the teacher’s label. On flipped hard-only samples, it moves the prediction further from the teacher’s label. At the same noise strength, flipped samples therefore have larger gradient weights, which drive faster noise memorization. Lemma C14 turns this comparison into a bound on the memorization speed. Up to a small error, every flipped hard-only sample learns at least as much noise as every correctly hard-only sample. Lemma C15 uses this ordering to show why the hard feature is preserved. Memorizing noise on a clean sample weakens its update in support of the hard feature, pushing the balance point down. Memorizing noise on a flipped sample weakens its opposing update, pushing the balance point up. C15 accounts for both effects by combining the noise ordering from C14 with $R>1$. It gives $$b_{t+1} \ge \min\\{b_t,\beta\\}, \qquad \beta = \frac{\log R}{4\alpha_{\max}} - o(1) > 0.$$ Since $\beta$ and the early saturation point $\widehat b$ both scale with $\log R$ under our assumptions, this bound guarantees that the student retains at least a constant fraction of $\widehat b$ through the end of the analyzed training interval, even after it fits all weak training labels.
AuthorsOpenReview ↗

Response to Reviewer xRMB (2/2)

W2. What does the teacher model explain about capability gaps and systematic teacher errors?

A smaller teacher or a difference between the teacher’s and student’s training distributions can leave the teacher with a weak response bhb_h along the hard-feature direction. For example, the hard feature may appear less often in the teacher’s training data. Noise memorized during teacher training can also influence predictions on new samples. This motivates our output-noise term ϵ\epsilon. When bhb_h is small, the hard-feature contribution 2αbh2\alpha b_h can be comparable to the noise. The noise is more likely to outweigh that contribution on hard-only samples with smaller α\alpha.

Table 1 in Common Response C1 shows a similar pattern in trained Qwen teachers. They classify easy-only and overlap samples more accurately than hard-only samples, while their average hard-only accuracy remains above chance. This is consistent with the teachers having partial knowledge of the hard feature. Table 2 also shows that, on each dataset, average teacher accuracy is higher in the highest-α\alpha quintile than in the lowest.

W3. What does the LLM evidence show about feature preservation, and how should we interpret the teacher-selection gain?

Q2. Do the LLM training trajectories show retained hard-feature responses as weak-label training accuracy approaches interpolation?

To examine preservation more directly, we tracked 20 dataset–teacher–student configurations in a separate Qwen-to-Qwen experiment for three epochs using the teacher’s original labels. For flipped hard-only training samples, we measured how often the student predicted the same incorrect label as the teacher.

Table 1. Hard-feature knowledge as the student fits flipped teacher labels.

EpochTraining accuracy on flipped hard-only samples (%)Hard-feature knowledge btb_t
161.02.38
275.02.22
388.42.79

The mean hard-feature response stayed positive as the student fit more of the teacher’s mistakes. These trajectories give evidence of feature preservation during fitting of flipped labels.

We agree that the average teacher-selection gain of 0.131 percentage points is modest. Selection by RR and by validation accuracy chooses the same teacher in 56 of 64 comparisons. In the eight cases where they differ, selection by RR yields higher student test accuracy in seven and equal accuracy in one, as summarized in Table 2. This suggests that RR can capture useful differences in supervision quality missed by validation accuracy, although the overall gain is small. Our response to Q3 details these disagreement cases and includes the confidence-weighted baseline.

Q3. How stable is the teacher-selection gain across feature-estimation seeds, and how does it compare with confidence-weighted validation accuracy?

We have consistently used eight seeds for the teacher-selection experiments reported in the paper. Appendix B.4 separately examines how the feature estimates change when we draw different samples to estimate the directions, keeping the reference models fixed. Increasing the estimation set from 500 to 2,000 samples raises the mean hard-feature rank correlation from 0.591 to 0.719. It also increases the overlap between the resulting hard-only subsets, measured by Jaccard similarity, from 0.739 to 0.781. These results quantify the stability of the estimated scores and subset assignments.

We also compare teacher selection by RR, validation accuracy, and confidence-weighted validation accuracy. Confidence-weighted accuracy weights each sample’s correctness by the teacher’s confidence in its predicted label and divides by the total confidence. For all 64 teacher-selection comparisons reported in the main text, Table 2 shows how selection by RR compares with selection by validation accuracy and confidence-weighted validation accuracy. We find that when the criteria select different teachers, selection by RR yields higher student test accuracy in most cases.

Table 2. Comparison of teacher-selection criteria.

ComparisonDifferent teacher choicesWins / losses / ties when choices differMean gain when choices differ (pp)Mean gain across all 64 (pp)
R vs. validation accuracy87 / 0 / 1+1.045+0.131
R vs. confidence-weighted accuracy65 / 0 / 1+1.813+0.170

Thank you for your response. I have no further concern. I will keep my positive score.

Reviewer_dwEtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper analyzes a setting of a randomly initialized two-layer ReLU student network trained on a teacher model which is a stochastic process (and not trained) which contains easy features, hard features, or both and noise. The main contribution of the paper is characterizing a hard-feature saturation level determined by feature-strength-weighted label correctness, and a proof that useful features can survive even if the network memorizes incorrect labels. The paper also includes synthetic and sentiment classification LLM experiments that try to capture the same effect, where they show that samples with correct labels and stronger hard features can improve student learning at fixed weak label accuracy.

Critical strengths

The main theoretical contribution of the paper is establishing that a student can outperform a teacher model by the feature preservation mechanism, and that the improvement is quantiatively characterized by feature-strength-weighted label correctness. However, I would classify this as strict weak-to-strong generalization. Further, the paper considers experiments on LLM sentiment classification and shows that in a similar setting, we have W2S generalization empirically. More improtantly, they show that selecting teacher models by feature-weighted accuracy outperforms selecting on just accuracy in terms of W2S improvement.

Critical weaknesses

The paper doesn't show that the proposed teacher model reasonable represents a trained teacher model, which takes a bit away from the claims of establishing any W2S generalization result. Why does training a teacher model introduce this limitation? If the noiseless teacher solves the task anyway, is the student just denoising a suboptimal teacher model?

While the exeprimental results on classifications with LLMs have interesting results on selecting the teacher, the connection to the theoretical setting is not fully established since its unclear that the feature preservation happens here.

Questions for authors
  1. Is there a simple explanation when a trained teacher model achieves the same limitation as in the setting?
  2. How does a student-trained-on-ground-truth compare with a student-trained-on-teacher-labels? Does the particular limitation of the teacher model "help" the student to generalize?
  3. What are the main mechanisms of W2S in this design and in the LLM experiment design? It reminds a bit of benign overfitting, but partly is also denoising.
3 comments
AuthorsOpenReview ↗

Response to Reviewer dwEt (1/2)

We sincerely thank the reviewer for the constructive feedback. Please see our responses to the reviewer’s questions.

W1. Does the proposed teacher model reasonably represent a trained teacher? If the noiseless teacher solves the task, is the student just denoising a suboptimal teacher?

Q1. Is there a simple explanation for when a trained teacher exhibits the limitation in our setting?

A trained teacher may reliably learn patterns that occur frequently and consistently in its training data, while acquiring only partial knowledge of rarer or less consistent patterns. It may recognize these harder patterns when they are pronounced but make errors when they are subtle. Our model captures this intuition by assuming that the probability of a correct label is nondecreasing in hard-feature strength.

We model the teacher through its responses rather than explicitly analyzing its training, which would require additional assumptions about its data, architecture, and optimization. Our goal is to understand how a student capable of both learning useful features and memorizing label noise extracts and preserves knowledge from a teacher that recognizes those features only partially.

Common Response C1 provides empirical support for this modeling choice. We train reference Qwen models of different sizes on 1,000 gold-labeled samples per model. We estimate an easy-feature direction from the smaller model and a hard-feature direction from the larger model after removing its easy-feature component. These directions define easy-only, overlap, and hard-only subsets and provide an empirical hard-feature strength α\alpha. We then evaluate trained Qwen teachers on these subsets.

Table 1. Teacher accuracy and estimated hard-feature strength. For each dataset, we evaluate all teacher–student combinations across eight seeds (4 teachers × 2 students × 8 seeds = 64 runs). The accuracy values are averages over 64 runs. E, O, and H denote easy-only, overlap, and hard-only. We also measure hard only sample’s hard feature strength α\alpha along our estimated hard-feature direction and report the mean separately for sample the teacher labels correctly (clean) and incorrectly (flipped).

DatasetAccuracy (%) E / O / HMean α\alpha clean / flipped
IMDb99.6 / 88.6 / 65.10.331 / 0.318
SST-298.2 / 96.1 / 75.40.098 / 0.086
RT97.5 / 96.7 / 68.00.075 / 0.068
Yelp98.5 / 98.8 / 84.40.155 / 0.127

Table 1 shows that teachers are less accurate on hard-only samples than on easy-only and overlap samples, yet achieve hard-only accuracy of 65.1% to 84.4%, above random guessing. Across all four datasets, correctly labeled hard-only samples also have greater mean estimated hard-feature strength than flipped samples. Furthermore, Table 2 in Common Response C1 shows a tendency for teacher accuracy to increase with α\alpha. Together, these findings support partial teacher knowledge and the assumed relationship between feature strength and label correctness. Definition 1 captures this intuition through the interaction between the teacher’s hard-feature knowledge bhb_h, feature strength α\alpha, and output noise ϵ\epsilon: stronger hard features are more likely to overcome the noise and receive correct labels.

Removing the output noise ϵ\epsilon would indeed make the teacher perfectly accurate in our model, eliminating the errors through which we represent its incomplete knowledge. Thus, denoising is part of the mechanism we study. Our theorem compares the student with the noisy teacher that actually supplies its training labels. The key question is why the student’s improvement survives even after it fits all of the teacher’s incorrect training labels. The student aggregates information across samples to learn a shared hard feature while fitting incorrect labels through sample-specific input noise. As detailed in Common Response C2, a positive fraction of the learned hard-feature response survives this memorization and supports generalization beyond the teacher.

AuthorsOpenReview ↗

Response to Reviewer dwEt (2/2)

W2. How do the LLM experiments connect to feature preservation in the theory?

Q3. What are the main mechanisms in the theory and in the LLM experiments? Are they denoising or benign overfitting?

Our theory connects both mechanisms. Early training resembles denoising: the student aggregates information across samples to learn a shared hard feature despite incorrect teacher labels. Later training exhibits benign overfitting: the student fits incorrect labels by memorizing sample-specific input noise while preserving part of its learned hard-feature response. This shared feature supports accurate predictions on unseen samples, whereas memorized noise has little effect on them.

The controlled LLM experiment in Sec. 4.2 examines hard-feature acquisition. We hold the training samples and hard-only label accuracy (50%) fixed while varying which samples receive correct labels. Assigning correct labels to samples with stronger estimated hard features leads to greater estimated hard-feature knowledge in the student.

To examine feature preservation more directly, we conduct an experiment covering 20 dataset–teacher–student configurations, fine-tuning larger Qwen students on smaller Qwen teachers’ original labels for three epochs. From epoch 1 to epoch 3, training accuracy on flipped hard-only samples, measured against the teacher’s labels, rises from 61.0% to 88.4%. Meanwhile, mean estimated hard-feature knowledge is 2.38, 2.22, and 2.79 at the three checkpoints. Thus, hard-feature knowledge is retained as students fit more teacher errors.

These LLM results are consistent with the acquisition-and-preservation pattern predicted by our theory.

Q2. How does a student trained on ground-truth labels compare with one trained on teacher labels? Does the teacher’s limitation help the student generalize?

In our model, using gold labels on the same training samples removes the opposing updates from flipped labels that cause hard feature learning to saturate early. The teacher’s errors make the learning problem harder. What helps the student is the information that remains in the teacher’s labels, especially when the teacher correctly labels samples with strong hard features. We study this setting because teacher labels are easier to obtain at scale than gold labels.

We sincerely thank all reviewers for their valuable comments and constructive feedback on our paper. As the reviewer discussion period is coming to an end, we would greatly appreciate it if you could take another look at our responses and share any further feedback or concerns.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Common Response (2/2)

C2. Theoretical novelty compared to prior works(W1/Q1 from reviewer xRMB, W2/Q1 from reviewer WCbY)

Some prior work studies weak-to-strong generalization when the weak teacher is blind to a hard feature. In [1], the weak teacher is a linear CNN, and the hard-only samples cannot be separated by a linear classifier. The strong student can still learn the hard feature from correctly labeled overlap samples, which contain both easy and hard features. In our setting, the weak teacher already has partial knowledge of the hard feature, so even its noisy labels on hard-only samples can help the student learn.

In the data-scarce proof of [1], the aggregate hard-feature gradient from clean samples dominates that from flipped samples. To establish this dominance using the bound in Lemma C.3, Eq. (13), they need more than 123212^{32} clean samples compared to flipped samples. This is an unrealistically large imbalance for noisy weak supervision.

In our setting, the clean and flipped gradients can become comparable. As the easy-feature response ata_t grows logarithmically, the gradients from easy-only and overlap samples shrink. The hard-feature knowledge btb_t approaches a finite level b^\widehat b, where the updates from clean and flipped hard-only samples balance. This level depends on both which samples the teacher labels correctly and how strongly each sample expresses the hard feature.

At the balance point, clean and flipped hard-only samples push the shared hard feature in opposite directions, and their updates cancel. However, the student can still reduce the loss on individual samples by memorizing sample-specific noise. The learned hard feature already helps fit clean labels, but makes flipped labels harder to fit. Thus, flipped samples have larger prediction errors and receive stronger gradients for noise memorization. Lemma C14 shows that this difference is reflected in the accumulated noise response. Up to a small error, every flipped hard-only sample accumulates at least as much noise response as any clean hard-only sample.

Memorizing noise on flipped samples makes their incorrect labels easier to fit, so they push less strongly against the hard feature. Memorizing noise on clean samples also weakens the updates that support the hard feature. The first effect pushes the balance point up, while the second pushes it down. Lemma C15 uses the noise ordering from Lemma C14 to control both effects and show that hard-feature knowledge remains above a positive level when R>1R>1. Here, RR is the total hard-feature strength of clean hard-only samples divided by that of flipped hard-only samples. This gives the following bound during noise memorization. bt+1≥min⁡bt,β,β=log⁡R4αmax⁡−o(1)>0.b_{t+1} \ge \min\\{b_t,\beta\\}, \qquad \beta = \frac{\log R}{4\alpha_{\max}}-o(1)>0. Since b^≤log⁡R/(2αmin⁡)\widehat b \le \log R/(2\alpha_{\min}), the lower bound is on the scale of the early saturation level under our assumptions. Combining it with the early learning result gives Theorem 2’s guarantee that the student retains a constant fraction of b^\widehat b throughout the analyzed late-training interval, including after it fits all weak training labels.

Our analysis connects a weak teacher’s partial knowledge to how much the student learns and retains. The balance between clean and flipped hard-only updates determines the early saturation level. We then show that the difference in noise memorization between these groups preserves a constant fraction of this early knowledge, even after the student fits all weak training labels. This quantitative connection between saturation and preservation explains how the student can retain its improvement over the teacher while fitting the teacher’s mistakes.

[1] J. Oh, J. Song, and C. Yun. From Linear to Nonlinear: Provable Weak-to-Strong Generalization through Feature Learning. NeurIPS, 2025.

AuthorsOpenReview ↗

Common Response (1/2)

We sincerely thank all the reviewers for their feedback. We first address the common questions from the reviewers.

C1. Motivation of the teacher-response assumption (W1/Q1 from reviewer dwEt, W2 from reviewer xRMB, W1 from reviewer WCbY)

Weak-to-strong generalization starts from the observation that a strong student can learn from a weak teacher’s noisy outputs and outperform the teacher. Weak-model outputs are cheaper and easier to obtain than gold labels, but using them requires understanding and controlling the noise they introduce.

Our starting intuition is that a weak teacher recognizes easy patterns reliably and has partial knowledge of harder ones. For example, in a dog-versus-cat classification task, the teacher might recognize the eye shape reliably but have only partial knowledge of ear shape, which may vary across samples. It may still classify images more reliably when the differences in ear shape are pronounced. We therefore assume that the probability of a correct teacher label increases with a larger hard-feature strength.

To validate this assumption empirically, we train models from the Qwen2.5 and Qwen3 families, using a separate set of 1,000 gold-labeled samples for each model. We compare their learned representations to estimate an easy-feature direction that reflects a pattern learned by the smaller Qwen models and a hard-feature direction that reflects an additional pattern learned by the larger Qwen models. We measure each sample’s components along these directions and use them to divide the samples into easy-only, overlap, and hard-only subsets.

For each dataset, we evaluate all teacher–student combinations across eight seeds (4 teachers × 2 students × 8 seeds = 64 runs). The accuracy values are averages over 64 runs. E, O, and H denote easy-only, overlap, and hard-only. We also measure hard only sample’s hard feature strength α\alpha along our estimated hard-feature direction and report the mean separately for sample the teacher labels correctly (clean) and incorrectly (flipped).

Table 1. Teacher accuracy and estimated hard-feature strength.

DatasetAccuracy (%) E / O / HMean α\alpha clean / flipped
IMDb99.6 / 88.6 / 65.10.331 / 0.318
SST-298.2 / 96.1 / 75.40.098 / 0.086
RT97.5 / 96.7 / 68.00.075 / 0.068
Yelp98.5 / 98.8 / 84.40.155 / 0.127

Across the four datasets, mean hard-only accuracy ranges from 65.1% to 84.4%, below both easy-only and overlap accuracy on each dataset, yet above random guessing. In the dataset-level averages, correctly labeled hard-only samples also have higher mean α\alpha than flipped samples.

We also divide each 500-sample validation set into five equal groups, from lowest (Q1) to highest (Q5) feature strength. On all four datasets, teachers are more accurate on average in the highest-α\alpha quintile than in the lowest. These results support our model of partial teacher knowledge, in which stronger hard features are more likely to receive correct labels.

Table 2. Hard-only teacher accuracy (%) by quintile of α\alpha.

DatasetQ1Q2Q3Q4Q5Q5 − Q1 (pp)
IMDb58.764.575.969.559.6+0.9
SST-266.870.675.876.876.3+9.4
RT63.068.067.971.571.3+8.3
Yelp79.983.280.082.892.8+12.9
Area_Chair_45tuOpenReview ↗

Reminder: Author-reviewer discussion period ending soon

Dear Reviewers,

Thank you again for your time and effort in reviewing this paper. This is a friendly reminder that the author-reviewer discussion period is coming to a close.

If you haven't already, please check out the authors' rebuttal and engage in the discussion by leaving comments. Even a brief acknowledgment confirming whether your questions were addressed would be very helpful for our discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.