acceptodds
Under review as a conference paper at ICLR 2027

An Active-Bottleneck Mechanism for Weak-to-Strong Generalization

Abstract

Weak-to-strong generalization (W2SG) occurs when a student trained on a teacher's predictions outperforms that teacher. We study when this happens under fully converged, ridgeless two-stage learning, with no early stopping, no explicit regularization, and no assumption that the student is more expressive than the teacher. In two-stage linear regression, a teacher is fit from labeled examples and a student is trained solely on the teacher's predictions on fresh, unlabeled inputs. Although both stages share the same hypothesis class and the same training rule, we show that the student outperforms the teacher exactly when lies in an explicit intermediate range: too few pseudo-labels leave the student without enough signal, too many let it inherit the teacher's noise. Under power-law covariance, we derive this range in closed form as a function of the spectral decay and noise level, including regimes where the improving region splits into two disjoint intervals of . We then study a random-feature model in which the student has strictly more features than the teacher, and identify two regimes, again given by explicit thresholds: one where improvement occurs only for in a bounded interval, and one where it occurs only once the student width exceeds an explicit threshold. Both regimes are governed by a single active-bottleneck principle: whichever of or is scarcer controls how much teacher error is filtered out, while increasing the other resource only reduces estimation noise. Together, these results show that finite data and finite width can themselves regularize a two-stage learner, with no explicit mechanism doing so.

Primary area:
learning theory
Keywords:
Weak-to-Strong Generalization, Knowledge Distillation, Deterministic Equivalence, Implicit Regularization, Random Features

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_wxeLOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies weak-to-strong generalization (W2SG) in fully converged, ridgeless two-stage learning. The main observation is that a student can outperform its teacher even when both use the same hypothesis class and training procedure. The authors attribute this to the finite number of pseudo-labeled samples, which can filter out part of the teacher's error. They analyze this effect under power-law covariance and extend the framework to random-feature models, where either sample size or student width can become the active bottleneck.

The paper provides deterministic-equivalent risk expressions and derives phase boundaries for several asymptotic regimes.

Critical strengths
  1. Clear and useful mechanism.

  2. Substantial scalar analysis.

  3. Correct exact geometric results.

  4. Informative conditional phase formulas.

Critical weaknesses
  1. My main concern is the connection between the deterministic-equivalent calculations and the actual ridgeless risks. Several derivations start from positive-ridge deterministic equivalents and then take the ridgeless limit. Since the covariance spectrum also changes with dimension, I am not convinced that the cited results directly justify these limit exchanges.

This matters because the phase diagrams depend on the signs of risk differences, not simply on approximations to the individual risks. It would help to state precisely which approximation guarantees apply in the spectral regimes considered.

  1. Remark 3.5 states that the fixed-sample and proportional regimes give a complete picture across the possible scalings of m.

I am not fully convinced by this conclusion at the critical exponent α=2\alpha=2. Theorem C.23 gives a zero leading-order gain in the sparse-growing regime, which by itself does not determine the eventual sign. Expanding the finite-p expression in equation (36) suggests competing contributions of orders 1/m and m/p. These become comparable when m is of order p\sqrt p, potentially producing an additional transition for some noise levels.

This does not invalidate the stated fixed-m or proportional calculations, but it suggests that the completeness claim in Remark 3.5 may be too strong.

Questions for authors
  1. Can the authors clarify which deterministic-equivalent results justify the ridgeless limit under the dimension-dependent covariance spectra? Is uniform control available in the relevant regimes?

  2. At α=2\alpha=2, could there be a transition when m=Θ(p)m=\Theta(\sqrt p)? If so, how should Remark 3.5 be modified?

  3. For Case II, can the authors clarify whether the phase boundaries are established for the actual expected risk or for the master deterministic equivalent, particularly when the predicted gain vanishes with dimension?

5 comments

We thank the reviewer for their comments.


General Response

We have revisited the role of the deterministic-equivalence framework of Atanasov et al. in our analysis. While this framework remains useful for identifying the corresponding master deterministic equivalents, we agree with the reviewer that relying on it alone does not fully justify the asymptotic behavior of the actual expected risks.

To address these concerns, we have strengthened Appendices C and D by developing a direct finite-dimensional ridgeless expected-risk analysis (please see the revised manuscript). Specifically, we derive exact risk decompositions and identities for the actual estimators and establish the uniform estimates needed to pass to the relevant asymptotic regimes. This allows us to establish the sign conclusions for the actual expected-risk differences, rather than relying solely on those of their deterministic equivalents.

These additions preserve the phase boundaries and threshold formulas reported in the main text, while strengthening their theoretical justification and clarifying the regimes in which the conclusions hold for the actual expected risks. Finally, we will update the main-text notation by replacing the DE notation with actual-risk notation wherever justified by the new results. The main theorem statements, phase boundaries, and threshold formulas remain unchanged.

We describe the corresponding additions in response to each question below.


Question 1. Can the authors clarify which deterministic-equivalent results justify the ridgeless limit under the dimension-dependent covariance spectra? Is uniform control available in the relevant regimes?

Response: We thank the reviewer for identifying this technical issue. Our original proportional-regime analysis follows the deterministic-equivalence framework of Atanasov et al. (2026). However, as the reviewer correctly points out, invoking this framework alone does not fully resolve the distinction between deterministic equivalents and actual expected risks, particularly when the covariance spectrum changes with dimension and the sign of a difference between two risks is of interest.

To address this concern, we have substantially strengthened Appendix C by deriving the proportional-regime expected-risk formula directly for the finite-dimensional ridgeless estimators. Thus, the revised argument does not require exchanging a positive-ridge limit with the high-dimensional limit.

In particular, Lemmas C.3–C.5 establish the uniform fixed-point, spectral, and quadratic-form estimates, while Theorems C.6 and C.7 establish the weighted pseudoinverse and Stage-II projection estimates required for the direct ridgeless derivation. Theorem C.8 then establishes the finite-dimensional expected-risk decomposition with an explicit, uniformly controlled remainder. Theorem C.10 derives the corresponding power-law asymptotic limit. We also treat the other sampling regimes separately: Theorem C.26 controls the approximation error in the sparse-growing regime, while Theorem C.37 establishes the fixed-sample expected-risk expansion.

Consequently, the revised analysis provides direct control of the actual expected-risk improvement in each regime, rather than relying solely on deterministic-equivalent formulas for the individual risks.


Question 2. At α=2\alpha=2, could there be a transition when m≍pm\asymp\sqrt p? If so, how should Remark 3.5 be modified?

Response: We agree that the vanishing leading-order gain at α=2\alpha=2 does not determine the eventual sign, and that the competing contributions of orders 1/m1/m and m/pm/p suggest an additional transition. Motivated by this observation, we have added a second-order analysis in Appendix C. Proposition C.25 identifies the critical m≍pm\asymp\sqrt p window for the scalar approximation. More importantly, Theorem C.30 establishes the corresponding transition for the actual expected gain, accounting for the finite-sample Stage-II variance correction. In particular, it identifies the shifted critical threshold for the actual expected risk.

We have accordingly revised Remark 3.5 to acknowledge this additional critical window and to distinguish the transition predicted by the scalar approximation from the one established for the actual expected gain. Thus, the fixed-sample and proportional regimes alone do not fully describe the behavior at the critical exponent α=2\alpha=2.


Question 3. For Case II, can the authors clarify whether the phase boundaries are established for the actual expected risk or for the master deterministic equivalent, particularly when the predicted gain vanishes with dimension?

Response: Our original Case II analysis followed the deterministic-equivalence framework of Atanasov et al. (2026), which yielded the phase diagram in Theorem 4.3 for the master deterministic equivalent. However, as the reviewer correctly points out, the phase boundaries of the deterministic equivalent do not, by themselves, establish the corresponding sign conclusions for the actual expected risks, particularly when the gain vanishes with dimension.

To address this concern, we have replaced the deterministic-equivalent analysis in Appendix D with a direct finite-dimensional analysis of the actual expected risks, starting from exact risk identities for the two bottleneck regimes. In particular, Theorem D.9 establishes the regular phase diagram directly for the actual expected population risks. The resulting normalized limits yield exactly the same phase boundaries as those stated in Theorem 4.3. Whenever the corresponding limiting numerator is nonzero, its sign determines the eventual sign of the actual expected gain. At exact zeros of the limiting numerator, Theorem D.9 does not by itself determine the lower-order sign. The interpolation boundary and joint critical regime are treated separately in Appendix D.5.

We have made this distinction explicit in Appendix D and will update the corresponding main-text notation in the final revision.


We hope that these additional results and clarifications address the reviewer's concerns and strengthen the theoretical justification of our main results.

Dear wxeL,

We would like to kindly follow up on our response to your review and the corresponding revisions to the manuscript.

We sincerely appreciate the concerns you raised, particularly regarding the justification of the ridgeless limits, the possibility of an additional critical scaling at α=2\alpha=2, and the distinction between deterministic-equivalent and actual expected-risk results in Case II. Your comments prompted us to substantially strengthen the theoretical analysis.

In particular, we have made the following revisions:

  1. Case I — Ridgeless analysis: We developed a direct finite-dimensional expected-risk analysis, establishing the required uniform estimates and controlled approximation bounds. The revised derivation no longer relies on exchanging the positive-ridge and high-dimensional limits.

  2. Critical exponent α=2\alpha=2: We added a second-order analysis identifying the additional m≍pm\asymp\sqrt{p} critical window, including the corresponding transition for the actual expected risk improvement. We also revised Remark 3.5 accordingly.

  3. Case II — Random-feature model: We replaced the deterministic-equivalent-based phase analysis with a direct finite-dimensional analysis of the actual expected risks. The revised Appendix D establishes the regular phase diagram for the actual expected-risk difference and separately addresses the interpolation boundary and critical scaling regimes.

Importantly, in the latest revision, we have incorporated these changes throughout the manuscript, including the main text, appendices, and experimental section, rather than limiting the revisions to the technical appendices. We have updated the corresponding statements, explanations, and notation to maintain consistency across the paper.

We would be very grateful if you could take another look at our response and the latest revised manuscript and let us know whether these changes adequately address your concerns, or whether any issues remain.

Thank you again for your time, careful assessment, and constructive feedback. We genuinely appreciate your contribution to improving our work.

Best regards,
The Authors

Reviewer_TiUsOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes an active-bottleneck mechanism for weak-to-strong generalization (W2SG), in which a finite sample size or student width can filter teacher errors without early stopping or explicit regularization. Using deterministic equivalents, it characterizes regimes where W2SG occurs in ridgeless linear regression with identical teacher and student hypothesis classes, and then extends the analysis to a random-feature model with a wider student.

Critical strengths

I find the active-bottleneck mechanism interesting. A finite sample size or student width can act as an implicit regularizer, allowing the student to improve on its teacher without early stopping or explicit regularization. In particular, distinguishing the resource that controls teacher-error filtering from the resource that primarily reduces estimation noise provides a useful perspective on the different roles of data and model size.

Critical weaknesses
  • Although the active-bottleneck mechanism is the paper's main idea, I initially found it difficult to understand concretely what it means and how it operates in the two settings. A conceptual diagram or summary table explaining how it limits teacher imitation and highlighting the roles of the bottleneck and non-bottleneck resources would make the main insight easier to grasp.

  • I am not fully convinced that Case I can be considered W2SG rather than self-distillation. The teacher and student share the same hypothesis class and feature space, with no capability advantage for the student. This setting differs from that of Burns et al. The paper would benefit from explaining more clearly what Case I reveals about W2SG.

Questions for authors

In Case I, in what sense are the teacher and student "weak" and "strong," respectively? Since both models share the same hypothesis class and feature space, I am not sure whether these results should instead be interpreted as self-distillation. Could the authors clarify how the Case I results bear on W2SG?

3 comments

We thank the reviewer for their comments, and also their positive assessment of our work.


On the interpretation and motivation of Case I

We responded to a similar concern from Reviewer JzZq. We agree with the reviewer that the original motivation of W2SG involves an asymmetry between a weaker teacher and a stronger student, and that Case I can also be interpreted as self-distillation. However, we believe Case I provides a complementary perspective on W2SG.

We want to bring attention to another, less-attended aspect of W2SG: the role of a sample bottleneck as an implicit regularizer. Our main motivation for Case I is to isolate the role of mm in student improvement. Introducing a representational discrepancy from the beginning, as in Case II, would confound the effect of mm with differences in model capacity. We therefore first consider identical teacher and student hypothesis classes to demonstrate that mm alone can determine whether student improvement occurs. We agree that presenting this result may obscure the distinction between same-class self-distillation and conventional capacity-asymmetric W2SG. We will clarify that Case I is intended to isolate the sample-size mechanism and to serve as a bridge to Case II, rather than to represent conventional capacity-asymmetric W2SG.

We agree that presenting this result as conventional W2SG may obscure the distinction between same-class self-distillation and conventional capacity-asymmetric W2SG. We will clarify that Case I is intended to isolate the sample-size mechanism and serve as a bridge to Case II, rather than to represent conventional capacity-asymmetric W2SG.

In contrast, Case II considers a genuine capacity-asymmetric W2SG setting, where NSN_S and NTN_T denote the numbers of student and teacher random features, respectively, and pp denotes the input dimension. We show that a necessary condition for W2SG is NS>NTN_S>N_T, so a student capacity advantage is required in our random-feature model. This advantage alone is not sufficient, however: the number of pseudo-labeled student samples mm, relative to NSN_S and pp, plays a crucial role in determining whether the stronger student can outperform the teacher.


On the difficulty of understanding the bottleneck mechanism

We agree that the active-bottleneck mechanism could be explained more concretely. We will add a conceptual diagram and further explanation illustrating how the active bottleneck (student sample size mm or student width $N_S$) limits teacher imitation and filters teacher error, while the non-bottleneck resource primarily reduces estimation noise.


Response to Question 1: As discussed in "On the interpretation and motivation of Case I," we agree that Case I can be interpreted as self-distillation. Since both models share the same hypothesis class and feature space, the distinction between weak and strong is based on their generalization performance rather than an a priori capacity advantage. The purpose of Case I is to isolate how the sample-size bottleneck can improve student generalization, providing a foundation for Case II, where a genuine capacity asymmetry is introduced. We will clarify this distinction in the revised manuscript.

I thank the authors for their response. I believe these clarifications will strengthen the paper. However, the promised conceptual diagram and the explicit distinction between Case I as self-distillation and conventional capacity-asymmetric W2SG do not yet appear to be reflected in the current manuscript, so I kindly ask the authors to incorporate these points into the revision. Provided that the revision reflects these points, I will maintain my positive score.

We thank the reviewer for the follow-up. We have incorporated the requested changes into the revised manuscript. Specifically, we have revised the Introduction and the standalone introduction of Case I to clarify its interpretation as same-class self-distillation, its complementary role in understanding W2SG, and its connection to Case II. We have also added the requested conceptual diagram (Figure 1), together with an explanation of the active-bottleneck mechanism and the distinct roles of the active and non-active Stage-II resources.

We thank the reviewer again for these helpful suggestions, which have improved the clarity of the manuscript.

Reviewer_JzZqOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper claims two contributions:

  1. in the setting of student-teacher training of one ridgeless linear regression by another, the student model can generalize better than the teacher depending on noise level and the scaling limit of m/p (m=student samples, p = dimension), both of which require m<p. This shows that there is an effect to teacher-student generalization coming from the "bottleneck" of available samples.
  2. In random-features weak-to-strong generalization setting, the authors analyize necessary and sufficient conditions for W2S in random feature networks trained in the proportional scaling limit, and show how these change as a function of # samples of student, input dim, # features of student, # of features of teacher.
Critical strengths

The main strength of the paper is identifying the "bottleneck" concepts for the random feature regression setting of W2S generalization and showing whether W2S occurs depends on these bottlenecks. In particular, they show necessary and sufficient conditions on N_S = # student feat, #N_T, p=# input dim, and m= # number of student samples, in order for W2S to happen in this setting. The most interesting part of the result is that we can now understand the interaction between N_T and N_S and whether W2S occurs. This is what remark 4.4 says.

Critical weaknesses

The authors, in my opinion, slightly misinterpreted what W2S generalization is and hence phrase their results in a "positive" way for W2S, however I would consider these "negative" for W2S.

The cruciual aspect of W2S, as introduce in Burns et al, is the asymmetry between the student and the teacher, in that the teacher is "stronger" i.e. more capable model in some way, i.e. by having larger capacity, more features, feaures that are more algined with the target, better implicit bias, etc. And as Burns et al was written there was something unexpected in this setup. So the idea is that this asymettry introduces the unexpected aspect. However, the fact that a similar teacher-outperformed-by-student outcome even in the regime where they are the same model to me shows that that mechanism is what could undermine the claim of W2S generalization being an interesting phenomenon (hence "negative" for the phenomenon). In some sense, this makes the result of the paper more interesting, but the presentation misses this.

Second, the results are not quntiative in terms of how much W2S improvement is there, which would be interesting to understand.

Questions for authors
  1. Can you quantify in any of the cases the size of W2S^{DE}? E.g. is it bounded away from 1 (or even above)?
  2. For the random features setting, can we understand how W2S^{DE} beahves as a function of N_S and N_T? In particular, do you think that W2S^{DE} grows as N_S/N_T grows?
4 comments

We thank the reviewer for their comments.


On the interpretation and motivation of Case I

We agree with the reviewer that the original motivation of W2SG involves an asymmetry between the teacher and student. However, we believe our Case I provides a complementary perspective.

  • Primary: We want to bring attention to another, less-attended aspect of W2SG: the role of a sample bottleneck as an implicit regularizer. Our main motivation for Case I is to isolate the role of mm in student improvement. Introducing a representational discrepancy from the beginning, as in Case II, would confound the effect of mm with differences in model capacity. We therefore first consider identical teacher and student hypothesis classes to demonstrate that mm alone can determine whether student improvement occurs. We agree that presenting this result may obscure the distinction between same-class self-distillation and conventional capacity-asymmetric W2SG. We will clarify that Case I is intended to isolate the sample-size mechanism and to serve as a bridge to Case II, rather than to represent conventional capacity-asymmetric W2SG.

  • Secondary: A related mechanism in the W2SG literature is the filtering of noisy feature directions. Ildiz et al. study optimal surrogates that explicitly zero out noisy coordinates, while Wu and Sahai, as well as Shin et al., consider students with access to stronger features than the teacher. In Case I, the distinction between strong and weak directions emerges differently. Although teacher and student share the same hypothesis class and feature space, the covariance structure separates high-variance (favored) from low-variance (less-favored) eigendirections, and the teacher's estimation error can have components along the latter. Since the student is trained on mm fresh teacher-labeled samples, mm indirectly controls the projection of the teacher's predictor onto the row space of the student design matrix. This finite-sample bottleneck can let the student retain informative components along the strong eigendirections in expectation while suppressing some of the teacher's noisy components along the weak ones. Our setting is thus conceptually related to feature filtering, but the distinction arises implicitly from the student's sample-size bottleneck rather than from different feature representations.

In contrast, Case II considers a genuine capacity-asymmetric W2SG setting, where NSN_S and NTN_T denote the numbers of student and teacher random features, respectively. We show that a necessary condition for W2SG is NS>NTN_S>N_T, so a student capacity advantage is required in our random-feature model. This advantage alone is not sufficient, however: the pseudo-labeled student sample size mm, relative to NSN_S and pp, plays a crucial role in determining whether the stronger student can outperform the teacher.


Question 1: Explicit asymptotic expressions characterizing the magnitude of the actual expected-risk improvement are in Appendix due to space constraints. For Case I, Theorem C.10 provides an explicit limiting expression for the expected risk improvement. The absolute gain is not universally bounded by 11 because the population risk is not normalized and the noise variance σ2\sigma^2 is unrestricted. For example, with α=1.5\alpha=1.5 and ϕT=ϕS=2\phi_T=\phi_S=2, we obtain

W2SG≈0.359σ2.\mathsf{W2SG}\approx 0.359\sigma^2.

For these parameters, the corresponding limiting teacher risk is approximately 1.594σ21.594\sigma^2. Therefore, this example corresponds to a reduction in the teacher's population risk. In particular, for σ2=4\sigma^2=4, the absolute improvement is approximately 1.4361.436, exceeding 11.

For Case II, Theorem D.9 characterizes the gain explicitly. Under the positive-signal-fraction proportional scaling, the absolute gain in the improving regimes is O(1/p)O(1/p), so it vanishes asymptotically. However, the relative improvement can remain nontrivial. For example, when 5% of the input coordinates contain signal, with NT=0.1pN_T=0.1p, m=0.25pm=0.25p, and NS=0.8pN_S=0.8p, our equation yields a normalized gain of approximately 0.1760.176. Accounting for the normalization factor a(ρm)=(1−ρm)/(1−ϑ)a(\rho_m)=(1-\rho_m)/(1-\vartheta), this corresponds to approximately a 13.9% reduction in the teacher's total population risk.

Thus, our analysis quantifies not only when W2SG occurs but also the magnitude of the resulting improvement. We will clarify these quantitative implications in the revised manuscript.


Question 2: The dependence of W2SG on the ratio NS/NTN_S/N_T is not globally monotonic. In particular, increasing this ratio by increasing NSN_S is different from increasing it by decreasing NTN_T. When keeping NTN_T and other parameters fixed, in the sample-bottleneck regime (m<NS)(m<N_S), increasing NSN_S increases W2SG, which eventually approaches a limiting value.

In contrast, in the feature-bottleneck regime (NS<m)(N_S<m), W2SG can be positive only over a bounded interval of student widths, with an interior maximum. When varying NTN_T while keeping NSN_S and mm fixed, the conditions for W2SG also exhibit a non-monotonic dependence on teacher width. In particular, positive W2SG becomes more favorable as NT/pN_T/p approaches the signal fraction ϑ\vartheta from either below or above. Moving away from this threshold makes the conditions for positive W2SG more restrictive.

Therefore, increasing NS/NTN_S/N_T does not necessarily lead to larger W2SG. Its effect depends on which width is varied, the signal dimension, and which Stage-II resource is the active bottleneck. The dependence of the absolute W2SG magnitude should also be distinguished from the conditions determining whether W2SG occurs, since the former additionally depends on the scale of the teacher's prediction error.

I thank the authors for their reply.

However, they do not address my biggest concern, which is the phrasing and intepretation of the result as "negative" for W2S (and the result perhaps more interesting). I think this is crucial.

We thank the reviewer for the clarification. We respectfully disagree, however, with interpreting Case I as a negative result for weak-to-strong generalization (W2SG). At the same time, we acknowledge that our previous presentation of Case I did not sufficiently clarify its intended role in the broader W2SG context, which may have contributed to this interpretation.

We agree that the original motivation of W2SG in Burns et al. involves an asymmetry between a student with larger model capacity and a teacher with smaller model capacity, where the student can leverage its prior knowledge to outperform its supervisor. Our Case I does not aim to reproduce this entire setting. Instead, we isolate an additional mechanism that can operate alongside the advantages of greater student capacity and prior knowledge: the amount of weak supervision can determine whether the student benefits from or inherits the teacher's errors. In particular, a finite pseudo-label budget can prevent the student from fully inheriting the teacher's estimation error, while still retaining useful signal.

This mechanism is also motivated by empirical observations at LLM scale. Busbridge et al. observe that W2SG can disappear as the amount of distillation data increases, leaving the initial improvement unexplained. Our Case I provides a theoretical explanation for how a finite pseudo-sample bottleneck can contribute to this behavior.

This is precisely why we begin with identical teacher and student hypothesis classes: introducing representational asymmetry from the beginning would confound the effect of pseudo-sample size with that of model capacity. We then introduce genuine capacity asymmetry in Case II and investigate how these two resources interact.

We believe this approach is consistent with existing theoretical work. For example, Moniri and Hassani, study a two-stage linear ridge-regression setting where teacher and student share the same linear hypothesis class and differ in their L2 regularization strengths. They show that appropriately regularizing the student can compensate for the teacher's under-regularization and lead to W2SG. Similarly, Geng et al. establish W2SG without a model-capacity advantage through a different mechanism involving initialization and early stopping. These works demonstrate the value of isolating individual mechanisms rather than requiring every theoretical setting to reproduce the original empirical setup.

To better communicate this perspective, we have revised both the Introduction and the standalone introduction of Case I. We now explicitly explain its connection to same-class self-distillation, clarify why we begin with identical teacher and student hypothesis classes, and emphasize that Case I is intended to isolate a complementary mechanism for student improvement rather than reproduce conventional capacity-asymmetric W2SG. We also make its relationship with Case II clearer: Case I isolates the role of pseudo-sample size, while Case II introduces a genuine student capacity advantage and investigates how pseudo-sample size interacts with student width. We believe this distinction is important because it allows us to understand the role of weak-supervision quantity independently before studying how it operates alongside the more conventional sources of student advantage.

Comments on the paper

1 comment
Area_Chair_2s3uOpenReview ↗

Discussion phase coming to an end

Dear reviewer wxeL,

As we are approaching the deadline of the discussion phase, please read the authors’ responses and share your thoughts in the discussion as soon as possible. In particular, please clarify whether the responses have addressed your main concerns. Please also update your review and scores if appropriate.

Your timely input will help us reach a well-informed decision. Thank you for your time and effort!

Best regards, Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.