acceptodds
Under review as a conference paper at ICLR 2027

Not All Teachers Are Created Equal: Confidence-Aware Prediction-Powered Learning from Heterogeneous Teachers

Abstract

Semi-supervised learning often uses teacher predictions to exploit unlabeled data, but unreliable predictions can introduce harmful supervision. This problem is especially acute when teachers are reliable only on particular inputs, classes, or domains. Prediction-powered inference (PPI) is a general statistical framework that combines predictions with labeled-data corrections to construct unbiased estimators. Building on this principle, we introduce confidence-aware prediction-powered correction (C-PP), which adapts to instance-dependent teacher reliability, and covariance-aware multi-teacher correction (CM-PP), which combines complementary teachers while accounting for their dependence. We establish finite-sample variance bounds and convergence guarantees, and develop online weight updates that avoid explicit variance estimation. Across image and tabular classification, and semantic segmentation, C-PP and CM-PP improve upon confidence-agnostic prediction-powered learning and established semi-supervised baselines under teacher-reliability shifts, partial coverage, and heterogeneous specialization.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Semi-Supervised Learning, Pseudo-Labeling, Online Learning, Prediction-Powered Learning, Debiasing, Confidence-Aware Learning, Multi-Teacher Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_oMeuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates semi-supervised learning with heterogeneous teachers whose prediction reliability varies across instances, classes, or domains. Building upon prediction-powered inference (PPI), the authors propose two methods: Confidence-Aware Prediction-Powered Correction (C-PP) and Covariance-Aware Multi-Teacher Prediction-Powered Correction (CM-PP). C-PP incorporates instance-dependent confidence scores symmetrically into the labeled and unlabeled teacher correction terms, preserving the unbiasedness of stochastic gradient estimates while potentially reducing their variance. CM-PP extends this formulation to multiple teachers by introducing teacher-specific confidence functions and aggregation weights that account for dependencies among teacher corrections. The paper establishes finite-sample variance bounds, derives covariance-aware aggregation rules, and provides convergence guarantees for online weight adaptation using AdaGrad and exponentiated-gradient optimization. The authors evaluate their methods on image classification, tabular classification, and semantic segmentation benchmarks, including MNIST, CIFAR-10/100, SVHN, Covertype, Tiny-ImageNet, and PASCAL VOC. The experiments demonstrate encouraging improvements over confidence-agnostic prediction-powered methods, particularly under heterogeneous teacher reliability, although the gains depend substantially on confidence quality and the experimental setting.

Critical strengths
  1. A key strength is the integration of instance-dependent teacher confidence into an unbiased prediction-powered correction framework. Unlike conventional pseudo-label weighting, the method applies the same confidence modulation to both labeled and unlabeled correction terms, providing a principled way to use locally reliable teacher predictions without introducing systematic gradient bias. This is particularly valuable when teachers have partial class coverage or domain-specific expertise. The supervised fallback also offers a useful mechanism for reducing dependence on unreliable teacher information.

  2. The extension to multiple heterogeneous teachers is another strength. The covariance-based variance analysis goes beyond treating teacher corrections as independent and provides a principled motivation for exploiting complementary information while accounting for redundancy. The empirical evaluation covers several data modalities and teacher-reliability regimes. Notably, on CIFAR-100, CM-PP achieves 19.35% Top-1 accuracy compared with 15.88% for MM-PP. On Tiny-ImageNet, binary-entropy C-PP achieves 39.73% Top-1 accuracy compared with 29.05% for PP-SSL. These results provide meaningful evidence that confidence-aware correction can improve prediction-powered learning in selected challenging settings.

Critical weaknesses
  1. My primary concern is the degree of novelty of C-PP relative to prior context-aware doubly-robust learning methods. In particular, Ruah et al. (2025), Context-Aware Doubly-Robust Semi-Supervised Learning, already formulate an unbiased prediction-powered correction with context-dependent interpolation weights. Their method assigns different correction weights to different data contexts and estimates these weights during training. The relationship appears especially close mathematically: when the confidence function in C-PP is chosen to be piecewise constant over observable contexts, the resulting correction can be represented by the context-specific weighting formulation of Ruah et al. Conversely, the proposed C-PP formulation can represent that context-dependent correction through an appropriate confidence function. The related-work section characterizes Ruah et al. as relying on fixed, tuned weights, whereas their formulation includes iteration-dependent weight estimation. This distinction should be corrected or justified. While the proposed continuous confidence modeling, multi-teacher extension, and convergence analysis may constitute additional contributions, the paper currently does not establish sufficiently clearly which parts are genuinely new relative to this closely related prior work. A direct mathematical comparison and an experimental baseline using context-aware doubly-robust learning are particularly important.

  2. A second major concern is that the empirical evidence does not yet sufficiently establish the robustness of the proposed mechanisms under practical confidence estimation. Several partial-teacher experiments, including CIFAR-10, use oracle support confidence derived from ground-truth labels. Although these experiments are useful for isolating the proposed mechanism, they do not demonstrate that the same gains can be achieved when teacher applicability must be inferred from observable information. More importantly, the practical confidence experiments reveal substantial sensitivity. In Table 15, C-PP with cached raw-margin confidence achieves 36.97% accuracy on SVHN, compared with 44.17% for PP-SSL. Augmentation-based C-PP achieves 47.99% under equal epochs, but only 43.87% under a matched training-time budget, with substantial variability. These results suggest that confidence quality and computation costs can largely determine whether the method is beneficial. In addition, the empirical contribution of covariance awareness is not sufficiently isolated. Although CM-PP is theoretically motivated by cross-teacher covariance, the reported comparisons do not clearly separate the benefit of modeling off-diagonal covariance from that of simply using teacher-specific weights and confidence scores. For example, under elevated corruption on PASCAL VOC, margin CM-PP achieves 56.21% mIoU compared with 56.09% for MM-PP, while the latter has substantial run-to-run variation. A matched comparison between full covariance-aware aggregation and a diagonal or covariance-agnostic aggregation method, using identical confidence functions, would significantly strengthen the central claim.

Questions for authors
  1. Could the authors provide an explicit mathematical comparison between C-PP and the context-aware doubly-robust formulation of Ruah et al. (2025), particularly their context-dependent correction weights? Under what assumptions is C-PP not reducible to a reparameterization of that method? Could the authors also clarify why the related-work section describes its weight tuning as fixed and provide a matched experimental comparison?

  2. How reliably do the proposed methods improve performance when oracle confidence is unavailable and confidence must be estimated from limited labeled data or teacher outputs? In particular, can the authors explain the SVHN failures in Table 15 and provide practical-confidence results for the CIFAR-10 partial-coverage setting, including confidence-quality diagnostics and time-matched comparisons?

  3. Can the authors isolate the contribution of cross-teacher covariance by comparing CM-PP against an aggregation approach that uses the same teacher-specific confidence scores and optimized marginal-variance weights but ignores off-diagonal covariance? Additionally, in the PASCAL VOC experiments, how is class-conditional teacher corruption generated for nominally unlabeled examples when the corruption procedure depends on ground-truth segmentation masks? Is this oracle information used exclusively to construct the synthetic benchmark?

  4. Could the authors clarify the applicability of conditional unbiasedness and convergence guarantees when training repeatedly on finite labeled and unlabeled datasets rather than using fresh independent samples at each iteration? Also, Appendix F.3 appears to factorize linear-MSE gradient variances as products of residual variances and input-covariance traces. Such factorizations generally require additional assumptions on the relationship between inputs and residuals, which are not explicitly stated. Could the authors provide the missing assumptions or corrected derivations and clarify whether the main theoretical results are affected?

5 comments

W1 + Q1: Relationship to Ruah et al.

Thank you for identifying this connection. We corrected the related-work description. We also added an explicit mathematical comparison and two matched Covertype experiments.

Mathematical relationship. We agree that the single-teacher correction forms overlap. CDR assigns an effective correction coefficient λt,C(x)CDR\lambda^{\mathrm{CDR}}_{t,C(x)} to an example in context C(x)C(x), while C-PP assigns λtct(x)\lambda_t c_t(x). The corrections coincide whenever

λtct(x)=λt,C(x)CDR.\lambda_t c_t(x) = \lambda^{\mathrm{CDR}}_{t,C(x)}.

Consequently, context-wise correction is a shared special case, and we do not claim novelty for that correction itself.

Our novelties compared to CDR concern modeling, analysis, and optimization:

  • Flexible confidence and multiple teachers. CDR estimates one correction coefficient per discrete context. C-PP separates confidence from aggregation, allowing general binary or continuous reliability signals that can vary within a context, without requiring a separately estimated gradient-moment ratio for each confidence value. CM-PP further extends this formulation to multiple heterogeneous teachers, with analysis accounting for dependence between their corrections.
  • Finite-sample variance bounds. CDR derives its tuning rule through asymptotic variance minimization using Stratified PPI. Our finite-sample gradient-variance bounds explicitly characterize the effects of confidence quality, coverage, labeled/unlabeled sample sizes, and cross-teacher dependence.
  • Finite-time convergence guarantees. Beyond CDR’s analysis of the tuning rule, which does not establish a finite-iteration convergence rate for adaptive training, we provide finite-time guarantees for student optimization with online aggregation-weight adaptation, relative to the best fixed aggregation weights.
  • Weight adaptation using both labeled and unlabeled gradients. CDR estimates the gradient covariances and variances for its context weights from labeled examples; unlabeled data enter that estimation formula through the context counts NcN_c. Our online update instead optimizes the squared norm of the complete prediction-powered gradient, directly incorporating observed gradients from both labeled and unlabeled batches without explicitly estimating covariance or variance ratios. In our matched experiments, this approach also has lower runtime than the implemented CDR baseline.

Matched experimental comparisons. We added two Covertype experiments using the same teacher, student, observable contexts, data splits, and validation-based hyperparameter selection (Appendix I.2.3). We compared supervised learning, PP-SSL, CDR, C-PP with context-based confidence, and C-PP with oracle confidence. Across labeled budgets of 50–1,000 at a fixed labeled-to-unlabeled ratio, context-based C-PP improves mean test accuracy over our CDR adaptation over five paired seeds. A second experiment fixes the labeled budget at 320 and varies unlabeled availability: C-PP’s advantage is most evident in the unlabeled-dominated settings and becomes smaller or reverses at some larger labeled-to-unlabeled ratios. Across the timed settings, CDR takes approximately ×1.25\times1.25 and ×1.35\times1.35 C-PP’s runtime in the two experiments, respectively. The appendix provides the full experiment settings and results.

These comparisons illustrate the practical value of separating context-based reliability from the global aggregation weight. A context-based confidence inspired by CDR together with C-PP’s online optimization achieves comparable or higher mean accuracy across most tested settings, with shorter runtime in both experiments.

W2 + Q2: Confidence sensitivity and oracle confidence.

We expanded the CIFAR-10 evaluation to four practical confidence signals, four partial teachers, five labeled-to-unlabeled ratios, and four student seeds. These experiments demonstrate improvements without oracle confidence, while the expanded SVHN analysis identifies how confidence calibration affects the gains.

Role of oracle confidence. Oracle confidence serves as a reference in controlled experiments that isolate teacher applicability. It is not required by C-PP or CM-PP. Our practical evaluations use confidence computed from observable inputs, teacher predictions, or representations.

Expanded SVHN analysis. The original SVHN experiments did not use predicted-label-specific thresholds. We increased the number of student seeds and extended Appendices I.2.5–I.2.6 to examine this sensitivity. The expanded diagnostics reveal differences in teacher reliability across predicted labels, motivating thresholds calibrated separately for each teacher-predicted class. This rule uses the predicted class at application time, without requiring the example’s true label. The additional seeds improve the reliability of the comparison, while the revised calibration addresses class-dependent confidence quality. Under this calibrated construction, all tested practical confidence families outperform PP-SSL across the reported operating points, including the least selective one. Augmentation stability is evaluated under both equal-epoch and matched-time budgets.

Expanded CIFAR-10 evaluation. We expanded the partial-coverage experiment to compare PP-SSL against C-PP using normalized margin, inverse entropy, feature distance, and augmentation stability. With 400 labeled student-training examples. Practical confidence improves mean performance over PP-SSL most clearly for the odd teacher. The 0-4 teacher remains challenging, with several signals performing close to supervised learning.

We use binary confidence, with thresholds calibrated to a target accepted accuracy. Different confidence signals can reach the same target while accepting different numbers of examples, changing how much useful teacher supervision is available. Full coverage and accepted-accuracy results are provided in the revised CIFAR-10 experimental details in Appendix I.2.4.

Q3: Use of ground-truth masks in the PASCAL VOC benchmark.

We use fixed, pretrained segmentation models as teachers. Ground-truth masks of nominally unlabeled examples are used exclusively to construct the synthetic benchmark before student training. For each teacher, the masks identify pixels belonging to its predefined weak-class set; corruption is applied to those pixels to create heterogeneous teacher reliability. The corruption realizations are fixed and shared across compared methods.

During student training, masks of unlabeled examples are unavailable to the learning procedure. Confidence scores are computed from teacher predictions after corruption, and ground-truth supervision comes only from the designated labeled set. We have made this offline benchmark construction explicit in Appendix I.3.2.

W2+Q3: The empirical contribution of covariance awareness.

Thank you for suggesting this comparison. We added a matched ablation that changes only the aggregation objective, keeping the teachers, confidence functions, student, data splits, and training protocol identical.

Writing the student gradient as g+∑jλjdjg + \sum_j \lambda_j d_j, where gg is the supervised gradient and djd_j is teacher jj’s unlabeled-minus-labeled correction, we compare

=\left\|g+\sum_j\lambda_jd_j\right\|^2, \qquad h_{\mathrm{diag}}(\lambda) =h_{\mathrm{full}}(\lambda) -\sum_{j\ne k}\lambda_j\lambda_k\langle d_j,d_k\rangle.$$ Both variants optimize teacher-specific weights online over the same simplex. The diagonal objective retains marginal correction magnitudes and supervised–correction interactions, isolating the effect of cross-teacher terms. We run this comparison for MM-PP and CM-PP using identical non-oracle margin confidence within each matched pair. On Covertype, we evaluate a specialist-only panel and a panel additionally containing an overlapping generalist. Full aggregation achieves higher mean test accuracy in the latter, while differences are small in the specialist-only panel. This is consistent with the expected benefit of accounting for cross-teacher interactions when corrections exhibit greater redundancy. On MNIST, we evaluate partial-label teachers with complementary, overlapping, and nested class supports, similar to Appendix C. With nested teachers trained on digits $\{7–9\}$, $\{5–9\}$, and $\{3–9\}$, full CM-PP using non-oracle margin confidence improves accuracy by 4.00 and 4.39 percentage points at labeled budgets of 40 and 200, respectively, with positive differences on all four paired seeds. Full aggregation also improves accuracy with complementary even- and odd-digit teachers at 40 labels, while the methods perform similarly at 200 labels. The benefit is configuration-dependent: diagonal aggregation performs better in two MNIST panels at 40 labels, but these gaps narrow or reverse at 200 labels, suggesting sensitivity to the aggregation objective when labeled data are scarce. Appendix I.3.4 reports the complete results using validation-selected checkpoints, together with correction-correlation diagnostics and comparisons across labeled-to-unlabeled ratios. Together, these matched expirements show that confidence and teacher-specific weights do not fully capture how teachers should be combined: accounting for interactions between their corrections can yield substantial additional gains, particularly when their expertise overlaps. ## Q4: Conditional unbiasedness under repeated data reuse. The question of extending single-pass theoretical guarantees to practical multipass training is not concrete to our paper but rather a fundamental and important question in the field of stochastic optimization. Therefore addressing it is an important research direction to the field, and is much beyond the scope of our work. ## Q4: Assumptions underlying the linear-MSE specialization. Thank you for identifying this omission. The simplified linear-MSE comparison assumes independence between features $x$ and residuals $\epsilon,\epsilon_f$, as in PP-SSL Appendix C; we omitted this assumption. We have revised Appendix F.3 to state the required conditions explicitly. These corrections affect only the illustrative linear-MSE specialization; the main variance bounds and convergence guarantees do not rely on these factorizations and remain unchanged.
Reviewer_vTCqOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies prediction-powered semi-supervised learning with heterogeneous, locally reliable teachers. C-PP incorporates instance-dependent confidence into an unbiased gradient estimator, and CM-PP combines multiple teacher corrections while accounting for their dependence. The paper provides finite-sample variance bounds and convergence guarantees, together with online updates of the aggregation weights. Experiments on image and tabular classification and semantic segmentation demonstrate improvements over prediction-powered and other learning baselines across several teacher-reliability settings.

Critical strengths
  1. The paper addresses an important and practical learning scenario in which teachers have heterogeneous expertise and are reliable only on parts of the student distribution.
  2. The method is well motivated, constructing an unbiased gradient estimator that incorporates instance-dependent confidence and teacher reliability. The experiments demonstrate its effectiveness relative to the reported baselines across several tasks.
  3. The paper provides a substantial theoretical treatment, including finite-sample variance bounds for the confidence-aware estimators and convergence guarantees for online aggregation relative to the best fixed weights.
Critical weaknesses
  1. Confidence functions perform differently across settings, and choosing their form and thresholds still requires setting-specific validation and tuning. Appendix H provides useful guidance, but a more effective selection procedure or a confidence design that reduces this dependence would strengthen the method's practicality, given the central role of instance-dependent confidence.
  2. The rationale for omitting MixMatch and FixMatch from the Tiny-ImageNet comparison in Table 1 is unclear. Although they are evaluated on MNIST in the appendix, their inclusion under matched conditions would make the main SSL comparison more complete.
  3. Presentation issues: the fonts in Figures 1 and 2 are too small to read comfortably, and Table 10 extends beyond the right text margin.
  4. Minor clarification: the relationship between this setting and multi-source domain adaptation deserves discussion. It would be interesting to understand whether the method could be applied to DA/MSDA, particularly the role of labeled target data and the shared-marginal assumption.
Questions for authors

Please refer to the weaknesses above.

2 comments

Thank you for the constructive review and for the useful suggestions on confidence selection, baselines, and the connection to domain adaptation.

W1: Confidence selection still requires setting-specific validation and tuning.

We acknowledge the fact that confidence model selection is an important part in our method. There is no universally best confidence rule: its useful form depends on the teacher’s reliability structure and how that structure relates to the student’s distribution. Appendix H.4 and Table 7 give guidance for common cases. Section 8 presents the flexibility of the confidence model and explains how several confidence options can use cross validation; several confidence models may be incorporated as separate predictive sources and jointly aggregated.

Empirical validation. In the revised version, we extended the confidence analysis on SVHN to test this guidance in a real setting (Appendices I.2.5–I.2.6). We compare several confidence families, examine the coverage to accepted-error trade-off as the threshold changes, and show how the structure of the teacher’s errors informs the choice. In the student experiments, every tested confidence family remains above confidence-agnostic PP-SSL across the reported operating points, including the least selective one. This suggests that, in this setting, improvement does not depend on finding one uniquely optimal confidence family; it does not imply that arbitrary confidence signals will help.

Confidence robustness. While confidence determines which samples enter each teacher’s correction; the aggregation weight controls the overall contribution of that teacher–confidence pair. The online update adapts these weights based on stochastic gradients, whose expected squared norm tracks variance, and CM-PP also accounts for dependence between teacher corrections. Since the supervised-only solution is a feasible fallback, the update can reduce reliance on an unhelpful correction in favor of trusted labeled data - this serves as a safeguard at the aggregation level.

W2: MixMatch and FixMatch comparisons.

We agree that the rationale for our baseline selection could have been clearer. We designed the experiments to cover complementary settings and method families rather than repeat every baseline on every dataset. MNIST isolates partial teacher coverage and compares C-PP with FixMatch and MixMatch. Tiny-ImageNet tests a larger, noisier teacher-reliability shift and includes confidence-aware baselines (UDA and FreeMatch) as well as confidence-agnostic baselines (soft knowledge distillation and PP-SSL).

We included FreeMatch as a representative of the broader confidence-thresholded, FixMatch-style SSL family. Like FixMatch, it uses confidence to select pseudo-labels, but it uses self-adaptive thresholds. This comparison helps assess whether C-PP’s gains could be explained simply by confidence-based pseudo-labeling: unlike FreeMatch, C-PP uses confidence within a labeled-data-corrected, unbiased gradient estimator.

To broaden the Tiny-ImageNet comparison, we added FixMatch and MixMatch using the same data split, student architecture, training budget, and seeds. Indeed, as presented in table 1, FixMatch and MixMatch performed similarly to FreeMatch.

W3: Presentation and formatting.

Thank you for pointing out these presentation issues. We have enlarged the text in Figures 1 and 2 and reformatted Table 10 to fit within the text margins.

W4: Relation to multi-source domain adaptation.

This sounds like an interesting application for CM-PP. Our setting is related to semi-supervised multi-source adaptation with source-trained predictors and a small labeled target set. The teachers are evaluated on target inputs, and the student is trained for the target objective. Our method does not require source-domain samples or explicit feature alignment; it uses teacher predictions as corrections estimated on labeled target data.

Domain shift. The shared-marginal assumption concerns the labeled and unlabeled target samples: both have input marginal PXP_X. It does not require the teachers’ training distributions to match the target distribution. With the teachers and confidence rule fixed before each batch, teacher–target mismatch does not bias the correction, but it can reduce the variance gains. Appendix A.3 captures this transfer degradation with εDP\varepsilon_{DP}, which bounds how the teacher’s confidence-conditioned error changes between its native distribution and the target distribution and enters the correction-variance bound.

Picking the confidence. Confidence can encode source relevance through a domain router, representation similarity, or a suitably scaled density-ratio score, as discussed in Appendix H.4. CM-PP then aggregates the resulting teacher corrections while accounting for their dependence. Our experiments include related forms of teacher heterogeneity and transfer mismatch, but we have not evaluated on standard MSDA benchmarks. At first glance, for DA, calibrated margin, density-ratio signal, or the two combined are promising candidates for confidence estimation and warrant further investigation. We added a brief discussion positioning our setting relative to MSDA in Appendix H.4.

Labeled target data is essential to estimate the correction. Without it, the method does not provide the stated unbiased target-gradient guarantee. If only source-labeled data are available and their input distribution differs from the target, the current estimator is generally biased for the target objective. Addressing this would require additional assumptions and a correction such as importance weighting under covariate shift with support overlap.

Reviewer_WbjKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies semi-supervised learning with teacher models that are reliable only on some inputs, classes or domains. It builds on prediction-powered learning, which uses a teacher's pseudo-labels on unlabeled data together with a correction computed on labeled data, so that training stays unbiased however accurate the teacher is.

The paper has two ideas:

  • C-PP weights each teacher prediction by a confidence score. The student then relies on the teacher where it is trustworthy and falls back to the labeled data elsewhere, while training stays unbiased.
  • CM-PP extends this to several heterogeneous teachers. Their contributions are weighted according to how reliable each teacher is and how much their errors overlap, and supervised-only training remains an option if no teacher helps.
Critical strengths
  • The proposed method is theoretically driven.
  • The authors tackles an important problem.
  • The theory is backed by empirical results.
Critical weaknesses
  • My primary concern is about the validity of the empirical results. In Table 2, one of the main results of the paper, MM-PP gets better as corruption increases ( MM-PP: 52.53 to 56.09 and Margin: CM-PP 55.94 to 56.21). How is this possible? Also, what is "moderate" and "elevated"? In addition, the fact that 8 of the 9 methods have identical standard deviations under Moderate and Elevated corruption makes the results some what suspicious.
  • A large portion of the theory seems to be building on the prior work of Shoham et al. (NeurIPS 2025), making the proposed method somewhat incremental in terms of novelty.
Questions for authors

Could you double check the results obtained in Table 2?

1 comment

Thank you for the careful reading of our paper.

W1: Table 2, identical STDs, what “moderate” and “elevated” mean.

Thank you for flagging this. The reported means were correct; by mistake, the standard-deviation values for elevated corruption were copied from the moderate-corruption column. We recomputed the standard deviations from the original logs and corrected Table 2. As mentioned in (Appendix I.3.2), “Moderate” and “elevated” refer to Gaussian noise with σ=0.1\sigma=0.1 and σ=0.15\sigma=0.15, respectively, applied to teacher inputs at pixels belonging to each teacher’s predefined weak-class set. For clarity we added these values in Table 2.

W1: Table 2, MM-PP rising with corruption.

MM-PP’s mean mIoU is higher under elevated corruption in this experiment, but this does not imply that stronger corruption is generally beneficial. Our bounds characterize gradient variance through individual teacher corrections and their cross-teacher covariances; they do not imply monotonic test mIoU as corruption increases.

The experiment deliberately stresses aggregation with only 92 labeled images overall and batches of four labeled and four unlabeled images. Each pixel receives one of 21 class labels, including background. Because pixels within an image are correlated, the large number of pixel predictions does not provide an equivalent number of independent examples. Thus, we believe the reason for this non-intuitive behaviour is the extreme data-scarcity and large class-number in this specific experiment, which lead to undertuned teacher-weights.

CM-PP adds pixel-level confidence to control unreliable corrections when global teacher weights are imperfect. With identical data budgets, it improves mean mIoU over MM-PP under moderate corruption and remains competitive under elevated corruption. Differences among confidence-aware variants are small relative to run-to-run variability. This,also demonstrates that confidence serves as strong way to induce prior knowledges and therefore mitigate effects that are related to teacher-weight undertaking in extreme scarce-data scenarios.

W2: Novelty relative to Shoham et al.

While we acknowledge that our convergence analysis is inspired by the approach of Shoham et al., our framework makes three distinct contributions.

First, the algorithm: we introduce instance-dependent confidence into the prediction-powered correction and combine multiple heterogeneous teachers through online aggregation over the simplex, including a supervised fallback.

Second, the bounds: PP-SSL is recovered as the special case K=1K=1, c≡1c\equiv1, and our variance bound improves whenever confidence reduces the effective teacher-error term, s<sPPs<s_{\mathrm{PP}} (Lemma 4.2). The analysis separates the effects of instance-dependent confidence and labeled-data rectification, and extends to multiple teachers by explicitly characterizing dependence between their corrections (Lemma 5.2). Individual teacher-error bounds alone do not capture these interactions. We then connect the resulting variance bounds to online aggregation guarantees, using the simplex geometry to control the dependence on the number of teachers, yielding an adaptation term proportional to log⁡(K+1)/T\sqrt{\log(K+1)}/T while retaining the leading convergence rate of the best fixed aggregation weights. A direct extension of the PP-SSL AdaGrad analysis using Euclidean gradient-norm bounds would instead introduce polynomial dependence on KK.

Third, usability: our analysis expresses the potential gains through interpretable quantities - teacher error on confident samples, confidence coverage, student error elsewhere, and dependence across teachers. This translates gradient-variance conditions into properties of the teacher, student, and data that practitioners can diagnose, helping explain when partially reliable teachers are useful and how to construct confidence signals for them.

Comments on the paper

1 comment

Dear Reviewers,

Thank you again for your constructive feedback, which has helped us strengthen the paper. With the discussion period ending soon, we would appreciate hearing whether our responses and revisions have addressed your concerns. If you have any remaining questions or concerns, we would be happy to address them.

Thank you for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.