acceptodds
Under review as a conference paper at ICLR 2027

Pseudo-Label Reliability Collapse: Identification, Decomposition, and Calibration for Continual Semi-Supervised Learning

Abstract

Continual semi-supervised learning (CSSL) faces a critical yet underexplored failure mode beyond catastrophic forgetting, pseudo-label reliability collapse, wherein erroneous pseudo-labels generated under irreversible data access constraints progressively corrupt feature representations across the task stream, precluding the iterative self-correction that underpins standard semi-supervised learning. Existing approaches from both communities are structurally insufficient to address this collapse, as confidence-based pseudo-label selection cannot detect over-confident but semantically inconsistent predictions. To this end, we propose a Reliability-aware Calibration Framework (RCF) grounded in a principled decomposition of pseudo-label reliability into predictive and semantic dimensions. Each unlabeled sample is embedded in a joint predictive-semantic reliability space, where predictive reliability captures augmentation-consistent prediction stability and semantic reliability measures prototype-supported class agreement in the frozen feature space. To estimate the true reliability state of unlabeled samples without ground-truth supervision, we model the resulting reliability representations with a two-component Gaussian Mixture Model whose parameters are inferred via Expectation-Maximization, yielding a calibrated, distribution-relative reliability score that is robust to task-wise distribution shifts. Building on this score, we introduce an adaptive, risk-constrained pseudo-label selection strategy with an asymmetric exponential moving average update rule, which rapidly exploits genuine reliability improvements while conservatively suppressing potential noise propagation. Extensive experiments demonstrate that RCF consistently and significantly outperforms state-of-the-art CSSL methods.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Continual Learning, Two-dimensional Pseudo-label Selection

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ZH5sOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies pseudo-label reliability in continual semi-supervised learning (CSSL), with a particular focus on the non-revisitable nature of the task stream. The proposed Reliability-aware Calibration Framework (RCF) represents each unlabeled sample using two reliability signals. The predictive signal combines prediction confidence with consistency across weakly augmented views, while the semantic signal measures the margin between the predicted class prototype and the strongest competing prototype in a frozen ViT feature space. Then a GMM is fitted to this two-dimensional reliability space. The posterior of the component with the larger predictive mean is treated as a reliability score. The method performs particularly well in the low-label ImageNet-R and ImageNet-A settings, and the paper also includes component ablations.

Critical strengths
  1. This paper focus on the non-revisitable nature of the task stream. This is meaningful and practical problem and cause a lot of wrong pseudo labels.
  2. The experiments verify that the method performs particularly well in the low-label ImageNet-R and ImageNet-A settings.
Critical weaknesses
  1. The theoretical argument for pseudo-label reliability collapse is not fully convincing. Proposition B.3 relies on a gradient-norm comparison that does not generally hold when historical-task and current-task gradients conflict. The current proof seems insufficient to establish reliability collapse as an inherent consequence of the non-revisitable setting.
  2. Can RCF also improve standard semi-supervised learning when the non-revisitable constraint is removed? Since the proposed predictive-semantic reliability estimator is not obviously specific to continual learning, an experiment on standard SSL would help clarify which gains come from general pseudo-label selection and which are specific to the CSSL setting.
  3. It remains unclear how general the proposed selector is across different LoRA-based continual learning methods. Could the authors evaluate RCF on another representative LoRA-based method, such as Keep-LoRA, to separate the benefit of reliability-aware pseudo-label selection from that of the underlying continual-learning architecture? In fact, the methods like Keep-LoRA in table 1 are largely worse than RCF. So, can RCF help Keep-LoRA improve performance?
  4. The experimental scope is still fairly narrow. All experiments are conducted on image classification benchmarks under a class-incremental setting, and the main model always uses the same pretrained ViT-B/16 backbone.
Questions for authors

See in weakness.

7 comments
AuthorsOpenReview ↗

Response to Reviewer ZH5s (Part 1)

w1. Can Proposition B.3 establish reliability collapse under non-revisitable learning when task gradients may conflict?

A1. We have revised the motivation and theoretical analysis in both the main text and the appendix. The revised argument establishes how pseudo-label errors affect within-task optimization, with explicit assumptions and bounds.

(1) Correcting the scope of the argument.

The original gradient-norm comparison does not hold in general: opposing historical and current-task gradients can reduce the norm of their sum. It therefore cannot establish reliability collapse as an inherent consequence of non-revisitability.

Our implemented architecture freezes the backbone and previous experts while updating only the active expert. We have accordingly refocused the motivation on within-task supervision: pseudo-labels guide the active expert's updates, and the updated expert generates predictions for subsequent epochs. This correction is reflected in the Abstract, Section 1, Section 3.2, and Appendix B.1.

(2) Quantifying pseudo-label-induced gradient perturbation.

The revised Proposition B.3 compares the pseudo-label objective F^(θ)\widehat F(\theta) with a true-label reference objective F(θ)F(\theta) on the same selected batch. The reference is used only for analysis. The current update state, selected samples, augmentation views, and unsupervised-loss weight λ\lambda are fixed, isolating the effect of label correctness.

For softmax cross-entropy, the difference between the two logit gradients is zero for a correct pseudo-label and has Euclidean norm 2\sqrt{2} for an incorrect one. If the logit Jacobians have operator norm bounded by MM, then

\qquad \|b(\theta)\|_2\leq\sqrt{2}\lambda M\epsilon,$$ where $\epsilon$ is the selected pseudo-label error rate. This bounds the supervision-induced gradient perturbation linearly in $\epsilon$, without comparing historical and current-task gradients. **(3) Connecting the perturbation to local optimization.** Assume that $F$ is $L$-smooth along the update segment. For the local gradient step $\theta^+=\theta-s\nabla\widehat F(\theta)$ with $0<s\leq1/L$, smoothness gives $$F(\theta^+)\leq F(\theta) -\frac{s}{2}\|\nabla F(\theta)\|_2^2 +\frac{s}{2}\|b(\theta)\|_2^2 \leq F(\theta) -\frac{s}{2}\|\nabla F(\theta)\|_2^2 +s\lambda^2M^2\epsilon^2.$$ The reference-gradient term contributes descent, while pseudo-label errors introduce a quadratic penalty in the bound. Reducing $\epsilon$ tightens both the gradient-perturbation bound and the local optimization bound; when $\epsilon=0$, the penalty vanishes. The revised proposition thus provides a quantitative rationale for accurate pseudo-label selection during active-expert training. It replaces the unsupported non-recoverability claim with an explicit connection between supervision errors and local optimization under the stated conditions.
AuthorsOpenReview ↗

Response to Reviewer ZH5s (Part 2)

w2. Can RCF improve standard SSL, and which gains are specific to CSSL?

A2. RCF also improves standard SSL. To evaluate its pseudo-labeling mechanism outside continual learning, we conduct joint training on ImageNet-R with 1% labels and seed 1993. All 200 classes are learned simultaneously using a frozen ViT-B/16 backbone and a single LoRA expert, with continuous access to the training pool. This setting removes task sequencing, expert expansion, and task-based routing. The protocol and results are reported in Appendix D.8.

The confidence-based baseline and RCF share the backbone initialization, labeled subset, optimizer, augmentation settings, and number of training updates. Both use two labeled-only warm-up epochs followed by eight semi-supervised epochs, with 150 updates per epoch. Their final checkpoints are evaluated on the same test set, allowing the benefit of RCF's pseudo-labeling pipeline to be assessed within the same single-expert learning framework.

MethodTest top-1 accuracy
Confidence-based baseline22.30%
RCF29.41%
Improvement+7.12 pp

The improvement is computed from unrounded accuracies. We also evaluate the same trained checkpoints after excluding test images with byte-identical counterparts in the training partition. The resulting 7.09-percentage-point gain confirms that the improvement persists after removing those duplicates.

The standard-SSL result demonstrates that RCF provides a general benefit through its predictive–semantic pseudo-labeling mechanism, without requiring task isolation or expert expansion. The CSSL experiments further demonstrate its effectiveness when training proceeds through sequential task distributions and historical data become inaccessible.

Together, these results highlight RCF's central advantage: it combines predictive consistency with semantic support to guide unlabeled supervision beyond confidence-only selection. Its reliability estimator operates on the available training pool, while adaptive selection accommodates the evolving model. This enables the same framework to improve learning in both joint standard SSL and continual task streams, supporting its use across different training protocols and expert architectures.

AuthorsOpenReview ↗

Response to Reviewer ZH5s (Part 3)

w3. Can RCF improve Keep-LoRA and other LoRA-based continual learners while keeping their underlying architectures unchanged?

A3. We evaluate RCF as a plug-in pseudo-label selector within Keep-LoRA and CL-LoRA. Each paired comparison retains the underlying architecture, data split, training settings, and evaluation protocol. This allows the benefit of reliability-aware selection to be assessed within the same learning framework, separating it from improvements obtained by changing the continual-learning architecture.

Results on ImageNet-R with 1% labels are summarized below. Values report average incremental accuracy, and improvements are expressed in percentage points.

ArchitectureConfidence-based selectorRCF selectorImprovement
CL-LoRA48.10%61.44%+13.34 pp
Keep-LoRA7.58%15.11%+7.53 pp

The Keep-LoRA(Keep-LoRA uses feature-based routing without ground-truth task IDs.) comparison directly addresses the reviewer's question: its AIA nearly doubles after integrating RCF, while its underlying architecture is retained. Thus, Keep-LoRA's lower confidence-based performance does not prevent it from benefiting substantially from improved pseudo-label selection. The CL-LoRA comparison(The CL-LoRA selector comparison uses ground-truth task IDs for task-restricted classification.) provides a second within-architecture validation, showing that the benefit extends across both evaluated LoRA-based learners. These transfer evaluations are reported in Appendix D.3.

RCF complements these architectures by improving how unlabeled samples are selected for supervision. It combines prediction consistency with prototype-supported semantic agreement, allowing selection to consider evidence beyond the model's confidence alone. Its distribution modeling and temporal updates then adapt the selection process as the learner evolves.

Together, these results demonstrate that RCF's benefit extends beyond the architecture used in our main experiments. It can strengthen existing LoRA-based continual learners through their pseudo-labeling pipeline while preserving their adaptation mechanisms. This makes reliability-aware selection a transferable component for improving learning under scarce labeled supervision.

Reviewer_G9dkOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3(+1)
Soundness
2(+1)
Read review
Paper summary

This paper investigates pseudo-label reliability in Continual Semi-Supervised Learning (CSSL), where noisy pseudo-labels may degrade learning under restricted access to historical task data. The authors propose a Reliability-aware Calibration Framework (RCF) that jointly characterizes predictive reliability through augmentation-consistent predictions and semantic reliability through prototype agreement in a frozen feature space. The two reliability signals are robustly standardized and modeled using a two-component Gaussian Mixture Model (GMM) to estimate sample-wise reliability posteriors. The framework further incorporates temporal posterior smoothing and an adaptive, risk-constrained selection strategy with asymmetric threshold updates. The resulting pseudo-labels supervise task-specific LoRA experts built upon a frozen pretrained Vision Transformer. Experiments on CIFAR-10, CIFAR-100, ImageNet-R, and ImageNet-A under different labeling ratios demonstrate competitive or improved average incremental accuracy, particularly in challenging low-label settings. Ablation studies, pseudo-label precision analyses, and additional plug-in experiments provide evidence for the effectiveness of the proposed reliability-aware selection mechanism.

Critical strengths
  1. Combining prediction stability with prototype agreement is a reasonable way to assess pseudo-label quality beyond confidence alone. The comparisons in Figures 1 and 3 provide useful evidence for the benefit of semantic information.
  2. RCF improves average incremental accuracy over CoVar from 21.59% to 34.30% on ImageNet-R and from 13.40% to 21.32% on ImageNet-A with 1% labels. The CL-LoRA experiment also shows a substantial improvement in pseudo-label precision (47.20% to 83.87%).
Critical weaknesses
  1. Proposition B.3 relies on a gradient inequality in Eq. (28) that does not generally hold, while Theorem B.11 treats GMM component posteriors as probabilities of pseudo-label correctness without sufficient justification. The convergence proof in Theorem B.8 also overlooks state-dependent EMA updates and appears inconsistent with the implementation's immediate upward update.
  2. The paper argues that earlier pseudo-label errors progressively corrupt later representations, but the backbone and previous LoRA experts are frozen, making this cross-task propagation mechanism unclear. The experiments mainly demonstrate better pseudo-label selection rather than directly establishing the claimed cumulative degradation across tasks.
Questions for authors

See in weakness.

11 comments
AuthorsOpenReview ↗

Response to Reviewer G9dk (Part 1)

We thank the reviewer for the careful assessment of our theoretical arguments and their consistency with the implemented architecture. In response, we have corrected the cross-task interpretation, refocused the motivation on within-task pseudo-label feedback, and revised the gradient, threshold-convergence, and noise-bound analyses in both the main text and appendix. We also clarify the distinction between GMM posterior scores and actual pseudo-label correctness, supported by additional task-wise comparisons of reliability scores and empirical precision. Below, we explain these corrections and supporting results.

w1. Given the frozen backbone, previous experts, and routing features, how can earlier pseudo-label errors corrupt representations used by subsequent tasks?

A1. Our original motivation incorrectly attributed cumulative cross-task representation corruption to the implemented task-isolated architecture. We have corrected this explanation in both the main text and the appendix.

As specified in Section 3.1 and Appendix C.3, pseudo-label supervision updates only the active task-specific expert, while the pretrained backbone and previously learned experts remain frozen. Appendix C.8 further specifies that the router is constructed from frozen backbone features without using class labels or pseudo-labels. Earlier pseudo-label errors can therefore affect their originating expert, but these architectural components do not provide the propagation pathway described in our original cross-task explanation.

We have revised the Abstract, the motivation and contribution statements in Section 1, and the opening of Section 3.2 to locate the feedback within the training of an individual expert. Its predictions generate pseudo-labels, those labels supervise its updates, and the updated expert produces predictions for subsequent epochs of the same task. Label errors enter this process as erroneous supervision. The current task's data remain accessible across epochs, allowing pseudo-label refinement during its training window. Section 3.1 now explicitly explains that task isolation preserves completed experts but does not resolve pseudo-label errors during active-expert training. Once that expert is frozen, subsequent expert training does not revise its parameters.

The revised motivation therefore concerns how pseudo-label errors affect the active expert's learning and how reliable selection improves its supervision. Correspondingly, the overview of Appendix B and the analysis in Appendix B.1 now focus on within-task supervision errors. Proposition B.3 quantifies pseudo-label-induced gradient perturbation and its effect on local optimization. The controlled label-correction experiment in Appendix D.5, Fig. 5 provides complementary empirical evidence by measuring the effect of correcting labels on the same selected samples. See our responses to w2 and w3 below for further details.

AuthorsOpenReview ↗

Response to Reviewer G9dk (Part 2)

w2. How is Proposition B.3's reliability non-recoverability claim justified when conflicting historical and current-task gradients can invalidate the gradient-norm inequality in Eq. (28)?

A2. Following the motivation correction described in w1, the relevant theoretical question is how erroneous pseudo-label supervision affects the active expert's optimization. The original historical/current-task gradient comparison does not hold in general under gradient cancellation and cannot establish the stated non-recoverability claim. We have replaced that argument with a within-task optimization analysis in Appendix B.1 and Proposition B.3.

The revised analysis compares two objectives on the same selected batch: F^(θ)\widehat F(\theta) uses the generated pseudo-labels, whereas the reference objective F(θ)F(\theta) replaces them with the corresponding true labels. The reference is used only for analysis. We hold the current update state, selected samples, augmentation views, and unsupervised-loss weight λ\lambda fixed, so the gradient difference isolates the effect of label correctness.

For softmax cross-entropy, replacing a label changes the logit gradient by the difference between two one-hot label vectors. This difference is zero for a correct pseudo-label and has Euclidean norm 2\sqrt{2} for an incorrect one. If the logit Jacobians have operator norm bounded by MM, averaging these contributions gives

\qquad \|b(\theta)\|_2\leq\sqrt{2}\lambda M\epsilon,$$ where $\epsilon$ is the selected pseudo-label error rate. This provides an explicit upper bound on the update-direction perturbation introduced by erroneous supervision. We then connect this perturbation to local optimization. If $F$ is $L$-smooth along the update segment, the gradient step $\theta^+=\theta-s\nabla\widehat F(\theta)$, with $0<s\leq1/L$, satisfies $$F(\theta^+)\leq F(\theta) -\frac{s}{2}\|\nabla F(\theta)\|_2^2 +\frac{s}{2}\|b(\theta)\|_2^2 \leq F(\theta) -\frac{s}{2}\|\nabla F(\theta)\|_2^2 +s\lambda^2M^2\epsilon^2.$$ The reference-gradient term contributes descent, while pseudo-label errors introduce a quadratic penalty in the bound. When $\epsilon=0$, this penalty vanishes. Reducing $\epsilon$ tightens both the gradient-perturbation bound and the local optimization bound. The revised proposition therefore provides a precise theoretical rationale for accurate pseudo-label selection: it limits the supervision-induced perturbation of the active expert's updates and reduces the error-dependent term in the optimization guarantee. This conclusion concerns within-task learning under the stated assumptions and requires no comparison between historical and current-task gradient norms.
AuthorsOpenReview ↗

Response to Reviewer G9dk (Part 3)

w3. Do the experiments directly establish cross-task reliability collapse and its causal mechanism, or primarily improvements in pseudo-label selection and downstream accuracy?

A3. The experiments support the revised motivation in w1 by examining the learning consequences of pseudo-label errors within the originating expert. The local analysis in w2 identifies how these errors perturb an update; we next evaluate their effect on training performance through a controlled label-correction experiment.

A comparison between different selection methods can involve changes in both label correctness and the samples used for training. To isolate the effect of label correctness, Appendix D.5, Fig. 5 compares two supervision settings on exactly the same selected samples. The experiment uses ImageNet-R with 5% labels under task-ID inference, keeping the evaluation focused on expert learning.

Setting A trains with the pseudo-labels selected by RCF. Setting B retains the selected samples but replaces their pseudo-labels with oracle labels:

Supervision on the same selected samplesAIA (%)
Original pseudo-labels — Setting A81.21
Oracle labels — Setting B86.48

Correcting the labels improves AIA by 5.27 percentage points. Because sample identity and supervision quantity are fixed, this gain cannot be attributed to selecting more examples or choosing a different subset. It demonstrates that residual label errors limit the learning benefit obtained from the samples already selected. Oracle labels are used solely for this diagnostic intervention.

The theory and experiment therefore provide complementary evidence: w2 bounds the perturbation introduced by erroneous supervision, while this intervention measures the performance consequence of correcting that supervision. Together, they support accurate pseudo-label selection during active-expert training. The empirical conclusion concerns the effect of label correctness on learning, rather than a collapse trajectory across task boundaries.

This establishes why supervision quality matters. In w4, we explain how RCF's joint evidence improves selection accuracy and distinguish the model scores guiding selection from the actual correctness of the selected labels.

Reviewer_SF9iOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The proposed Reliability-aware Calibration Framework (RCF) uses a frozen pretrained ViT with task-specific LoRA experts. It estimates pseudo-label reliability along two dimensions: predictive reliability, derived from predictions on two weakly augmented views and their normalized Jensen–Shannon divergence; and semantic reliability, measured by class-prototype similarity margins in the frozen ViT feature space with LoRA disabled. After robust standardization using the median and median absolute deviation (MAD), a two-component Gaussian mixture model estimates each sample’s reliability posterior. These posteriors are smoothed over time, and pseudo-labels are selected using an adaptive threshold subject to a target reliability constraint. An asymmetric threshold update is intended to make selection responsive to improvements while remaining cautious when reliability deteriorates.

Critical strengths
  1. The calibration framework produces substantial gains in difficult settings. Robust standardization and mixture modeling replace a fixed confidence threshold with a task-dependent estimate of reliability.
  2. The active LoRA expert measures prediction consistency under local perturbations, while the frozen backbone supplies a separate semantic reference. This helps address cases where a model is consistently confident about an incorrect prediction.
Critical weaknesses
  1. Each unlabeled sample requires two weakly augmented views and one strongly augmented view. The weak views are processed both by the active expert and by the frozen backbone with LoRA disabled. The method also fits a full-covariance 2D GMM over the unlabeled data using multiple EM initializations each epoch. Training time and memory comparisons with the baselines would help assess the practical cost.
  2. The method introduces many hyperparameters without sufficient sensitivity analysis.
  3. Some components offer little benefit at higher label ratios.
Questions for authors
  1. Are η\eta and αu,αd\alpha_u,\alpha_d fixed across datasets and label ratios, or tuned separately?
  2. At a 1% label ratio, some datasets may provide only one or two labeled examples per class. Averaging three weak augmentations of these examples cannot fully address sampling noise or multimodal class distributions. How reliable are the resulting prototypes, and how sensitive is qjq_j to the choice of labeled examples?
5 comments
AuthorsOpenReview ↗

Response to Reviewer SF9i (Part 1)

We thank the reviewer for the careful reading and constructive feedback. In response, we have conducted additional experiments on hyperparameter sensitivity and prototype reliability, and compiled runtime records to complement the existing analyses. Below, we present these results alongside the relevant ablation studies in the main text and appendix, clarify parameter settings across datasets and label ratios, and explain the complementary contributions of RCF’s components.

w1. How does RCF compare with baselines in training time and GPU memory, given its multi-view processing and per-epoch multi-start GMM fitting?

A1. RCF's computational investment delivers improvements in both pseudo-label precision and continual-learning accuracy. Its adaptation remains parameter-efficient: only 0.155M parameters are optimized for the active task. Appendix D.6 and Table 16 provide runtime profiling and architecture-level comparisons, while the available ImageNet-R 1% training-time and GPU-memory figures are summarized below.

MethodRecorded full-run wall time (h)GPU memory (GiB)
TACLE4.233.06
USP7.242.58
CoVar4.914.19
CL-LoRA24.729.31
RCF10.543.97

Durations are historical records. Memory entries comprise code-based baseline estimates and a separate synthetic-input training-step measurement for RCF.

The multi-start GMM contributes only a small fraction of calibration time. For 1,200 unlabeled samples, full-covariance GMM fitting, posterior smoothing, and threshold updates together require approximately 0.102 seconds, accounting for 0.38% of the 26.75-second calibration procedure. This profiling shows that the two-dimensional mixture model supports distribution-adaptive selection at a low computational cost.

The main investment—extracting complementary predictive and semantic evidence—produces measurable selection gains. At the same 20% coverage on ImageNet-R with 1% labels, the joint reliability score achieves 77.1% pseudo-label precision, compared with 72.6% for confidence-based selection and 72.8% for predictive evidence alone (Section 4.3, Fig. 3(c)). Thus, the extracted evidence improves precision by 4.5 percentage points over confidence-based selection without reducing the selection budget.

The complete framework also achieves substantial classification improvements in the main evaluation (Table 1): RCF obtains 34.30% AIA, compared with 21.59% for CoVar, 15.02% for TACLE, and 7.01% for USP. These results place the additional computation in context: RCF provides markedly cleaner pseudo-label supervision and stronger learning performance, while retaining a small active parameter count and inexpensive probabilistic calibration.

w2. Insufficient hyperparameter sensitivity analysis.

A2. Appendix D.2 provides sensitivity analyses of the robust-scale floor β\beta (Table 7), reliability target η\eta (Table 8), posterior momentum and threshold-update coefficient (Table 9), selection and hard-CE temperatures (Table 10), and LoRA rank (Table 11). These analyses cover the main choices governing evidence normalization, pseudo-label selection, temporal stabilization, and adaptation capacity.

To complement these studies, we examine the remaining settings related to GMM fitting and unlabeled supervision on ImageNet-R with 1% labels and seed 1993. Each experiment varies one parameter while keeping all other settings fixed.

ParameterValueAIA (%)
Default configuration—34.30
Covariance shrinkage (ρ\rho)0.033.49
Covariance shrinkage (ρ\rho)0.434.23
Maximum EM iterations1033.15
Maximum EM iterations5033.31
GMM initializations133.29
GMM initializations533.37
Maximum unsupervised-loss weight0.133.27
Maximum unsupervised-loss weight0.533.84

The default configuration uses covariance shrinkage of 0.2, a maximum of 25 EM iterations, 3 GMM initializations, and a maximum unsupervised-loss weight of 0.2, achieving an AIA of 34.30%. Across the tested variations, the largest decrease from this result is 1.15 percentage points. Increasing the EM iteration limit or the number of initializations beyond the defaults does not improve AIA, supporting the default choices without requiring larger fitting budgets. Together with Tables 7–11, these results extend the sensitivity analysis across RCF’s main components and demonstrate stable performance within the tested ranges.

AuthorsOpenReview ↗

Response to Reviewer SF9i (Part 2)

w3. Some components offer little benefit at higher label ratios.

A3. The components of RCF address complementary aspects of pseudo-label selection. Predictive evidence pjp_j captures augmentation consistency, while semantic evidence qjq_j assesses whether predictions are supported by labeled-data prototypes. Robust standardization RStd(⋅)\mathrm{RStd}(\cdot) aligns the scales of these two coordinates for GMM fitting. Posterior smoothing hj(e)h_j^{(e)}、mhm_h aggregates reliability evidence across epochs, and asymmetric threshold τi(e)\tau_i^{(e)}、αu,αd\alpha_u,\alpha_d updates regulate how the selection boundary adapts during training. Together, these mechanisms connect sample-level evidence, distribution modeling, and temporal selection.

Their respective contributions are examined in Section 4.2: Fig. 3(a) compares predictive, semantic, and joint evidence; Fig. 3(b) evaluates robust standardization; and Table 2 separately examines posterior smoothing and threshold updates. Section 4.3, Table 3 further isolates the augmentation-stability term in predictive reliability. These analyses assess the different stages of the framework rather than attributing its effectiveness to a single component.

The observed contributions depend on the supervision regime, consistent with the functions of these components. Under scarce supervision, semantic support, stable coordinate scaling, and temporal aggregation are particularly valuable because the active expert receives little labeled guidance. As labeled supervision increases, predictive evidence becomes more informative, changing the relative contribution of the other reliability controls. Individual components therefore need not produce equally large accuracy gains at every label ratio. The ablations support their complementary roles within a unified selection framework, with the strongest combined benefits under scarce supervision, where reliable pseudo-label selection is most critical.

Q1. Are η\eta and αu\alpha_u,αd\alpha_d fixed across datasets and label ratios, or tuned separately?

A1. As specified in Appendix C.4, η\eta is determined by the label ratio using the same rule across all datasets: η=0.95\eta = 0.95 for 1% labels and η=0.90\eta = 0.90 for 5% and 25% labels. No dataset-specific tuning is performed. Here, η\eta specifies the target mean posterior score used to construct the candidate threshold. The higher target under scarce supervision imposes a stricter selection criterion, while the lower target allows broader use of unlabeled samples when more labeled supervision is available.

In contrast, αu=1\alpha_u = 1 and αd=0.3\alpha_d = 0.3 remain fixed across all datasets and label ratios. These settings immediately adopt higher candidate thresholds while smoothing downward adjustments, allowing the selection criterion to tighten promptly and relax gradually.

The corresponding sensitivity analyses are provided in Appendix D.2: Table 8 evaluates η\eta, and Table 9 evaluates αd\alpha_d with αu=1\alpha_u = 1 fixed, documenting the effects of these choices on performance.

AuthorsOpenReview ↗

Response to Reviewer SF9i (Part 3)

Q2. How reliable are the prototypes, and how sensitive is qjq_j to labeled-example selection at a 11% label ratio?

A2. In RCF, prototypes serve as class-level semantic references in the pretrained, frozen feature space. The score qjq_j measures the predicted class’s similarity advantage over the strongest competing class and is combined with predictive evidence for pseudo-label selection. Three-view averaging is used to stabilize each labeled example’s feature representation, while the influence of labeled-example selection is evaluated directly through a controlled resampling experiment.

Complementing the ablations in Section 4.2, Fig. 3(a), and Appendix D.1, Table 6, we resampled the labeled examples 10 times for each of 20 tasks on ImageNet-R with 1% labels. Each draw preserves the number of labeled examples per class. The trained model, predictions, and augmentation views remain fixed, and all draws are evaluated on a common probe set excluding every prototype-construction example. This design isolates the effect of labeled-example selection on qjq_j.

We evaluate both ranking consistency and selection quality. The mean pairwise Spearman correlation between rankings from different labeled sets is 0.551, indicating moderate consistency. We further evaluate the accuracy of pseudo-labels selected by qjq_j at fixed coverage levels:

Selection by qjq_jMean pseudo-label accuracyMean within-task resampling SD (pp)
Top 10%10\%95.16%95.16\%5.445.44
Top 20%20\%89.24%89.24\%5.895.89
Top 40%40\%75.80%75.80\%4.804.80
All samples44.58%44.58\%0.000.00

Precision is first averaged across draws within each task and then across the 20 tasks. SD reports the average within-task standard deviation across draws, measuring labeled-set sensitivity rather than retraining variability.

These results demonstrate that qjq_j retains useful discriminative information when the labeled examples are resampled. Across the evaluated selection budgets, high-scoring subsets achieve substantially higher mean pseudo-label accuracy than the full probe set. Together with the semantic-evidence ablations, these findings support the effectiveness of the semantic module under scarce supervision and its complementary role in RCF’s joint reliability estimation.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We sincerely thank all reviewers for their careful assessment and constructive feedback. Their comments have helped us sharpen the contribution statements, strengthen the theoretical analysis, and broaden the experimental evaluation.

The central contribution of this paper is that RCF reframes pseudo-label selection in continual semi-supervised learning as latent reliability estimation, jointly modeling predictive stability and prototype-supported semantic agreement. Task-adaptive mixture modeling captures complementary evidence beyond prediction confidence, while selection constrained by a target mean model-based score and asymmetric temporal calibration adapt the selection boundary as the active expert evolves. Together, these mechanisms improve pseudo-label selection and learning performance under scarce supervision, prioritizing supervision quality while adapting coverage throughout each task's training. Our responses clarify the connection between this selection mechanism, supervision quality, and learning performance, supported by revised theoretical arguments and complementary experiments.

The main revisions and supporting analyses are summarized below:

  1. Aligned motivation and theoretical analysis. We have refocused the motivation on within-task pseudo-label feedback, consistent with the implemented task-isolated architecture. The revised Appendix B.1 analyzes pseudo-label-induced gradient perturbation and its effect on local optimization. Theorem B.8 and Corollary B.9 explicitly handle state-dependent threshold updates and the implemented immediate upward update. Theorem B.11 and Corollary B.12 state the calibration conditions connecting reliability scores to correctness-based noise bounds.

  2. Strengthened reliability and robustness evidence. Epoch-wise measurements across three random seeds document within-task pseudo-label deterioration (Appendix D.10, Table 20), providing direct temporal evidence for the revised motivation. We also provide task-wise comparisons between selection scores and empirical precision, controlled labeled-set resampling to assess prototype sensitivity, and sensitivity analyses of the remaining GMM and unsupervised-loss settings. These results complement the existing ablations and Appendix D.2, supporting selection effectiveness and parameter robustness while clarifying the interpretation of the reliability target.

  3. Broadened experimental coverage. We evaluate RCF in standard SSL, within Keep-LoRA and CL-LoRA, under class-incremental learning with repetition, and with ResNet-18-based semi-supervised iCaRL. We also compare a replay-free RCF adaptation with a replay-enabled DER++ reference. These evaluations demonstrate applicability across learning architectures, backbone families, and training protocols within image classification.

  4. Clarified computational cost. We compile available training-time records and GPU-memory information to complement Appendix D.6 and Table 13. The runtime breakdown identifies feature extraction as the dominant calibration cost and quantifies the comparatively small contribution of GMM fitting and posterior/threshold updates.

Below, we address each reviewer’s comments with the corresponding analyses, experimental results, and manuscript references.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.