- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes DSE-SSF for semi-supervised few-shot class-incremental learning. It trains a separate LoRA expert for each session on a frozen ViT-B/16. DSGE refreshes a kNN graph during training and propagates labels to unlabeled data. ADES selects pseudo-labels using an EMA expert, a consistency loss and an adaptive threshold. BFME perturbs features toward the decision boundary and enforces prediction consistency. The experiments cover CIFAR-100, CUB-200 and ImageNet-100, with diagnostics of pseudo-label error and old-class drift and a theoretical analysis in the appendix.
Critical strengths
- All of them address the failure modes that were measured at the start in Figure 1.
- On CUB-200, DSE-SSF outperforms DiL in final accuracy (83.98 vs. 80.05). Removing any of the three parts on CIFAR-100 reduces the final accuracy by 3-5 points (Table 2) and exceeds the variation among seeds.
Critical weaknesses
- The novelty is limited. DSGE is similar to the iterative graph propagation of Iscen et al. (2019) and has a hard agreement filter. It is difficult to reconcile with the claim on line 324 that "existing graph methods construct a fixed graph once"; ADES is essentially Mean Teacher with a KL-based threshold, and BFME is feature-space VAT. The paper does not formalise "semantic evolution", and it also provides no ablations for the above canonical methods despite citing all of them, or comparisons with strong CIL baselines using pretrained backbones. Therefore, based on the above results, it cannot be determined whether the improvements are due to the new modules or the frozen ViT and separate LoRA experts for each session.
- Some of the empirical claims are not supported by the results. (1) Table 1 does not support the claim that DSE-SSF "consistently outperforms" the existing methods. Final accuracy on CIFAR-100 is 83.82±1.90 for DSE-SSF and 83.68±0.33 for DSGD; is 10.01 for DSE-SSF, compared to 7.23 for DSGD and 9.88 for KeepLoRA++. Avg. accuracy on ImageNet-100 for DSE-SSF is 86.67; both DSGD (89.20) and KeepLoRA++ (87.88) are higher. (2) based on , the estimated base-session accuracy on ImageNet-100 for DSE-SSF is approximately 87.2%, and that of DSGD, KeepLoRA++, and DTE are 93.3%, 92.7%, and 92.5%, respectively. DSE-SSF 5-6 points are lower in the base session. (3) DSGE has little impact on new-class accuracy. Removing it reduces average new-class accuracy by only 0.93 points, from 95.45 ± 1.56 to 94.52 ± 1.68. The above difference is within the given standard deviation and has little margin for error at this accuracy. The above results are inconsistent with the idea that DSGE "enhances novel-class acquisition" (L. 461-462).
- Errors exist in the formulation and theory. (1) Equation (11) is inconsistent with the behaviour mentioned in Section 3.3. Mean KL divergence is non-negative; thus, only increasing is possible. It can never be lowered as said. (2) Theorem 1 is not applicable as presented. The explored points depend on the model trained on the labeled samples and their augmentations; they are not i.i.d. draws from the data distribution. Therefore, Rademacher's inequality in Eq. (47) is not applicable to the augmented sample of size . Then, just -fold replication of the training set would reduce the complexity and confidence terms by without introducing new information. (3) The second inequality in Eq. (50) is in the wrong direction. Substitution of by requires the inverse of Eq. (49). Replacement of by also requires the reverse of Eq. (48).
Questions for authors
- How does DSGE differ from Iscen et al. (2019) or DTE beyond replacing entropy weighting with a hard agreement filter? Have you made direct comparisons?
- What happens when ADES is replaced with plain Mean Teacher and BFME with feature-space VAT?
- Is BFME used only for labelled data, or also on pseudo-labelled data in ? If it contains the latter, then how does an incorrect pseudo-label affect ?
- Frozen ViT-B/16 with a prototype classifier or RanPAC how do they perform under the above protocol? DSE-SSF's performance without unlabeled data?
- How is the expert chosen at test-time when task IDs are unknown? Router is mentioned only once in Appendix C.3.
- Given that the shared projection and LayerNorm are frozen, and each session has its own LoRA (L. 415), is the 0% drift in Figure 3(b) guaranteed by design? Drift without ADES that you have observed?
- Is the EMA in Eq. (9) updated at the end of an epoch or after each iteration? m = 0.99 and 30 epochs; thus, an update per epoch would leave the teacher near its initialisation.
7 comments
Response to Reviewer c4dy
Q4. Frozen ViT, RanPAC, and DSE-SSF Without Unlabeled Data (No-U)
To directly test whether our performance comes merely from the frozen ViT-B/16 and per-session LoRA isolation rather than unlabeled semantic evolution, we added a strict No-U (No Unlabeled Data) control on ImageNet-R (Sec. 4.2 & Table 6, Appendix C.3). No-U keeps the exact same frozen ViT-B/16, SHA-256-matched Base checkpoints, per-session LoRA experts, and labeled supports per incremental class, while completely disabling all unlabeled training, pseudo-label calibration, and unlabeled routing records (verified to have zero unlabeled sample usage across all seeds).
Holding the backbone and expert architecture identical, exploiting unlabeled data via Full improves ImageNet-R Final accuracy by points and Average accuracy by points over No-U, with positive gains in every seed (driven by stronger novel-class acquisition). While training-free prototype classifiers and closed-form random-projection methods like RanPAC explore orthogonal classifier designs on frozen features, our matched No-U ablation and Table 2 ablations ( to points on CIFAR-100) directly confirm that DSE-SSF's semi-supervised evolutionary training provides substantial gains beyond the frozen ViT and LoRA architecture, with minimal overhead ( trainable LoRA parameters per session; Table 8, Appendix C.4).
Q5. How Is the Expert Chosen at Test Time Without Task IDs?
No task ID is used at test time. Similar to prototype-based routing in continual LoRA methods (e.g., CL-LoRA), all experts are evaluated in parallel and combined via input-dependent fusion weights computed from cached feature prototypes and calibrated output statistics (storing no raw images). We moved this complete formulation into Sec. 3.1 and detailed the feature-cache budget in Appendix C.1.1.
Q6. Is the 0% Drift in Fig. 3(b) Guaranteed by Freezing? What Is the Drift Without ADES?
You are spot-on regarding Fig. 3(b): because the Base expert and shared projection are frozen during incremental sessions, the Base feature path is invariant by construction. The actual target of ADES is to stabilize the trainable active expert (whose LoRA weights and cosine classifier are actively updated via gradient descent during the joint-training epochs).
To directly measure whether ADES prevents representation drift in the trainable parameters, we tracked the active expert's feature shift from the end of -shot support warm-up to the end of joint semi-supervised training within each session across ImageNet-R and CUB-200 (Table 5, Appendix C.3).
Table 5. Trainable active-expert representation stability on ImageNet-R and CUB-200. Session averages within each seed, reported as three-seed mean sample SD over fixed class-balanced probes ( samples/session). Cosine Drift is ; -NN- Churn is the fraction of changed neighbors.
| Configuration | ImageNet-R Drift | ImageNet-R Churn (%) | CUB-200 Drift | CUB-200 Churn (%) |
|---|---|---|---|---|
Full (with ADES) | ||||
w/o ADES | ||||
w/o companion consistency |
Removing ADES causes the active expert's feature drift to more than double, increasing by on ImageNet-R (ADES reduces drift by ) and by on CUB-200 (ADES reduces drift by ), while -NN- churn jumps from and . We explicitly clarified Fig. 3(b)'s frozen-Base invariance in Sec. 4.3, Fig. 3's caption, and Appendix C.3, and reported these direct trainable active-expert measurements in Table 5 (alongside fixed-Base cohort retention in Table 4).
Response to Reviewer c4dy
Q7. Is EMA Updated per Epoch or per Step?
In Full, the EMA temporal expert is updated once per epoch ( updates over the joint-training epochs, at optimization steps per epoch), matching the epoch index in Eq. (9). Why does over epochs work well here? Because the temporal expert is initialized after the supervised warm-up epochs on the -shot support set , rather than from random or cold parameters. Thus, starts as an already support-adapted model and acts as a conservative anchor that retains weight on the clean support-warmed state while gradually incorporating of the unlabeled joint-training trajectory. (In MT-adapted in Table 3, we also tested step-wise EMA with matched terminal decay .) We made this post-warm-up initialization and update schedule explicit in Sec. 3.3 and Appendix C.1.2.
Response to Weaknesses W2 (Claims, Base Gap, New-Class Gain) & W3 (Eq. 11 & Theory)
- Calibrating claims & explaining the ImageNet-100 Base gap (W2.1 & W2.2): We calibrated our performance claims in the Abstract, Sec. 1, Sec. 4.1, and Appendix C.1.3 to reflect exact dataset and metric results rather than blanket "consistently outperforms" phrasing. Regarding ImageNet-100 ( Base + incremental classes): Table 1's Final ( vs. ) and ( vs. ) imply a Session-0 Base accuracy of for DSE-SSF vs. for DSGD (as baselines use their native Base training recipes while DSE-SSF adapts rank- LoRA, LayerNorm, a -d projection, and a cosine head). Far from weakening our incremental contribution, this shows that DSE-SSF's Final accuracy on ImageNet-100 () is achieved despite starting points lower in Base accuracy—dropping only points across incremental sessions compared to points for DSGD (while the lower Session-0 starting point mechanically lowers the 9-session
Avg.mean). Furthermore, all our new component and replacement experiments use SHA-256-identical Base checkpoints per seed to eliminate any starting-checkpoint differences. - Global new-class acquisition without the local-head ceiling (W2.3): In Table 2 (CIFAR-100),
Avg. Newwas evaluated on the active expert's -way local head in isolation, where accuracy is already near the ceiling (). When we evaluate the incremental classes on ImageNet-R under full -class global competition (Sec. 4.2 & Table 6, Appendix C.3), adding DSGE improves novel-class recognition by points (positive in every seed) and improves overall -class Final accuracy by points, directly substantiating our novel-class acquisition claim. - Fixing Eq. (11) and Theorem 1 / Eq. (52) (W3): Thank you for catching the typographical recurrence in Eq. (11) and the reversed substitutions in the old Theorem 1 proof. In our implementation, the selection threshold at epoch , expansion round is recomputed relative to the fixed base threshold rather than accumulated recursively:
where is the mean untempered over the remaining unlabeled pool. Thus, as teacher-student disagreement decreases during training, naturally decreases (loosens) toward . We corrected Eq. (11) in Sec. 3.3 and updated the schematic expression in Fig. 2. Finally, as detailed in our response to Reviewer 9jbN (Q1), we retained the standard -sample bound in Eq. (47) and replaced the independent-sample counting and one-sided substitutions in Theorem 1 with our tight, dependency-free pairwise margin stability bound () and corrected Eq. (52) in Appendix A.
Response to Reviewer c4dy
We thank the reviewer for the thorough, constructive critique and for recognizing our quantitative motivation diagnostics and three-seed ablations. Your detailed questions regarding canonical baselines, active-expert drift, EMA dynamics, and theoretical bounds have helped us substantially strengthen both our empirical evidence and mathematical presentation in the revised manuscript.
Q1. How Does DSGE Differ from Iscen et al. [1] and DTE [4]?
In DSE-SSF, "dynamic semantic evolution" refers to a concrete closed-loop feedback mechanism during incremental expert adaptation (Secs. 3.2–3.3): as the active LoRA expert updates its representation every η=5 epochs, its k-NN manifold topology is reconstructed; sample selection then requires joint agreement between graph propagation and the expert's local cosine classifier (Eq. 7), subject to margin constraints, per-class budgets, and an uncertainty threshold dynamically adjusted by the temporal expert's disagreement (Eq. 11).
We have revised Sec. 2 and Sec. 3.2 (Line 324) to explicitly credit Iscen et al. [1] for alternating representation learning and label propagation, and focused our distinction from both [1] and DTE [4] on our multi-criteria graph-model agreement gate and temporal uncertainty feedback. In direct controlled comparisons within the exact same multi-expert architecture,
FulloutperformsLP-adapted[1] by +0.83±0.71 Final and +0.91±0.35 Average points on ImageNet-R, and achieves +0.55±0.27 points higher final Fixed-Base retention on CUB-200 (Table R1, positive in all three seeds), complementing our 11–26-point gains over DTE [4] in Table 1.Q2. Replacing ADES with Mean Teacher [2] and BFME with Feature-Space VAT [3]
We have implemented and evaluated both canonical replacements within our exact expert architecture under identical initialization, support sets, and update budgets (1,200 joint steps; see Response to Reviewer hQ1L, W1 and Table R1):
MT-adapteduses independent augmentations, unidirectional teacher-to-student KL (temperature 1), fixed-threshold selection (removing ADES's adaptive threshold feedback), and step-wise EMA (mstep=0.991/4≈0.99749).Feature-space VATuses label-agnostic predictive KL divergence on normalized features (ℓ2 radius 0.05, finite-difference scale 0.001, 1 power iteration), whereas BFME uses label-guided cross-entropy for 3 ℓ∞-projected ascent steps (radius 0.005) followed by symmetric KL.On CUB-200 (Table R1),
FulloutperformsMT-adaptedby +0.14±0.12 paired Final points andFeature-space VATby +0.22±0.08 paired Final points, with positive gains across all three seeds. On ImageNet-R, their overall accuracies are closely matched toFull(MT-adaptedslightly higher,Feature-space VATslightly lower). Because each single-component replacement retains the other two native DSE-SSF mechanisms in the loop, these experiments complement our component-removal ablations (Table 2, where removing ADES or BFME drops CIFAR-100 Final accuracy by 3.49 and 5.26 points) and our direct mechanism probes (Table R3).Q3. Does BFME Use Pseudo-Labels, and What Happens When a Pseudo-Label Is Wrong?
Yes, BFME is applied to the combined training batch drawn from both the 5-shot labeled support set Dlt and the selected reliable pseudo-labeled set D^t. Specifically, the sample's label y (true support label or selected pseudo-label) guides the inner ascent cross-entropy in Eqs. (13)–(14) to locate low-confidence boundary regions, while the outer loss in Eq. (15) enforces label-free symmetric-KL consistency between the reference feature h′ and the explored feature h′+ϵ.
If an admitted pseudo-label is incorrect, it can steer the inner perturbation ϵ toward the wrong class boundary, and the outer consistency loss does not by itself correct the label error. This is precisely why DSGE and ADES's selective filtering is placed upstream of BFME. In our three-seed
Fullruns, the pseudo-labels admitted into D^t achieve an empirical precision of 98.57±0.32 (at 54.66±0.42 coverage) on CUB-200 and 98.26±0.42 (at 33.62±0.28 coverage) on ImageNet-R. Thus, $ > 98\%$ of the pseudo-labeled samples entering BFME carry true class labels, ensuring that boundary exploration predominantly targets genuine decision boundaries. We have clarified this sample scope and the distinct roles of labels in Eqs. (14) and (15) in Sec. 3.4 of the revised manuscript.