- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes Sparse Autoencoder Geometric Anchoring (SAGA), a representation-level regularization method for mitigating catastrophic forgetting in class-incremental learning. The key idea is to use Sparse Autoencoders (SAEs) to identify input-dependent, task-specific directions in feature space that should be protected during subsequent training. Specifically, SAGA computes the Jacobian of a task-boundary SAE encoder and applies singular value decomposition to construct low-rank geometric constraints on representation drift. These constraints are combined with experience replay and historical snapshots to balance stability and plasticity.
The paper provides theoretical analyses connecting Jacobian-based anchoring to latent feature matching, characterizing low-rank approximation errors, and establishing a conditional bound on old-task output drift. Experiments are conducted on CIFAR-100, CUB-200, ImageNet-R, and TinyImageNet using pretrained ViT-B/16 and randomly initialized ResNet-18 backbones. Results show that SAGA outperforms the evaluated conventional continual learning baselines across three pretrained settings, while results on from-scratch training are more mixed. Additional ablations investigate the effects of replay, historical anchoring, retained rank, and alternative geometric constraints.
Critical strengths
1. An interesting perspective on representation-level regularization.
The paper explores the use of SAE encoder Jacobians to construct sample-dependent, low-rank constraints on representation drift. This provides a potentially useful perspective on selectively preserving representations during continual learning, rather than uniformly penalizing feature changes.
2. Empirical evaluation across different training settings.
The paper evaluates the proposed method on both pretrained and randomly initialized backbones and includes ablations on replay, historical anchoring, and alternative geometric constraints. The results show improvements over several conventional continual learning baselines in pretrained settings, while also revealing limitations in other configurations.
Critical weaknesses
-
The paper primarily compares SAGA against relatively old continual learning baselines, such as ER, DER++, and ER-ACE, while omitting comparisons with more recent approaches, particularly PEFT/LoRA-based methods for pretrained models. This makes it difficult to assess the competitiveness of SAGA against current continual learning techniques. More importantly, SAGA retains additional historical snapshots containing images, representations, and geometric factors, whereas the conventional baselines operate with a fixed replay buffer. Without comparisons under matched total memory budgets, it is unclear whether the reported improvements primarily result from the proposed geometric constraints or the additional memory resources.
-
The central claim of the paper is that SAE-induced local geometry provides more effective constraints for mitigating forgetting. However, the experiments do not convincingly establish this advantage. First, Jacobian-based and latent-based anchoring employ different loss normalization schemes, making their comparisons potentially unfair. Second, SAE-Jacobian anchoring does not consistently outperform simpler alternatives such as PCA and classifier SVD, raising questions about the necessity of introducing SAEs. Furthermore, the paper lacks direct visualizations or empirical analyses demonstrating that the identified geometric directions correspond to task-relevant information or genuinely help preserve previously learned knowledge. Consequently, the effectiveness and necessity of the proposed geometric mechanism remain insufficiently validated.
Questions for authors
-
Given that the performance improvements appear to rely heavily on experience replay, could the authors provide a baseline that retains the same replay strategy and historical snapshots but removes the Jacobian regularization loss to isolate the actual contribution of the proposed geometric constraints?
-
Since SAE-Jacobian anchoring does not consistently outperform simpler alternatives such as PCA and classifier SVD, could the authors further demonstrate the necessity of SAE-induced geometry and provide comparisons with latent anchoring under matched regularization strengths?
4 comments
Baseline with the same replay strategy and snapshots (Q1)
Thanks for this remark. We now report storage for every method and separate the effect of the regularizer from the effect of data access.
Storage. SAGA can store the per-sample geometry, but it can equivalently store each snapshot's SAE and the anchored images' features, and recompute the directions at the task boundary (SAE takes about 3 MiB per snapshot), still being an explicit, rank-controllable quadratic formulation of SAE-based preservation. With the second implementation, SAGA retains 193–229 MiB across our scenarios: the snapshot images plus 27 MiB of features and SAEs. We will clarify this option in the manuscript.
- Same data and training, no anchor. We keep SAGA's snapshots, replay sampling and schedule fixed and set only the anchor weight to 0. Accuracy drops by 12.9 / 10.0 / 11.5 points (CIFAR-100-224 / CUB / ImageNet-R), and by 7.7 on ResNet-18 CIFAR-100. Therefore, the gain comes from our Jacobian-based regularizer, not from greater access to historical data or additional training passes.
- Equal storage. We give ER, DER++ and ER-ACE SAGA's exact stored images (1,156–1,472), replayed on SAGA's schedule, with each baseline's best of 3 learning rates. SAGA still leads the best baseline by +13.0 on CIFAR-100-224 and +2.5 on CUB, and trails ER-ACE by 5.0 on ImageNet-R. These results suggest that even under equal storage in some cases, SAGA’s anchoring can yield substantial improvements.
- Compute. The 6.4–7.1× figure was relative to SGD. Against ER and ER-ACE on the same GPU, SAGA is only about 1.5× slower per task.
| Method | Stored | CIFAR-100-224 | CUB | ImageNet-R |
|---|---|---|---|---|
| Best baseline, 500 images | 72 MiB | 63.86 | 68.89 | 58.36 |
| Best baseline, SAGA's images | 166–211 MiB | 62.99 | 71.18 | 66.60 |
| SAGA, without anchor | 193–229 MiB | 63.13 | 63.69 | 50.06 |
| SAGA | 193–229 MiB | 76.01 | 73.68 | 61.58 |
Overall, the results show that the geometric anchor itself (not the additional memory) is the main source of SAGA's gains: on identical stored data it adds 10–13 points on the pretrained benchmarks. We will add these storage-matched comparisons and the memory accounting to the revised paper.
Matching loss normalization schemes and further comparison with geometric alternatives
We thank the reviewer for raising comments about matching loss normalization schemes and further comparisons with geometric alternatives to show Jacobian benefits.
Fair latent comparison
We have now matched the normalization and independently tuned the regularization coefficients for Jacobian, latent, and frozen-support anchoring for a fair comparison.
The results are consistent with our theoretical comparison: at fixed support and full retained rank, Jacobian anchoring is the local quadratic approximation of latent matching, with the equal inside a fixed support at r = K. They show broadly comparable performance, with latent matching performing better in some settings. The latent loss is discontinuous: 22–45% of it has zero gradient once latents leave the TopK. The Jacobian loss is a smooth quadratic with no such dead zones. Therefore, the formulations differ when the representation moves across TopK support boundaries: latent matching can lose its restoring gradient for snapshot latents that are no longer active, whereas the Jacobian anchor continues to constrain the snapshot directions. This makes it less sensitive to the anchor weight, although final accuracy at tuned weights is the same. We will revise the paper to avoid claiming a general accuracy advantage for Jacobian anchoring, and to highlight this Jacobian-based anchoring characteristic.
Table 1. Jacobian vs latent vs frozen-support latent at matched strength. One normalization for all three objectives; each tuned independently on the same lr × anchor-weight grid on the validation split; class-IL test accuracy, mean ± std over 3 seeds.
| Setting | K | Jacobian | Latent | Latent, frozen support |
|---|---|---|---|---|
| CUB-200 (ViT) | 16 | 73.33 ± 0.60 | 73.85 ± 0.54 | 73.25 ± 0.57 |
| CUB-200 (ViT) | 64 | 74.56 ± 1.04 | 74.72 ± 0.97 | 74.71 ± 0.60 |
| ImageNet-R (ViT) | 16 | 62.32 ± 0.99 | 64.15 ± 1.49 | 61.10 ± 0.38 |
| ImageNet-R (ViT) | 64 | 64.58 ± 0.71 | 66.19 ± 1.12 | 64.40 ± 0.31 |
| CIFAR-100 (ResNet-18) | 16 | 27.81 ± 2.85 | 27.26 ± 2.12 | 26.53 ± 0.88 |
| TinyImageNet (ResNet-18) | 16 | 35.99 ± 1.30 | 32.17 ± 5.09 † | 32.23 ± 6.25 |
This does not undermine our contribution, which is still an explicit, rank-controllable quadratic formulation of SAE-based preservation. This enables explicit rank control, which latents cannot do. In matched-strength experiments, accuracy remains relatively high even at low ranks. This supports rank truncation as an interpretable way to control how many representation directions are protected, while retaining flexibility in the others.
Table 2. Sensitivity to the anchor weight
| Anchor type | Accuracy lost using one shared weight (mean / worst setting) | Drop one grid step from own best weight |
|---|---|---|
| Jacobian | 0.27 / 0.83 | 1.06 |
| Latent | 0.68 / 2.10 | 1.58 |
| Latent, frozen support | 0.05 / 0.28 | 1.24 |
Continuation regarding the geometry comparisons in the next comment.
... further comparison with geometric alternatives
Geometrical comparisons
To more thoroughly examine what benefits can SAE Jacobian-based anchoring bring, we analyze TopK support changes, centered kernel alignment (CKA), drift outside of the protected subspace and leakage. Compared with PCA and classifier-SVD anchoring, SAGA achieves comparable global stability by CKA while consistently obtaining lower drift outside the protected subspace on both CUB-200 and CIFAR-100-224 than other methods. It also best preserves local structure, producing the fewest TopK support changes in both settings. SAGA and PCA have a similar leakage, and SVD, as expected, obtains a lower value. Even though SAGA does not have the lowest readout leakage, it achieves the lowest forgetting on CUB-200 and competitive forgetting on CIFAR-100-224. This highlights an interesting distinction in our analysis: leakage measures how much of the classifier’s readout lies outside the protected subspace, but not necessarily how much continual learning updates actually move representations in those specific directions. Thus, static classifier sensitivity should be used along with other measures.
Overall, these findings support SAGA as preserving global representation stability and local SAE support structure. We will clarify that leakage is a conditional sensitivity factor in our bound, not a standalone predictor of forgetting. We will also add the additional analyses to the appendix of the paper and mention it shortly in the main paper. Moreover, this distinction motivates a direction for future work: jointly tracking readout sensitivity and update-induced representation drift may help identify which task-relevant information is disrupted first during continual learning, providing a more mechanistic account of forgetting – we will also mention it in the updated manuscript.
Table: Earlier tasks at the last task boundary. Stability and forgetting of tasks 1–9 after training on task 10, by anchor geometry (r = 16, normalized weights). Mean over earlier tasks, then over 3 seeds.
| Dataset | Geometry | Leakage β | Leakage L | CKA ↑ | TopK change ↓ | Drift outside subspace ↓ | Forgetting, live head ↓ | Forgetting, frozen head ↓ |
|---|---|---|---|---|---|---|---|---|
| CUB-200 | SAE (SAGA) | 0.619 | 0.963 | 0.966 | 0.147 | 48.8 | 14.2 | 2.2 |
| CUB-200 | PCA | 0.612 | 0.935 | 0.967 | 0.199 | 54.6 | 16.8 | 3.2 |
| CUB-200 | Classifier SVD | 0.586 | 0.850 | 0.924 | 0.255 | 63.5 | 21.6 | 5.6 |
| CIFAR-100-224 | SAE (SAGA) | 0.587 | 0.937 | 0.964 | 0.097 | 44.4 | 24.8 | 1.3 |
| CIFAR-100-224 | PCA | 0.567 | 0.857 | 0.965 | 0.174 | 49.3 | 24.2 | 1.0 |
| CIFAR-100-224 | Classifier SVD | 0.545 | 0.785 | 0.941 | 0.146 | 50.0 | 22.4 | 3.9 |
Interpretation of the symbols: Leakage and measure how much of the task's readout A lies outside the protected subspace P (computed at the task's own boundary; for the SAE geometry, averaged over images). CKA: similarity of the task's held-out features now vs. at its own boundary. TopK change: share of an image's active SAE latents that changed since its boundary. Drift outside subspace: , the feature change outside the image's own protected directions. Forgetting: drop in a task's accuracy from its own boundary to the last one, with the current classifier (live) or with the classifier saved at that task's boundary (frozen).
We thank the reviewer for all the time and effort spent on providing us with helpful comments!
We thank the reviewer for finding SAGA’s use of SAE Jacobians for sample-dependent, low-rank representation anchoring an interesting perspective on continual learning. We also appreciate their recognition of the evaluation across pretrained and randomly initialized backbones, the ablations, and the balanced reporting of both gains and limitations.
We did our best to address all specific comments and provide additional experimental analysis. We address each comment in a separate comment. We hope that these revisions satisfactorily address the reviewer’s concerns!
We will be happy to answer if any of the issues remain.