acceptodds
Under review as a conference paper at ICLR 2027

GYRO: Anchoring Representations to Simplex Frames for Continual Self-Supervised Learning

Abstract

Geometric self-supervised learning that use spectral objective provides a principled foundation for extracting robust object manifolds from unlabeled data streams. However, when applied in a continual setting, learned representations rotate freely and drift into established object manifolds. Here, we introduce GYRO, a continual self-supervised learning method that minimizes this drift by anchoring representations to nearest persistent simplex frames. Building on Manifold Capacity Theory, GYRO constrains rotational invariance and enforces geometric stability without requiring class labels, auxiliary alignment losses, or rigid image-to-prototype assignments. Our approach acts directly on the geometry of the embedding space, allowing the network to flexibly adapt its parameters while maintaining separable manifolds. Extensive evaluations across standard benchmarks show that GYRO consistently outperforms replay, distillation, and regularization baselines. Our results highlight that injecting persistent geometric references transforms unconstrained spectral objectives into stable continual learners, demonstrating that representation drift is fundamentally an alignment problem rather than a loss of network capacity.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Self-Supervised Learning, Continual Learning, Manifold Capacity

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Me4GOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces GYRO, a continual self-supervised learning method that reduces rotational invariance by anchoring representations to persistent reference directions.

The goal is that different views or augmentations of the same image should be similarly represented in the embedding space, while different images should be spread across the embedding space.

The method is an extension of MMCR, where each task receives a randomly oriented simplex frame that remains fixed after initialization. Every time an image is presented, its centroid is mixed with the nearest prototype of its task’s frame, and the nuclear norm is evaluated on the mixed centroids.

GYRO does so by introducing fixed reference directions, arranged as a regular simplex, for each task. Each image’s representation is mixed with its nearest reference before calculating the existing spectral objective, combining an image mean representation, selected reference, and anchoring strength.

With /beta=0, GYRO represents MMCR exactly.

Evaluated on two natural image benchmarks: Split CIFAR-100 (50 tasks) and Split Tiny ImageNet (100 tasks)

Critical strengths

The paper has a clear methodological contribution. The modifications of MMCR are easy to follow and understand.

Preserving learned geometry is a plausible mechanism for reducing catastrophic forgetting. The paper provides evidence that GYRO achieves greater geometric stability than comparable methods. More specifically, the method outperforms replay, distillation, and regularization baselines.

The reported final accuracies show where the contribution is strongest. As the sequence becomes longer, replay becomes increasingly beneficial, and GYRO achieves the highest final accuracy on both the 20- and 50-task sequences

Critical weaknesses

The authors provide empirical evidence that geometric stability minimizes this drift by anchoring representations to the nearest persistent simplex frame, which is achieved with GYRO. However, the authors do not convincingly establish a causal explanation for the superior performance. What is the reason that a network that flexibly adapts its parameters while maintaining separable object manifolds, performs better.

The authors write: “Hyperparameters were selected on test accuracy for every method, ours included”. This means that the test set has therefore participated in model selection. Reported accuracies can be (and probably are) too optimistic. A convincing evaluation requires choosing configurations on validation data, freezing them, and then reporting final test performance.

Figure 2 uses one run per method for ResNet, the best of three seeds for Tiny ImageNet, with the baseline ER using a different seed. The differences in the results are small. I would have expected more runs and a more statistical analysis in this paper, for the strong claims in the paper.

Questions for authors
  1. Can you rerun Tables 1–2 with hyperparameters selected on validation data, comparable tuning budgets, and an untouched test set? Would the results survive independent model selection?


  2. What evidence establishes the proposed rotational mechanism?
Can you separate common rotation, task-dependent movement, and residual deformation, for example using orthogonal Procrustes alignment,and show which changes predict forgetting?

  3. How much of the improvement comes from task identity?
Can you compare against a baseline with the same per-image task IDs and test streams where task boundaries are less closely aligned with class membership?

3 comments
AuthorsOpenReview ↗

Response to Reviewer Me4G (1/2)

We thank you for reading our paper closely and for pointing out where our explanation fell short. It was helpful to know that the method itself was clear to you. We address your weaknesses and questions in order below.

W1. The authors provide empirical evidence ....

We agree that Figure 2 in the submission showed a correlation on single runs, and that a causal explanation calls for further evidence. The main reason GYRO performs better is that it reduces confusion between tasks. In every method we tested, 98 to 99% of the remaining errors assign a test image to a class of another task (50 CIFAR-100 tasks and 100 Tiny ImageNet tasks). MMCR optimises the geometry of the current batch, and it contains no term that ties the representation of an earlier task to a persistent reference across tasks. GYRO adds such a reference by assigning each task a fixed frame in the embedding space. Because the images of a task remain aligned with their own frame, this helps keep the tasks separated from one another. As the table below shows, GYRO has the fewest cross-task errors, while accuracy within the right task is the same for every method. To verify that the frame is the cause, we ran targeted interventions that change one ingredient at a time:

  • Removing the anchor (β=0, effectively MMCR + replay) on CIFAR-100 (50 tasks) costs 4.0 points.
  • Randomizing the anchor (assigning each stored image to a random earlier task's frame) costs 3.6 points, bringing GYRO's performance back down to plain replay.
  • Isolating tilt vs. length on Tiny ImageNet (rescaling merged rows to the centroid's length so the anchor only tilts them) completely eliminates the gain (18.3 vs 30.8).
Final Class-IL (%)Test images assigned to another task's class (%)Accuracy when the task is right (%)
Fine-tune49.6049.9699.13
CaSSLe51.0848.3598.90
ER53.4246.0498.99
DER++53.2246.2899.08
LUMP55.1544.4199.20
GYRO57.4542.0299.08

Action: In the revised paper, the abstract and introduction give this explanation in place of "rotate freely", and the error analysis and the interventions are in App. K.1 and K.2.

Q2. What evidence establishes the proposed rotational mechanism? ....

We ran the Procrustes analysis you suggest and separated the three parts. A common rotation accounts for most of the drop of an old task's probe, with or without the anchor, and it is harmless because re-fitting the probe absorbs it. Non-rigid change is the smaller part. On which change predicts forgetting, how far a task moved relative to the others does not predict its accuracy loss, but how much separation from other tasks it lost does. We kindly invite you to refer to our CQ2 (response to common questions).

W2. The authors write: ....

Q1. Can you rerun Tables 1–2 with hyperparameters ....

Yes, the results survive independent model selection. We kindly request you to refer to our CQ1 (response to common questions)

W3. Figure 2 uses one run per method ....

We agree. Fig. 2 now uses the same three seeds for every method and reports means with error bars. As you point out, the differences in disruption are modest, so we treat the figure as a description of the geometry and base our explanation on other evidence (please refer to our responses to W1 and CQ2). The accuracy differences on these same seeds are clear. GYRO is ahead of every other method with a paired 95% interval above zero, for example by +2.30 points over LUMP on CIFAR-100 (50 tasks) and by +3.93 points on Tiny ImageNet (100 tasks). This lead is also maintained across five random class orders. Fig. 3 now has three seeds in every cell, and every cell is measured directly.

Action: Fig. 2 is replaced by the matched-seed version with error bars in the revised paper, and the paired intervals on these seeds are in App. K.1.

AuthorsOpenReview ↗

Response to Reviewer Me4G (2/2)

Q3. How much of the improvement comes ....

Task identity by itself does not explain the gain. We gave ER and LUMP the same per-image task IDs that GYRO stores (task-balanced replay), and we gave GYRO the wrong IDs. Below, we report Final Class-IL accuracy (%), mean over three seeds, with the change from the same method in brackets (*: 95% interval excludes zero).

CIFAR-100, 50 tasksTiny ImageNet, 100 tasks
GYRO57.4531.83
GYRO, stored images anchored to a random earlier task's frame53.82 (−3.63*)25.65 (−6.18*)
ER53.4219.36
ER with the same task IDs54.04 (+0.62)19.55 (+0.19)
LUMP55.1527.90
LUMP with the same task IDs54.96 (−0.19)27.50 (−0.40)

The same task IDs do not help ER or LUMP. In GYRO the ID matters only because it selects each stored image's own anchor, and pointing it to another task's frame costs 3.6 and 6.2 points. The ID need not be a task label either: frames opened on a fixed clock, with no knowledge of task boundaries, stay within 1.3 points of true boundaries. The improvement comes from anchoring each task to its own fixed reference, not from the extra task information.

On streams less aligned with classes, we moved 20% of each class's images to the next task (blurry stream, CIFAR-100, 50 tasks), and GYRO keeps the highest mean accuracy (57.56, against 56.03 for LUMP). When every task contains every class there are no task-specific classes to keep apart, and per-task frames bring no gain.

Streams, CIFAR-100, 50 tasks

Class-incrementalBlurry (20% to next task)Every task has every class
GYRO57.4557.5658.03
LUMP55.1556.0361.24
ER53.4254.8259.05
Fine-tune49.6051.1057.84

Action: These comparisons are now in App. K.5 of the revised paper, which also states that GYRO targets class-incremental streams.

Thank you again for your precise questions. We hope our answers on the causal explanation, the model selection and the statistics behind Fig. 2 were as direct as the questions. If they resolve your concerns, we would appreciate it if you would consider raising your score.

AuthorsOpenReview ↗

Follow up response to Reviewer Me4G

Thank you again for your precise questions. We have uploaded the revised paper, and it includes the evidence you asked for. The abstract and introduction now explain the gain as fewer confusions between tasks, Fig. 2 uses the same three seeds for every method with error bars, and the interventions, the independent model selection and the task-identity controls are in App. K. Following the discussion with Reviewer JXmq, the abstract, the contributions and the discussion also state the claims more precisely, and these changes are marked in red. We would be glad to know whether this addresses your concerns, and we are happy to answer any further questions in the remaining time.

Reviewer_JXmqOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

Maximum Manifold Capacity Representations (MMCR) maximizes the nuclear norm of per-image view centroids, which is invariant under global rotations. GYRO, "a continual self-supervised learning method", gives each task a fixed, randomly oriented simplex frame of K prototypes and mixes each centroid with its nearest prototype, a convex combination, before the nuclear norm. Theorem 1 decomposes the deficit of this representation into view-spread, anchor-distortion and spectral terms. The authors have shown results on CIFAR-100 and Tiny-ImageNet with best final accuracy.

Critical strengths

Strengths:

  1. The idea of the paper is simple, I like the label-free approach.
  2. Clear geometric motivation: The paper identifies the absence of persistent directional references in MMCR’s rotation-invariant objective as a potential contributor to representation drift during continual learning.
  3. Theoretical characterization of the mechanism: The analysis identifies which rotational symmetries remain and decomposes the objective into contributions from view spread, anchor distortion, and spectral structure. This is the biggest strength in my opinion.
  4. Simple integration into the learning objective: GYRO mixes image centroids with their nearest fixed simplex anchors directly within the spectral objective, avoiding an additional loss or other Oracle teacher.
  5. Empirical comparisons supported by informative ablations: Experiments report improvements over evaluated replay, distillation, and regularization baselines, while some ablations been done.
Critical weaknesses

Weaknesses

  1. Test-set-based hyperparameter selection: The paper explicitly states that hyperparameters were selected using test accuracy for all methods (section 7). This introduces selection bias and weakens confidence in the reported gains. Why not apply the same procedure to every baseline? Otherwise it's not a fair comparison.

  2. The proposed explanation of forgetting is not fully established. Global rotations preserve pairwise geometry and need not harm a newly fitted linear probe. The theory characterizes a fixed-batch objective, but does not establish that rotational freedom causes forgetting during training. This is very important as in practice for continual learning, forgetting should be properly addressed.

  3. The geometric analysis uses one run per method, on Tiny ImageNet, it selects the best of three seeds and uses a different seed for ER (Figure 2). Some sensitivity results use a single seed, while untested heatmap cells are filled with neighboring averages (Figure 3). These choices weaken the reliability of the mechanism and robustness claims. Again lack of consistency in experimentation.

  4. Dependence on task boundaries and replay limits applicability. GYRO requires task-specific frames and stored task identities, and earlier frames influence learning only through replayed images. The paper reports no accuracy improvement without replay. Its effectiveness therefore remains unclear for settings where retaining past images is infeasible.

  5. Evaluation is restricted to two image datasets, a ResNet-50 backbone, and one fixed class order across seeds, with competing methods adapted to MMCR. GYRO also trails LUMP on five-task CIFAR-100 and has not been evaluated at the larger replay budgets reported for baselines. Broader architectures, task orders, native baseline objectives, and matched memory-budget sweeps are needed.

Questions for authors

alidation protocol: Since hyperparameters were selected using test accuracy, can you report results using a separate validation set, with configurations fixed before test evaluation? Does GYRO retain its advantage?

Evidence for the proposed mechanism: How do you distinguish harmful task-dependent drift from a harmless global rotation? Can an experiment isolate orientation anchoring from the other changes GYRO introduces to the spectral objective?

  1. Can you repeat Figure 2 across matched seeds and report means and variability, rather than selected runs? Does reduced geometric disruption consistently predict reduced forgetting?

  2. Can you replace the interpolated cells in Figure 3 with actual experiments and evaluate the main configurations across multiple seeds?

  3. What happens when task boundaries are unavailable or imperfect? Can a shared or automatically inferred frame provide comparable benefits without storing explicit task identities?

  4. Does GYRO remain competitive at larger, matched replay budgets and across randomized class orders? Can you clarify how adapting competing methods to MMCR affects their performance relative to their original objectives?

11 comments
AuthorsOpenReview ↗

Response to Reviewer JXmq (1/3)

Thank you for your detailed review. We are glad that the label-free design and the theoretical analysis were useful to you, and we take your concerns about soundness seriously. Please find our response to each point below.

W1. Test-set-based hyperparameter selection ....

Validation protocol. Since hyperparameters were selected ....

Yes, GYRO retains its advantage when hyperparameters are selected on a held-out validation set. As you suggested, we held out 10% of each class's training images and selected every baseline on validation accuracy, while keeping a single GYRO configuration (K=10, β=1/3) fixed for both datasets and every task count. GYRO is above every baseline at 20 and 50 CIFAR-100 tasks and at 50 and 100 Tiny ImageNet tasks, and its gain over the strongest baseline is statistically significant in all four settings. We kindly refer you to see of our CQ1 (response to common questions).

W2. The proposed explanation of forgetting ....

Evidence for the proposed mechanism ....

We agree that a global rotation does not harm a newly fitted probe, and that Theorem 1 and Proposition 1 describe the objective on a fixed batch, not the training dynamics. To separate harmful drift from a harmless rotation we used Procrustes alignment. The rotation is real but a re-fitted probe absorbs it, and what predicts a task's accuracy loss is how much separation from other tasks it lost, not how far it moved (please refer to CQ2 (response to common questions)). To isolate the anchor from the other changes GYRO makes to the objective, we changed one part at a time. Below, we report Final Class-IL accuracy (%), mean over three seeds, and the paired change from GYRO with its 95% interval.

Final Class-ILChange from GYRO
CIFAR-100, 50 tasks
GYRO57.45
Anchor removed (β=0)53.42−4.03 [−5.56, −2.50]
Stored images anchored to a random earlier task's frame53.82−3.63 [−4.76, −2.50]
Tiny ImageNet, 100 tasks (K=20, β=0.5)
GYRO30.81
Anchor rescaled to the centroid's length29.18−1.63 [−4.55, 1.29]
Merged row rescaled to the centroid's length (direction only)18.27−12.53 [−13.51, −11.56]

The gain comes from giving each task its own fixed reference, and it acts through the anchor term of Theorem 1(i). Removing the anchor, or pointing stored images to another task's frame, brings GYRO back to plain replay. The merge changes both the direction and the length of a row, and a row is longer the closer its centroid is to its anchor (Theorem 1(i)). Rescaling the anchor keeps the result within noise. Removing this length term, so that the anchor only changes the direction of a row, loses the gain.

Action: The intervention analysis is now in App. K.2 of the revised paper. The limitations state that the theory describes the objective on a fixed batch, and we will repeat this where Theorem 1 and Proposition 1 appear.

W3. The geometric analysis uses one run ....

Q1. Can you repeat Figure 2 ....

Q2. Can you replace the interpolated cells ....

We agree that single runs and interpolated cells were not a sound basis for these two figures, and we have redone both (Figures 2 and 3 of the revised paper). Every method now uses the same three seeds, and the figure reports means with error bars. Among the four methods of the figure, GYRO has the lowest mean disruption on both datasets. To the second part of Q1, lower disruption does not consistently predict less forgetting. Across methods, lower disruption goes with higher final accuracy on Tiny ImageNet but not on CIFAR-100, where CaSSLe changes least and is not the most accurate. Within a run, more disruption after a task does not mean more forgetting of that task. What does predict a task's accuracy loss is its loss of separation from other tasks (refer to CQ2 (response to common questions).

In Figure 3, every cell now has three seeds at 5, 20 and 50 tasks, and none is interpolated. At 20 and 50 tasks no cell is below β=0, and the best cells gain 2.5 and 3.7 points. At 5 tasks, where there is little forgetting to prevent, small β is neutral and large K with large β costs accuracy

Action: Figures 2 and 3 are replaced by these versions in the revised paper, and their captions state the number of seeds.

AuthorsOpenReview ↗

Response to Reviewer JXmq (2/3)

W4. Dependence on task boundaries ....

Q3. What happens when task boundaries ....

We agree that the submission assumed known task boundaries and a replay buffer. To test the behavior of the model when the task boundaries are unavailable or imperfect, GYRO simply initializes a new frame at regular intervals (every half, full, or two tasks). We apply a random offset to the start time so these intervals do not align with the true task changes. Consequently, each stored image only records the active frame at the time of storage (serving as a temporal timestamp rather than a semantic task label). We also tested perturbed boundaries by shifting each true boundary by up to a quarter of a task. The table below reports Final Class-IL accuracy (%) averaged over three seeds. Bracketed numbers indicate the performance delta compared to using true boundaries.

When a new frame opensCIFAR-100, 50 tasksTiny ImageNet, 100 tasks
At the true task boundaries57.1231.97
Every half task (no boundaries used)56.77 (−0.35)31.84 (−0.14)
Every task (no boundaries used)56.44 (−0.68)31.43 (−0.54)
Every two tasks (no boundaries used)56.41 (−0.71)30.73 (−1.25)
At wrong boundaries (shifted by up to ¼ task)56.39 (−0.73)30.73 (−1.24)
LUMP55.1527.90
ER53.4219.36
Fine-tune49.6024.96

In all boundary-agnostic settings, GYRO remains within 1.3% of the baseline with true boundaries, and none of the differences are statistically significant (paired t-test, 95% confidence level). Furthermore, we tested a "blurry" continuous stream (where 20% of a class's images leak into the subsequent task) and GYRO still maintained the highest mean accuracy (CIFAR-100).

Regarding a shared or automatically inferred frame, the fixed-interval scheme is an automatically assigned frame: it requires one stored integer per image and no task identity. A single frame shared by all tasks does not provide the benefit (App. B), and the stored index must be correct, since assigning stored images to a random earlier frame reduces accuracy by 3.6 points on CIFAR-100. We agree that replay is required, as the paper reports. GYRO is a replay-based method, like ER, DER++ and LUMP, and settings where past images cannot be retained are outside its scope.

Finally, we agree that GYRO inherently relies on replay. As noted in the submission, it is a replay-based method (comparable to ER, DER++, and LUMP) and is not applicable to strict exemplar-free continual learning.

Action: The boundary-free experiments are now in App. K.5 of the revised paper. Section 3 states the stored frame index, and the limitations state the reliance on replay.

W5. Evaluation is restricted to two image datasets ....

Q4. Does GYRO remain competitive ....

We agree that the evaluation was restricted on these points. We ran the four sweeps you requested: randomized class orders, a second architecture, the baselines at their native objectives, and larger matched replay budgets. Below, we report Final Class-IL accuracy (%), mean ± std, with GYRO at its fixed configuration (K=10, β=1/3).

(a) GYRO keeps its lead under randomized class orders. We repeated the comparison over five random class orders on CIFAR-100, under the validation protocol of CQ1.

CIFAR-100, 20 tasksCIFAR-100, 50 tasks
GYRO57.9 ± 0.4455.57 ± 0.25
LUMP57.28 ± 0.3353.28 ± 0.44
ER55.68 ± 0.4252.07 ± 0.76
CaSSLe52.63 ± 0.8050.18 ± 0.76
EWC52.57 ± 0.5847.94 ± 0.82
Fine-tune52.94 ± 0.3348.22 ± 0.65

(b) Architecture. In addtion to Resnet-50, we trained ResNet-18 on CIFAR-100, at the replay budget of the main tables (2,000 stored images, 8 replayed per step).

5 tasks10 tasks20 tasks50 tasks
GYRO55.5 ± 0.355.0 ± 0.653.5 ± 0.550.7 ± 0.9
LUMP (α=0.4, 0.4, 1, 0.4)56.91 ± 0.4654.71 ± 0.2352.10 ± 0.1947.78 ± 0.24
ER (= GYRO, β=0)55.9 ± 0.553.4 ± 0.350.0 ± 0.245.5 ± 0.5
DER++55.57 ± 0.3053.20 ± 0.2949.48 ± 0.5145.91 ± 0.31

This is the same picture as the five-task CIFAR-100 as with few tasks there is little forgetting to prevent, and GYRO's gains are on longer sequences.

AuthorsOpenReview ↗

Response to Reviewer JXmq (3/3)

(c) Native objectives and number of views. To see how adapting to MMCR affects the baselines, we ran each of them on Split Tiny ImageNet (batch 128) with MMCR at 10 views, MMCR at 2 views, SimCLR (2 views) and SimSiam (2 views). The remaining baselines are in the appendix.

MethodMMCR-10 (20 tasks)MMCR-2 (20 tasks)SimCLR (20 tasks)SimSiam (20 tasks)MMCR-10 (100 tasks)MMCR-2 (100 tasks)SimCLR (100 tasks)SimSiam (100 tasks)
Fine-tune29.1 ± 0.526.9 ± 0.428.1 ± 0.817.8 ± 0.324.9 ± 0.824.7 ± 0.424.5 ± 0.410.9 ± 0.3
EWC25.6 ± 0.720.6 ± 0.825.9 ± 0.718.4 ± 0.415.1 ± 0.217.8 ± 0.219.9 ± 0.414.4 ± 0.1
CaSSLe29.1 ± 0.427.0 ± 0.128.7 ± 0.3—24.9 ± 0.524.4 ± 0.226.8 ± 0.7—
ER31.5 ± 0.328.0 ± 0.529.6 ± 0.415.4 ± 0.527.8 ± 0.525.2 ± 0.125.6 ± 2.712.9 ± 0.4
DER++31.7 ± 0.227.7 ± 0.228.5 ± 1.014.7 ± 0.728.1 ± 0.225.7 ± 0.124.8 ± 2.61.5 ± 0.1
LUMP30.8 ± 0.127.6 ± 0.228.2 ± 0.718.9 ± 0.327.7 ± 0.225.3 ± 0.524.8 ± 0.412.6 ± 0.6
GYRO32.0 ± 0.527.4 ± 0.3——29.1 ± 0.425.8 ± 0.5——
Osiris-D†——26.0 ± 0.5———25.3 ± 0.4—
Osiris-R†——26.1 ± 0.3———24.7 ± 0.8—

†Osiris [1] is a state-of-the-art unsupervised continual learning method that is built on a sample contrastive objective and cannot be adapted to MMCR. We therefore ran its two variants, Osiris-D and Osiris-R, at their native objective (SimCLR, 2 views) in the setting of Tables 1–2, with the weight of its additional loss term selected on validation.

Adapting to MMCR helps the replay baselines, and this gain comes with the number of views: at 2 views MMCR is close to SimCLR for most methods, both are well above SimSiam, and Osiris at its native objective stays below GYRO with 10 views.

[1] Y. Zhang, L. Charlin, R. Zemel and M. Ren. Integrating Present and Past in Unsupervised Continual Learning. Conference on Lifelong Learning Agents (CoLLAs), 2024.

(d) With a larger buffer, GYRO stays ahead on long sequences. We write a budget as stored images / replayed images per step. At 5,000 / 8 it leads at 20 and 50 CIFAR-100 tasks, and on Tiny ImageNet it keeps improving with the budget while the baselines do not. With more replayed images per step on CIFAR-100, GYRO is still ahead at 20 and 50 tasks at 16 per step, while at 32 per step LUMP with a re-tuned α leads. GYRO's advantage is therefore at small per-step replay.

CIFAR-100, ResNet-50

MethodBudget5 tasks20 tasks50 tasks
GYRO2,000 / 1661.23 ± 0.3159.34 ± 0.4256.87 ± 0.37
ER2,000 / 1662.0 ± 0.457.5 ± 0.155.7 ± 0.1
LUMP (α=0.1)2,000 / 1662.6 ± 0.359.0 ± 0.956.4 ± 0.1
GYRO2,000 / 3260.11 ± 0.5658.15 ± 0.4556.18 ± 0.03
ER2,000 / 3260.6 ± 0.356.2 ± 0.553.8 ± 0.3
LUMP (α tuned)2,000 / 3263.8 ± 0.462.7 ± 0.161.8 ± 0.4
GYRO5,000 / 861.0 ± 0.261.2 ± 0.558.9 ± 0.6
ER5,000 / 861.1 ± 0.656.5 ± 0.653.4 ± 0.6
DER++5,000 / 860.6 ± 0.455.8 ± 0.653.4 ± 0.1
LUMP (α=0.4, 1, 0.4)5,000 / 863.03 ± 0.5658.03 ± 0.4655.30 ± 0.47

Tiny ImageNet, 100 tasks (GYRO at K=20, β=0.5; LUMP at α=1, not re-tuned)

Method2,000 / 85,000 / 810,000 / 16
GYRO30.81 ± 0.7532.23 ± 1.9435.40 ± 0.44
LUMP27.90 ± 1.1427.43 ± 0.7127.15 ± 0.84
DER++20.17 ± 0.3520.38 ± 0.6219.60 ± 0.21
ER19.02 ± 0.2618.84 ± 0.4618.61 ± 0.98

Action: The limitations of the revised paper now state that GYRO's advantage is specific to small per-step replay. The class-order, architecture and replay-budget results are in App. K.6, the objective and view comparison in App. I, and Osiris in App. K.7.

Thank you again for the thorough review. We have tried to answer each of your soundness concerns with new experiments, and we narrowed our claims where the evidence required it. If you find that these responses address your concerns, we would be grateful if you would reconsider your assessment.

Reviewer_3RUDOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes GYRO, a continual self-supervised learning (CSSL) framework that employs persistent simplex frames to address representation drift and catastrophic forgetting. The authors identify a limitation of the nuclear norm objective in Maximum Manifold Capacity Representations (MMCR): its invariance to global rotations provides no fixed geometric reference for preserving representations across sequential tasks.

To address this limitation, GYRO assigns a fixed simplex equiangular tight frame to each task and blends each image centroid with its nearest prototype in the corresponding task frame before maximizing the nuclear norm. This approach introduces persistent geometric references into the existing spectral objective without requiring class labels, additional alignment losses, or frozen teachers. The paper further provides theoretical analyses of the anchored objective through deficit decomposition and frame-preserving transformations to characterize its geometric properties.

The proposed method is evaluated using ResNet-50 on Split CIFAR-100 and Split Tiny ImageNet across different task granularities. Experimental results demonstrate improved final Class-IL accuracy and reduced forgetting compared with replay-, distillation-, and regularization-based methods, particularly over longer task sequences. Additional ablation studies and representation geometry analyses further support the effectiveness of the proposed approach.

Critical strengths

1. Clear Methodological Contribution Through Persistent Geometric References in the Spectral Objective

This paper identifies the rotational invariance of MMCR as a key limitation and proposes directly incorporating task-specific simplex frames into the existing nuclear norm objective. By combining embedding centroids with fixed geometric references without requiring additional alignment losses or teacher networks, the method offers a distinct approach compared with conventional distillation- and regularization-based continual learning methods. Furthermore, Theorem 1 and Proposition 1 provide theoretical insights into the effects of anchoring on the objective and the remaining rotational symmetries, strengthening the mathematical motivation.

2. Improved Performance on Long Continual Learning Sequences and Comprehensive Ablation Studies

The proposed method demonstrates improved final accuracy and reduced forgetting on Split CIFAR-100 and Split Tiny ImageNet across different task granularities, particularly as the number of tasks increases. The ablation studies examine the effects of replay, task-specific versus shared frames, anchoring targets, and anchor strength. These results provide empirical support for the effectiveness of combining persistent geometric references with replay to improve MMCR-based continual learning.

Critical weaknesses

1. Insufficient Evidence That Rotational Invariance Is a Primary Cause of Representation Drift and Forgetting

  • The paper argues that the global rotational invariance of the nuclear norm contributes to representation drift and forgetting. However, global orthogonal transformations preserve pairwise similarities and relative manifold geometry, and therefore do not necessarily cause representation interference or forgetting.
  • Forgetting may instead arise from non-uniform representation changes across tasks.
  • The pairwise cosine similarity metric in Figure 2 is also rotation-invariant, making it insufficient to establish rotational drift as the primary cause.
  • Further analysis distinguishing global rotations from non-rigid representation deformations would strengthen the central motivation.

2. Limited Evaluation Independence Due to Test-Set-Based Hyperparameter Selection

  • Appendices G–H indicate that hyperparameters were selected using test accuracy without a separate validation split, potentially introducing selection bias despite applying the same protocol to all methods.
  • GYRO additionally selects the prototype count KK and mixing coefficient β\beta according to task granularity, raising concerns about the influence of test-set tuning on reported performance.
  • Although improvements under default settings are encouraging, validation-based hyperparameter selection or independent evaluation with fixed hyperparameters would provide stronger evidence of generalization.
Questions for authors

Q1. How Can Rotational Invariance Be Established as a Primary Cause of Catastrophic Forgetting?

  • The paper attributes representation drift to the rotational invariance of the MMCR objective. However, global rotations preserve pairwise similarities and relative manifold geometry.
  • The cosine-similarity disruption metric in Figure 2 is also rotation-invariant, making it insufficient to directly support this claim.
  • Could the authors apply Orthogonal Procrustes alignment to distinguish rotational drift from non-rigid deformation and analyze their respective contributions to forgetting?

Q2. Do the Performance Improvements Hold Without Test-Set-Based Hyperparameter Selection?

  • Appendices G–H indicate that hyperparameters are selected using test accuracy without a separate validation split, potentially introducing selection bias.
  • Could the authors verify whether the reported improvements persist when selecting KK and β\beta using held-out validation data or adopting fixed hyperparameters across all task granularities?

Q3. Are the Performance Gains Attributable to the Specific Geometry of the Simplex ETF?

  • Although the paper compares shared and task-specific frames, it remains unclear whether the regular simplex structure itself is essential.
  • Could the authors compare simplex ETF anchors with random unit-vector anchors or orthonormal reference vectors under matched prototype counts and replay budgets?
  • Such comparisons would help distinguish the contribution of simplex geometry from the general benefits of task-specific fixed references.
2 comments
AuthorsOpenReview ↗

Response to Reviewer 3RUD

Thank you for your careful and constructive review. We appreciate your note that placing fixed geometric references inside the spectral objective is a clear methodological idea, and your suggestions on how to test it more rigorously. We respond to each of your points below.

W1. Insufficient Evidence That ...

We agree that a global rotation preserves pairwise geometry and cannot cause forgetting by itself, and we have revised the "rotate freely" phrasing. Procrustes alignment shows that the rotation is real but harmless, and that forgetting is predicted by the loss of separation between tasks, not by movement. GYRO's gain comes from giving each task its own fixed reference. We kindly invite you to refer to the full analysis in our CQ2 (Response to Common Questions).

W2. Limited Evaluation Independence Due ....

Yes, the improvements hold. With one configuration (K=10, β=1/3) for both datasets and every task count, GYRO is above every baseline at 20 and 50 CIFAR-100 tasks and at 50 and 100 Tiny ImageNet tasks, and the gain over the strongest baseline is statistically significant in all four. Selecting every method on a held-out validation split gives the same picture, in line with your observation that the gains are on longer task sequences. We kindly invite you to see our response to CQ1 (Response to Common Questions) for the full table and analysis.

Q3. Are the Performance Gains Attributable ....

The gain comes from task-specific fixed references, and it is robust to the shape of the frame. As you suggested, we compared the simplex with orthonormal and random unit-vector frames at matched prototype count K, merge weight β and replay budget, in the four long-sequence settings of the paper. Below, we report Final Class-IL accuracy (%), mean ± std over 16 seeds.

CIFAR-100, 20 tasks (K=20)CIFAR-100, 50 tasks (K=5)Tiny ImageNet, 50 tasks (K=10)Tiny ImageNet, 100 tasks (K=20)
Simplex58.72 ± 0.3556.98 ± 0.4831.16 ± 0.6030.46 ± 0.73
Orthonormal58.58 ± 0.4857.01 ± 0.4931.64 ± 0.8430.41 ± 1.00
Random unit vectors58.86 ± 0.2757.08 ± 0.5530.70 ± 1.1030.02 ± 0.78

These results highlight three takeaways. First, the shape of the frame does not change accuracy at these settings. This is expected, because at these sizes the three frames have nearly the same geometry. Below, we report the angle between two prototypes of the same frame (for random frames, mean ± std over pairs), at the K and embedding dimension D of each setting.

CIFAR-100 (D=128), 20 tasks (K=20)CIFAR-100 (D=128), 50 tasks (K=5)Tiny ImageNet (D=256), 50 tasks (K=10)Tiny ImageNet (D=256), 100 tasks (K=20)
Simplex93.0°104.5°96.4°93.0°
Orthonormal90.0°90.0°90.0°90.0°
Random unit vectors90.0° ± 5.1°90.0° ± 5.1°90.0° ± 3.6°90.0° ± 3.6°

All three frames place their prototypes at or close to right angles. A regular simplex with K vertices has pairwise cosine −1/(K−1), which approaches zero as K grows, and random unit vectors in D dimensions have pairwise cosine of about ±1/√D.

Second, what matters is that the reference is specific to the task. Replacing the task-specific frames by one frame shared by all tasks costs 4.0 points (CIFAR-100, 20 tasks, K=20), in line with App. B, and anchoring stored images to another task's frame costs 3.6 points (CIFAR-100, 50 tasks).

Third, we keep the simplex as the principled default, because it is the frame for which our analysis is exact. It is the equiangular, maximally separated arrangement of K unit vectors for any K ≤ D+1. Its rows sum to zero, which gives νᵢ ≥ 0 for Theorem 1, the anchor-rank bounds of Lemma 1 are derived for it, and its symmetry group in Proposition 1 has an explicit form.

Action: This comparison is in App. K.3 of the revised paper, which states that the benefit comes from task-specific fixed references and that other frames reach the same accuracy when K ≪ D.

Thank you again for your constructive suggestions. The Procrustes analysis and the validation-based selection you proposed are now part of the revised paper. If our responses address your concerns, we would be grateful if you would consider updating your score. We are happy to answer any further questions.

AuthorsOpenReview ↗

Follow up response to Reviewer 3RUD

Thank you again for your constructive review. We have uploaded the revised paper, and it includes each analysis you suggested. It contains the Procrustes analysis with the revised wording on rotation, the results with one configuration and with validation-based selection, and the comparison of simplex, orthonormal and random frames (App. K.3). Following the discussion with Reviewer JXmq, the abstract, the contributions and the discussion now state the contribution as persistent, task-specific prototypes, which agrees with your question on the shape of the frame. These changes are marked in red. We would be glad to know whether this addresses your concerns, and we are happy to answer any further questions in the remaining time.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Response to Common Questions (1/2)

We thank the reviewers for their careful reading and constructive comments. Two concerns were shared across reviews, so we answer them here once. CQ1 covers hyperparameter selection on test accuracy (3RUD W2/Q2, JXmq W1, Me4G W2/Q1). CQ2 covers the evidence for the rotation mechanism (3RUD W1/Q1, JXmq W2, Me4G Q2). The remaining points will be answered in the individual responses.

CQ1: Do the gains hold when hyperparameters are chosen on a validation set, or kept fixed, instead of chosen on test accuracy?

We agree that selecting hyperparameters on test accuracy weakens the evaluation. As you noted all the methods were evaluated using same testing protocol. We have implemented both of your suggestions. We held out 10% of each class's training images for validation, tuned the hyperparameters for each method based on validation accuracy, and reported metrics on the 10% held-out test set across 3 seeds. We now report Final Class-IL accuracy (%), mean ± std over three seeds on two versions of GYRO: one with fixed hyperparameters kept constant across different task settings, and another with hyperparameters tuned specifically for each task setting.

MethodCIFAR-100, 5 tasksCIFAR-100, 20 tasksCIFAR-100, 50 tasksTiny ImageNet, 20 tasksTiny ImageNet, 50 tasksTiny ImageNet, 100 tasks
GYRO, fixed (K=10, β=1/3)59.91 ± 0.2158.27 ± 0.1056.04 ± 0.3127.77 ± 0.7131.24 ± 0.3229.50 ± 0.25
GYRO, tuned60.35 ± 0.1858.27 ± 0.5356.04 ± 0.3130.82 ± 1.6831.02 ± 0.3730.70 ± 0.56
LUMP61.72 ± 0.4957.47 ± 0.1953.70 ± 0.4928.89 ± 0.3526.67 ± 0.7127.42 ± 0.17
CaSSLe60.44 ± 0.2353.24 ± 0.6050.84 ± 0.3427.02 ± 0.4128.91 ± 0.2128.67 ± 0.15
ER60.39 ± 0.4556.21 ± 0.3252.67 ± 0.3121.75 ± 0.9820.66 ± 1.0721.19 ± 0.25
DER++59.72 ± 0.0955.76 ± 0.4852.68 ± 0.2422.48 ± 1.8421.58 ± 0.7022.14 ± 0.31
NGP60.07 ± 0.1753.56 ± 0.4949.16 ± 0.2325.93 ± 0.7226.60 ± 1.3224.32 ± 0.86
EWC60.24 ± 0.4753.18 ± 0.2948.68 ± 0.2529.85 ± 0.4326.68 ± 0.3524.13 ± 0.33
Fine-tune60.29 ± 0.6253.51 ± 0.3648.78 ± 0.3525.53 ± 0.6725.98 ± 0.4425.67 ± 0.15

GYRO keeps its advantage under validation-based selection, with or without tuning K and β. With a single fixed configuration, GYRO is above every baseline at 20 and 50 CIFAR-100 tasks and at 50 and 100 Tiny ImageNet tasks, and its gain over the strongest baseline is statistically significant in all four (paired test, 95% level). The choice of K and β does not drive this result: all six GYRO configurations in the grid are above every baseline configuration in these settings, and selecting on validation moves the fixed result by at most 1.2 points in either direction. LUMP is ahead at 5 CIFAR-100 tasks where there is little forgetting to prevent but GYRO's gains are on longer sequences.

Action: Tables 1 and 2 will report the validation-selected numbers, the appendix will list every grid and selected setting, and the released code will include the validation split and the selection script.

AuthorsOpenReview ↗

Response to Common Questions (2/2)

CQ2: A global rotation keeps pairwise geometry, so how can rotation cause forgetting? Can Procrustes alignment separate rotation, task-dependent movement and non-rigid change, and which one predicts forgetting?

We agree that global rotation preserves pairwise geometry and does not cause forgetting by itself. Section 3 helps clarify this, but our abstract and introduction overstate the issue by saying "representations rotate freely". We will fix this phrasing. As suggested, we ran orthogonal procrustes alignment analysis on both datasets on 3 seeds. We fitted the linear probe at the end of each task. Below, we report the accuracy at the end of training on all tasks in three ways: unchanged, after a global orthogonal map (fitted on held-out images), and after re-fitting the probe.

Probe at end of taskunchangedafter a global orthogonal mapProbe re-fitted
Tiny ImageNet (100 tasks), GYRO0.8240.6070.7710.812
Tiny ImageNet, no anchor (β=0)0.8100.5400.7180.759
CIFAR-100 (50 tasks), GYRO0.9560.6810.9100.937
CIFAR-100, no anchor (β=0)0.9420.6100.8840.921

These results highlight three key takeaways. First, representations do rotate, the unchanged probe falls almost to chance, but a single global rotation recovers 0.15 to 0.33 of this drop for every method, including GYRO. Second, this rotation is harmless because re-fitting the probe easily absorbs it. Finally, non-rigid change is the smaller part of the performance drop as re-fitting recovers only another 0.03 to 0.04 beyond the rotation, meaning only 0.01 to 0.05 is permanently lost.

To test that non-uniform changes across tasks cause forgetting, we tracked early tasks (the first 60 of 100 in Tiny ImageNet, and the first 30 of 50 in CIFAR-100). We re-fitted a probe on them at each later step. This isolates the effect from rotation and increasing class counts. We found that a task's relative movement does not predict its accuracy loss on both datasets (refer to the table below). However, the loss of separation from other tasks does predict accuracy loss, across. The harmful change is therefore loss of separation between tasks, not movement as such. GYRO successfully prevents this degradation. Accuracy on the fixed early tasks stays stable over the rest of the sequence points.

Change in an early task (Spearman ρ with its accuracy loss)Tiny ImageNet (first 60 of 100 tasks)CIFAR-100 (first 30 of 50 tasks)
How far its classes moved relative to other tasks−0.05 [−0.10, 0.00]+0.05 [−0.03, 0.12]
How much separation from other tasks it lost0.28 [0.23, 0.33]0.29 [0.21, 0.36]

Intervention tests further confirm the anchor's specific role. On CIFAR-100 with 50 tasks, removing the anchor (β=0) drops final accuracy by 4.0 points [2.5, 5.6]. Keeping the anchors but assigning stored images to a random earlier task drops it by 3.6 points [2.5, 4.8]. This shows the gain comes from giving each task its own fixed reference, not from preventing global rotation.

Action: We will update the "rotate freely" phrasing, present Fig. 2 as descriptive, and include these new analyses in the appendix.

AuthorsOpenReview ↗

Official Comment by Authors

In this comment, we summarize the discussion to aid the Area Chair in evaluating our work. For each reviewer, we list the key concerns and a summary of our response. For greater detail, we kindly refer the Area Chair and readers to our responses to the reviewers and to our Response to Common Questions (CQ1, CQ2). In the revised paper, the changes of the first round are marked in green and the changes of the second round in red.

Reviewer 3RUD (Rating: 3, Confidence: 3)

ConcernResponse
Evidence that rotation causes forgettingWe agree that a global rotation is harmless. A Procrustes analysis shows that a re-fitted probe absorbs the rotation, and that forgetting is predicted by the loss of separation between tasks (CQ2, App. K.4). The "rotate freely" wording is removed.
Hyperparameters selected on test accuracy (also raised by JXmq and Me4G)Every method is now selected on a held-out validation split. With one fixed configuration (K=10, β=1/3), GYRO is above every baseline at 20 and 50 CIFAR-100 tasks and at 50 and 100 Tiny ImageNet tasks, and the gain is significant in all four (CQ1, App. J).
Whether the gain comes from the shape of the simplexSimplex, orthonormal and random frames perform alike over 16 seeds. The gain comes from references that are fixed and specific to each task, since one frame shared by all tasks costs 4.0 points (App. K.3).

Reviewer JXmq (Rating: 1 → 2, Confidence: 3)

ConcernResponse
The explanation of forgetting is not establishedRefer to 3RUD concern 1. We also tested the anchor by intervention. Removing it costs 4.0 points on CIFAR-100 (50 tasks) and 13.5 on Tiny ImageNet (100 tasks). Drawing the frames of finished tasks again costs 2.1 and 9.3 points (5 seeds). Forgetting decreases as the weight of the anchor increases (App. K.2).
Single runs in Fig. 2 and interpolated cells in Fig. 3Both figures now use the same three seeds for every method, with error bars and no interpolated cells.
Dependence on task boundaries and replayFrames opened at regular intervals, with no knowledge of task boundaries, stay within 1.3 points of true boundaries (App. K.5). The limitations state that GYRO is a replay-based method.
Evaluation restricted in class order, architecture, objective and replay sizeWe ran the four requested sweeps (App. I, K.6, K.7). GYRO keeps its lead over five random class orders and with a larger buffer on long sequences. LUMP is ahead with two views and with 32 replayed images per step, and the paper states this.
Follow-up: the claims are broader than the evidenceWe agree, and the abstract, the contributions and the discussion now state the narrower interpretation. The contribution is persistent, task-specific prototypes, Theorem 1 describes the objective on one batch, and the results are stated for class-incremental streams with a small replay share and longer task sequences. At equal training time, two-view methods stay 2.3 and 3.4 points below GYRO.

In the comment of 17 Nov, the reviewer wrote that the additional experiments "significantly strengthen the empirical evidence that persistent task-specific anchors are useful in multi-view, replay-based continual self-supervised learning". After the revised paper reflected the narrower interpretation in the abstract, the contributions and the discussion, the reviewer raised the score from 1 to 2.

Reviewer Me4G (Rating: 3, Confidence: 2)

ConcernResponse
Causal evidence for why GYRO worksIn every method, 98 to 99% of the remaining errors assign an image to a class of another task. GYRO lowers these confusions, while accuracy within the right task is unchanged (App. K.1). See also JXmq concern 1.
Fig. 2 uses one run per methodRefer to JXmq concern 2. On the same three seeds, GYRO is ahead of LUMP by 2.30 points on CIFAR-100 (50 tasks) and by 3.93 on Tiny ImageNet (100 tasks).
Whether the gain comes from the task identityThe same task IDs do not help ER or LUMP (changes of at most 0.6 points), and wrong IDs cost GYRO 3.6 and 6.2 points. When every task contains every class, LUMP is ahead (App. K.5).

We thank the reviewers for their time and their constructive comments, and we hope that our responses have addressed their concerns. We also hope that the Area Chair finds this summary useful.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.