acceptodds
Under review as a conference paper at ICLR 2027

Preserving Plasticity in Continual Robot Learning from Pixels with Stiefel-Constrained Optimization

Abstract

A robotic agent that continually learns new tasks from pixel-based inputs must maintain the plasticity of deep neural networks that process high-dimensional observations. We show that existing methods based on regularizing neural network parameters to mitigate loss of plasticity remain insufficient in the continual visual-control settings. We show that changing the optimizer from AdamW to Skewon (Solonko et al. 2026) mitigates the loss of plasticity. The Skewon optimizer constrains the parameter matrices of the deep neural network to the Stiefel manifold, meaning their columns are orthonormal. We show that this optimizer does so by preserving the input–output Jacobian norm alongside sustained learning. We demonstrate continual learning from images for up to 100 million frames for locomotion tasks using the proximal policy optimization algorithm and Skewon. We further demonstrate its efficacy in simulated manipulation tasks. We introduce a new optimizer: Gram-Skewon, which preserves pretrained weight Gram matrices, which is applied to continual behavior cloning for manipulation tasks using the SO-101 robotic arm in the physical world and outperforms AdamW. Gram-Skewon achieves 62% success versus 37% for AdamW in the real-world task chain.

Primary area:
reinforcement learning
Keywords:
Reinforcement Learning, Continual Learning, Optimization

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_BgcsOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies loss of plasticity in continual robot learning from visual observations, and shows that using a Skewon optimizer rather than AdamW can be a good mitigation to this problem. The Skewon optimizer enforces orthogonality in DNN matrices, and therefore, regularizes the network and prevents the pathological behavior that are caused by saturated ("dead") neurons. The authors relate plasticity to norm of Jacobians, and showed that on a controlled LQR task, Skewon gives larger and a stable norm of Jacobians, while AdamW can span a wide distribution. Finally, they showed that their optimizer made improvements over AdamW on four continual locomotion settings, and two simulated manipulation chains, and a sequential behavior-cloning fine-tuning of SmolVLA’s action expert on a physical SO-101 robot.

Critical strengths
  • The paper addresses an important problem: maintaining the ability to adapt during continual learning for robots. The evaluation spans a controlled setting, simulated locomotion and manipulation, and physical hardware.
  • Improvements on the SO-100 Cup-Align task are substantial. Skewon achieved 92% compared to 3% with AdamW, while both optimizers have similar performance when learning the task from scratch.
Critical weaknesses
  • I'm not convinced that the Jacobian evidence establishes the proposed explanation of plasticity. First of all, section 3.1 is confusing because it is not the setup that later experiments use. So why mentioning this setup? The setup does not provide evidence to support their argument either; input sensitivity is not equivalent to trainability on a new objective. A larger Frobenius norm need not imply better conditioning or preservation of useful directions.
  • Stiefel optimization and Skewon are existing methods, so the contribution of this paper is limited to assess their performance in visual-control studies.
Questions for authors
  • Can you clarify the role of the setup in section 3.1? Is the Jacobian discussion intended as an intuitive motivation, an empirical hypothesis, or a theoretical explanation of the method’s effectiveness? Clarifying this distinction would help establish what the subsequent experiments are intended to validate.
  • Section 3.2 describes established Stiefel optimization methods and the existing Skewon optimizer. Could the authors state more explicitly which methodological components are new, distinguishing the application of Skewon to continual visual control from the Gram-Skewon extension? What additional insight does the empirical study establish beyond the previously observed benefits of orthogonality-based regularization?
  • Can the authors provide within-chain LQR learning trajectories, relating the Jacobian measured before adaptation to subsequent learning performance? How does the interpretation change when the Jacobian is measured on a common observation set?
4 comments
AuthorsOpenReview ↗

Response to Reviewer Bgcs

Thank you for your detailed review of our work and pushing us to clarify and improve our work. We address your concerns below.

role of the setup in section 3.1; Jacobian norm and input sensitivity are not sufficient and no theoretical results are presented

The Jacobian discussion is intended for intuitive motivation. Ours is primarily empirical work based on the previous theoretically grounded work [1,2,3]. The simple feed-forward network in Section 3.1 is used only to define the input-output Jacobian J(x)=∂F(x;W1,…,WL)/∂xJ(x) = \partial F(x; W_1, \ldots, W_L) / \partial x; in our experiments we measure the Jacobian of the policy with respect to its observation.

Building on the theory by Rosseau et al [1] we argue that maintaining the invariant property W⊤W=IW^{\top} W = I potentially improves the conditioning. We introduce this to set the stage for the correlation between the Jacobian norm and plasticity. We have updated the writing for 3.1 and lead with the following explanation:

Rosseau et al. (2026) denote the empirical neural tangent kernel (NTK) as Kθ(X)=Jθ(X)Jθ(X)⊤K_{\theta}(X) = J_{\theta}(X) J_{\theta}(X)^{\top}, where Jθ(X)=∂F(X;θ)/∂θJ_{\theta}(X) = \partial F(X; \theta) / \partial \theta is the Jacobian with respect to the parameters. The first-order change in loss under a task-gradient gg at task index τ\tau is ΔLτ=−ηg⊤Kθ(Xτ)g+O(η2)\Delta L_{\tau} = -\eta g^{\top} K_{\theta}(X_{\tau}) g + O(\eta^2), where η\eta is the learning rate, LτL_{\tau} is the loss for task τ\tau, and XτX_{\tau} is the input dataset for task τ\tau. Ensuring that the learning rate satisfies η≲1/λmax⁡(Kθ(X))\eta \lesssim 1 / \lambda_{\max}(K_{\theta}(X)), so that learning remains convergent, the worst-case decrease in loss for unit-norm task-gradient directions is:

λmin⁡(Kθ(X))λmax⁡(Kθ(X)),\frac{\lambda_{\min}(K_{\theta}(X))}{\lambda_{\max}(K_{\theta}(X))},

which is maximised exactly when λmin⁡=λmax⁡\lambda_{\min} = \lambda_{\max}, i.e. when Kθ(X)K_{\theta}(X) is isotropic. Their conclusion is that the condition number of Kθ(X)K_{\theta}(X), and not its magnitude, determines task-agnostic plasticity, since η\eta is bounded by 1/λmax⁡(Kθ(X))1 / \lambda_{\max}(K_{\theta}(X)) and learning is slowest along the directions of λmin⁡(Kθ(X))\lambda_{\min}(K_{\theta}(X)).

Their Section 2.3 connects this to the input-output Jacobian: under a task-agnostic input prior with E[XX⊤]=In\mathbb{E}[X X^{\top}] = I_n, a dynamically isometric network, one whose input-output Jacobian singular values remain concentrated near one, has an empirical NTK that concentrates near isotropy in expectation.

We agree that a larger Jacobian norm need not imply better conditioning. We therefore additionally report the condition number κ(J)=σmax⁡(J(x))/σmin⁡(J(x))\kappa(J) = \sigma_{\max}(J(x)) / \sigma_{\min}(J(x)) over the non-zero singular values for the LQR chains (Figure 7 in the Appendix). Under AdamW, κ(J)\kappa(J) rises along the chain while σmax⁡(J(x))\sigma_{\max}(J(x)) stays nearly constant, so σmin⁡(J(x))\sigma_{\min}(J(x)) approaches zero and ∥J(x)∥F\|J(x)\|_F declines with it. Moreover, we show how the two change together: as the condition number increases the Jacobian norm decreases. We also test trainability on a new objective directly: checkpoints from the Skewon chain adapt to a held-out task far better than AdamW checkpoints (Table 2).

We are now intuitively explaining this in Section 3.1. We do not provide detailed theory as ours is not theoretical work.

which methodological components are new and what do we establish beyond the previously observed benefits of orthogonality-based regularization

We have two main methodological contributions: We present first end to end results for continual learning from frames for MuJoco and ManiSkill tasks with tens of millions frames. This empirical calibration of hyper-parameters and application of a new optimizer is substantial and opens up avenues of research as well as applications. We have also introduced chains of tasks for robotic control and continual learning. These are first of their kind empirical results. We extend the Skewon algorithm to present Gram-Skewon optimizer for the pre-trained VLAs (Section 6) where we cannot ensure the property: W⊤W=IW^{\top} W= I. This optimizer ensures that for a large pre-trained network the quantity W⊤WW^{\top} W remains invariant even if it is not equal to identity. We also show that this works on a real robot.

Not only are these important empirical results but we also present a novel optimization based perspective for maintaining plasticity.

Additionally, we observe that maintaining a hard constraint performs better than regularisation, as shown by our experiments. The revised Table 1 compares two ways of using orthogonality on the same chain: Parseval [3] regularises towards it, and Skewon enforces it as a hard constraint. While our work builds on the results of Chung et al [3], we see stark improvements in comparison by introducing a hard constraint. These are now detailed in the introduction (lines 70 to 91), the empirical section, and the entire discussion section. We hope this addresses your concern.

AuthorsOpenReview ↗

Continued Response to Reviewer Bgcs

within-chain LQR learning trajectories relating the Jacobian measured before adaptation to subsequent learning performance

We show how the LQR learning trajectories and the Jacobian norm, over a common observation set, decays as the return also decays for Adam optimizer (figure 7 in Appendix of in the latest version). We also present the condition number and show how the input-output Jacobian becomes ill-conditioned as training progresses under AdamW. This does not happen for Skewon. Note that this need not always be true meaning for some randomly sampled task chains the performance does not drop as the chain progresses with AdamW optimizer. This is simply because some LQR tasks are too easy to solve and their performance need not degrade even if they are later in the chain.

We hope our response and new experiments have addressed your concerns. Please let us know.


References:

[1] Andries Rosseau, Robert Muller, and Ann Nowe. Preserving plasticity in continual learning via dynamical isometry.

[2] Arslan Chaudhry, Naeemullah Khan, Puneet K. Dokania, and Philip H. S. Torr. Continual learning in low-rank orthogonal subspaces.

[3] Wesley Chung, Lynn Cherif, David Meger, and Doina Precup. Parseval regularization for continual reinforcement learning.

Thanks for addressing my questions, but I'm not fully convinced by the author's response.

The re-write of section 3.1 seems to be a paraphrase of Rosseau et al. (2026). I still don't think a smaller condition number or a larger norm of the Jacobian implies higher plasticity. This is too general a statement to provide any scientific insight.

With that, the novelty of the paper seems to be limited to testing a pre-established optimizer (with slight variants) on some new tasks designed by the authors. I will keep my score.

AuthorsOpenReview ↗

Response to Reviewer Bgcs

Thank you for your continued engagement with our work.

Jacobian and Section 3.1: We present the work of Rosseau et al. (2026) in the background section. This is intended to be informative of the past work and keep our paper self contained. It is not a contribution of ours and our conclusions do not depend on it. We will remove the Jacobian line from the abstract and also shorten Section 3.1 in response.

The scientific insight is maintaining a hard Stiefel constraint mitigates loss of plasticity. This is building on past work as does most research.

Novelty: We respectfully disagree with you on this. The Gram-Skewon is not a slight variant. It is a new contribution for adapting pre-trained networks. Skewon requires W⊤W=IW^{\top} W = I, so it cannot be applied to a pretrained network without altering it. Gram-Skewon holds W⊤WW^{\top} W at its pretrained value. It requires a different inverse-free retraction (Appendix D). This is the setting of real-world fine-tuning. It also improves final-task success from 37% to 62% on a physical robot

We present a comprehensive results on tasks designed to test plasticity, including a real robot. These are first of their kind results for robotic control from pixels and are building towards solving an important problem: agents that can continually learn and adapt in ever-changing environments.

We hope you will reconsider your score in light of these clarifications.

Reviewer_oCMfOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper first studies the plasticity loss in continual robot learning from visual observations, then replaces AdamW with Skewon, which preserves the input-output Jacobian norm and performs better than Parseval regularization. Finally the paper introduces a new optimizer Gram-Skewon, which preserves pretrained weight Grame matrices without projecting onto Stiefel. Experiments over multiple simulated and real-world benchmarks support the claims.

Critical strengths

The paper is clearly presented and easy to follow. This paper identifies that hardconstraint in the optimizer performs better than a soft penalty in the loss. The Gram-preserving manifold is a novel way for pretrained fine-tuning to preserve the learned singular values of pretrained weights while allowing their singular directions to change.

I really appreciate the extension from small scale DNN to larger scale VLA like SmolVLA in the real-world, which shows the potential of the proposed Gram-Skewon optimizer. The attached code is very helpful.

Critical weaknesses

Though FIgure 6 shows clear advanced return of the proposed method, the higher return not necessarily means the plasticity has been retained considering that the learning task keeps changing. To further clarify this, I suggest hold out one task and never use it in the main learning chain. When performing the continuous learning experiments, save checkpoints at a fixed interval and finetune alll the checkpoints from different methods and ages on the held out task. By measuring the performance on the same held out task, we will have a relatively fair indirect indicator of the plasiticy.

Questions for authors

No

3 comments
AuthorsOpenReview ↗

Response to Reviewer oCMf

Thank you for your engagement with our work and appreciating the presentation and the application to large scale VLAs. We address your concern below:

Testing on one hold out task

The hold out task test for the SO 100 checkpoints, in simulation, confirms that Skewon mitigates loss of plasticity. Thank you for pointing this out. We train an agent on a hold out task (pick ball) AdamW using the last checkpoint from the Skewon chain of tasks and it retains better plasticity compared to the checkpoints from AdamW. The table below is now in the main body as Table 2:

Chain checkpointContinual optimizerStackCup-AlignSort
SkewonAdamW.69 ± .04.65 ± .08.71 ± .08
SkewonSkewon.65 ± .04.65 ± .06.56 ± .10
AdamWAdamW.13 ± .10.13 ± .04.06 ± .06

Here each of the last three columns denotes the success rate on the pick ball task from the task checkpoint it was trained on. The chain checkpoint column denotes the optimizer of the chain and the continual optimizer denotes the optimizer that was used to further chain this task. Note that we cannot further train the AdamW chain checkpoint using the Skewon optimizer because Skewon requires the matrices satisfy the condition W⊤W=IW^{\top} W = I.

We hope this addresses your concern.

The new results addressed my concern. Thank you.

AuthorsOpenReview ↗

Response to comment by Reviewer oCMf

Thank you for your continued engagement with our work.

Since the new results have addressed your concern, we would be grateful if you would consider updating your score to reflect this.

Reviewer_6J73OpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies loss of plasticity in continual robot learning from pixel observations and argues that AdamW, even with standard mitigations (layer norm, L2, reward normalization, Parseval regularization), does not prevent it. The authors propose replacing AdamW with Skewon, an existing optimizer that constrains weight matrices to the Stiefel manifold, and link its benefit to preserving the input–output Jacobian norm. They evaluate on a pixel-based LQR toy problem, continual MuJoCo locomotion chains (up to 100M frames), and ManiSkill manipulation chains, reporting consistent gains over AdamW and Parseval on later tasks. They also introduce Gram-Skewon, which preserves the Gram matrices of pretrained weights, and use it to continually fine-tune SmolVLA on a physical robot arm, improving final-task performance.

Critical strengths
  • The authors present a practical, well-executed empirical study in a useful setting. Most plasticity work uses proprioceptive states or toy classification. This paper covers pixel-based control across a controlled LQR testbed, four MuJoCo locomotion chains, two ManiSkill manipulation chains and a physical robot. The baselines for AdamW are adequate, and the standard interventions are tested (layer norm, L2, reward normalization, Swish) with a reasonable sweep. Parseval (regularization towards orthogonal weights) is included as the closest regularization-based alternative. The gains on later tasks are significant and consistent over the tasks (e.g. 92% vs 3% on SO-100 Cup-Align, 94% vs 58% on Panda Pyramid). The from-scratch controls help separate plasticity from task difficulty.

  • Gram-Skewon is a nice and useful contribution. Forcing pretrained weights onto the Stiefel manifold would destroy the pretrained model and what it learned, and the authors avoid this well. Their approach is directly applicable to fine-tuning VLAs, their derivation is careful, and the details (spectrum floor for badly conditioned layers, no inverses after initialization) are clearly targeted toward practical execution.

Critical weaknesses
  • Conceptually, the contribution is in some dimensions modest, and I believe the paper in some ways does not sufficiently show what actually provides most of the gains. Skewon is existing work, Stiefel-constrained continual learning goes back to Chaudhry et al. (2020), and the Jacobian–plasticity connection is already made by Rosseau et al. (2026). An informative missing comparison would be plain Muon, or other orthogonalized updates without the manifold constraint. Without it, we can’t tell whether the hard Stiefel constraints matter specifically on top of Muon. The Jacobian evidence (Figs. 5 and 7) is correlational, and the explanation would be more convincing with an intervention that influences the Jacobian norm directly. Notably, on Ant-Fluid, Parseval retains 0.99x of its initial Jacobian norm yet performs far worse than Skewon, suggesting the norm is not the full explanation?

  • The evaluation is less thorough than the headline numbers suggest, especially on the real robot. That experiment is a single chain with a single seed. Trial counts differ between methods (e.g. Stack: 50 vs 40), and AdamW’s learning rate was picked as "the largest that trained stably on the first stage" which is light tuning for the main baseline. The authors record forgetting (action MAE on earlier tasks) but don't report it, and it is unclear whether Table 2 evaluates the end-of-chain policy on all tasks or each stage's own checkpoint. If the latter, it remains unclear whether Gram-Skewon trades retention for plasticity. In MuJoCo, training was "stopped after an observed separation," which is an outcome-dependent stopping rule and should be justified or replaced with a fixed budget. Comparisons are also not compute-matched despite the roughly 1.4–2x overhead.

  • I also feel the prior work on pixel-based control in continual learning, e.g. in the atari setting, was not addressed.


My current stance is weak acceptance: the conceptual contribution is modest, since Skewon and Stiefel-constrained continual learning already exist. However, the paper makes good practical use of them in an important and underexplored setting: pixel-based continual robot learning, including a physical robot. The AdamW baselines and empirical comparison are convincing enough, and Gram-Skewon is a useful new contribution for adapting pretrained models. My main concern is that the source of the gains is not isolated completely and that the real-robot evaluation is thin. I would raise my score if the authors add an unconstrained Muon ablation and clarify the real robot protocol (what Table 2 evaluates, number of seeds, forgetting results).

Questions for authors
  • Can you compare against Muon or another orthogonalized update without the manifold constraint? This would show whether the hard Stiefel constraint matters or whether the spectral-norm update accounts for most of the gain.

  • In Table 2, is the end-of-chain policy evaluated on all five tasks, or is each task evaluated with the checkpoint from its own stage? Can you also report the forgetting curves (action MAE on earlier tasks) and, ideally, results over more than one seed or chain? Could AdamW be given a comparable learning-rate sweep in this setting?

  • On Ant-Fluid, Parseval nearly preserves its Jacobian norm but performs much worse than Skewon. How do you reconcile this with the claim that Jacobian norm explains plasticity? Have you tried an intervention that changes the Jacobian norm directly, to test whether the link is causal?

  • Training in Fig. 6 was stopped “after an observed separation.” What was the criterion, and do the conclusions hold under a fixed, pre-specified budget? Do the advantages persist when AdamW gets matched wall-clock compute?

  • In both ManiSkill chains, Skewon lags AdamW on the first task (Grasp). Is this slower early learning, a sign of under-tuning, or an inherent trade-off of the constraint? Relatedly, would Gram-Skewon also work in the RL experiments, starting from a non-orthonormal initialization, so that the effect of orthonormality can be separated from that of preserving the spectrum?

Smaller remarks/writing:

  • Figure 8 overlaps part of the text of Figure 7, in the appendix.
8 comments
AuthorsOpenReview ↗

Response to Reviewer 6J73

Thank you for your detailed review of our work and your belief in our work. We address your concerns below.

we can’t tell whether the hard Stiefel constraints matter specifically on top of Muon

Muon also degrades in performance, as shown by our new experiments. We have now added comparisons with Muon for the ant fluid variants (Figure 6 (b)) and for the SO-100 tasks (Table 1(a)). We are running additional experiments to further validate this (Panda and 3 other MuJoCo tasks) and we will have them for the new version. For now, it concludes that the Stiefel constraint mitigates loss of plasticity as observed in previous work [1,2] and an unconstrained orthogonal update such as Muon need not preserve plasticity.

The Jacobian evidence (Figs. 5 and 7) is correlational

We agree that this evidence is correlational. Rosseau et al. (2026) present a more causal structure and argue that a well-conditioned input–output Jacobian mitigates loss of plasticity, and support this with a first-order argument rather than a formal theorem (their Section 2.2, Eqs. 3–5). We briefly present it here:

Building on the theory by Rosseau et al [2] we argue that maintaining the invariant property W⊤W=IW^{\top} W = I potentially improves the conditioning. We introduce this to set the stage for the correlation between the Jacobian norm and plasticity. We have updated the writing for 3.1 and lead with the following explanation:

Rosseau et al. (2026) denote the empirical neural tangent kernel (NTK) as Kθ(X)=Jθ(X)Jθ(X)⊤K_{\theta}(X) = J_{\theta}(X) J_{\theta}(X)^{\top}, where Jθ(X)=∂F(X;θ)/∂θJ_{\theta}(X) = \partial F(X; \theta) / \partial \theta is the Jacobian with respect to the parameters. The first-order change in loss under a task-gradient gg at task index τ\tau is ΔLτ=−η g⊤Kθ(Xτ) g+O(η2)\Delta L_{\tau} = -\eta\, g^{\top} K_{\theta}(X_{\tau})\, g + O(\eta^2), where η\eta is the learning rate, LτL_{\tau} is the loss for task τ\tau, and XτX_{\tau} is the input dataset for task τ\tau. Ensuring that the learning rate satisfies η≲1/λmax⁡(Kθ(X))\eta \lesssim 1 / \lambda_{\max}(K_{\theta}(X)), so that learning remains convergent, the worst-case decrease in loss for unit-norm task-gradient directions is

λmin⁡(Kθ(X))λmax⁡(Kθ(X)),\frac{\lambda_{\min}(K_{\theta}(X))}{\lambda_{\max}(K_{\theta}(X))},

which is maximised exactly when λmin⁡=λmax⁡\lambda_{\min} = \lambda_{\max}, i.e. when Kθ(X)K_{\theta}(X) is isotropic. Their conclusion is that the condition number of Kθ(X)K_{\theta}(X), and not its magnitude, determines task-agnostic plasticity, since η\eta is bounded by 1/λmax⁡(Kθ(X))1 / \lambda_{\max}(K_{\theta}(X)) and learning is slowest along the directions of λmin⁡(Kθ(X))\lambda_{\min}(K_{\theta}(X)).

Their Section 2.3 connects this to the input-output Jacobian: under a task-agnostic input prior with E[XX⊤]=In\mathbb{E}[X X^{\top}] = I_n, a dynamically isometric network, one whose input-output Jacobian singular values remain concentrated near one, has an empirical NTK that concentrates near isotropy in expectation.

We agree that a larger Jacobian norm need not imply better conditioning. We therefore additionally report the condition number κ(J)=σmax⁡(J(x))/σmin⁡(J(x))\kappa(J) = \sigma_{\max}(J(x)) / \sigma_{\min}(J(x)) over the non-zero singular values for the LQR chains (Figure 7 in the Appendix). Under AdamW, κ(J)\kappa(J) rises along the chain while σmax⁡(J(x))\sigma_{\max}(J(x)) stays nearly constant, so σmin⁡(J(x))\sigma_{\min}(J(x)) approaches zero and ∥J(x)∥F\|J(x)\|_F declines with it. Moreover, we show how the two change together: as the condition number increases the Jacobian norm decreases. We also test trainability on a new objective directly: checkpoints from the Skewon chain adapt to a held-out task far better than AdamW checkpoints (Table 2).

We are now intuitively explaining this in section 3.1 as well. We do not provide detailed theory as ours is not theoretical work.

Unequal trial counts in Table 2

This is so because these are tasks of different complexities and therefore have different variance. For example, for sort there are 3 cube colors and 2 cup color combinations therefore 6 combinations= 60 trials. Whereas for the cup task there are only 4 cube colors therefore we have 40 trials. We test each variant of the task an equal number of times i.e. 10.

In MuJoCo, training was "stopped after an observed separation," which is an outcome-dependent stopping rule and should be justified or replaced with a fixed budget. Comparisons are also not compute-matched.

Our wording was unclear and we have revised it. The protocol is as follows.

AuthorsOpenReview ↗

Response to Reviewer 6J73 Continued

Stopping rule. Within a variant, every method (AdamW, Skewon, Parseval, and now Muon) receives an identical data budget: 5M frames per task, the same number of tasks, and the same task-switch points. No method is stopped earlier or later than another. The only quantity chosen after seeing results was the chain length of each variant. Chains are run in 25M-frame segments, and we stopped adding segments once the methods had separated: 50M frames for Ant-Fluid, 75M for Ant-Friction, and 100M for the two humanoid variants. We agree this choice is outcome-dependent. It does not affect the comparison at any given frame count, because Figure 6 shows the full curves. At a common pre-specified budget of 50M frames (10 tasks), the length of our shortest chain, Skewon outperforms both AdamW and Parseval on all four variants.

Compute matching. Skewon costs 1.37 X AdamW’s wall-clock time per frame, and we now state this in the main text as a cost of the method. We match data rather than compute for two reasons. First, in continual RL the frame count determines when the task changes, so the two budgets cannot be matched at once. Equal wall-clock would mean giving AdamW either more frames per task or more tasks, and in both cases it would no longer be solving the same problem as Skewon. Second, AdamW’s shortcoming does not require more training. Its performance relative to the other methods declines as the chain progresses; on Ant-Fluid its normalized return falls to 0.010 ± 0.003 by the end of the chain. Loss of plasticity is a failure that grows with training, which is why comparing at equal samples is the standard protocol in this literature [1,2,3,4].

prior work on pixel-based control in continual learning

We have now added this in the related work section: explaining prior work on pixel based atari [5, 6] from pixels for continual learning. While relevant these are not the same setting because the actions are discrete whereas robotic control has continuous actions. This both complicates the problem and has a different interpretation of the input-output Jacobian.

In both ManiSkill chains, Skewon lags AdamW on the first task (Grasp). Is this slower early learning, a sign of under-tuning, or an inherent trade-off of the constraint?

This is a trade-off noted by Chung et al [7], that restricting the space of matrices to the Stiefel manifold might reduce the expressive capacity and therefore the rate of reaching optimality for the agent. For context, we did test multiple learning rates and entropy coefficients (as noted in Section C) but we consistently found that Adam outperforms Skewon on the first task. We believe this task specific: for example, when trained from scratch Skewon outperforms on stack and sort and matches the performance on place and cup-aling tasks. We believe this is subject of further study.

would Gram-Skewon also work in the RL experiments, starting from a non-orthonormal initialization, so that the effect of orthonormality can be separated from that of preserving the spectrum

Yes it would work from a non-orthonormal initialization but Gram-Skewon is less compute efficient than Skewon. Intuitively, when the optimizer maintains the constraint W⊤W=IW^{\top} W =I the gradient update has a closed form solution [2] and is more compute efficient than with an arbitrary W⊤WW^{\top} W. Therefore, Gram-Skewon is introduced as a necessity for continually adapting large pre-trained networks. Our work builds on past work which argues of a spectrum with all eigenvalues being 1 [1], and further studies are required to understand the broader effect of the spectrum on the plasticity similar to [4].

We hope these changes and explanation address your concerns.

AuthorsOpenReview ↗

References for the Response to Reviewer 6J73


References

[1] Arslan Chaudhry, Naeemullah Khan, Puneet K. Dokania, and Philip H. S. Torr. Continual learning in low-rank orthogonal subspaces.

[2] Andries Rosseau, Robert Muller, and Ann Nowe. Preserving plasticity in continual learning via dynamical isometry.

[3] Shibhansh Dohare, J Fernando Hernandez-Garcia, Qingfeng Lan, Parash Rahman, A Rupam Mahmood, and Richard S Sutton. Loss of plasticity in deep continual learning

[4] Arjun Prakash, Naicheng He, Kaicheng Guo, Saket Tiwari, Ruo Yu Tao, Tyrone Serapio, Amy Greenwald, and George Konidaris. Spectral collapse drives loss of plasticity in deep continual learning. In Proceedings of the 43rd International Conference on Machine Learning, 2026.

[5] LOSS OF PLASTICITY IN CONTINUAL DEEP REINFORCEMENT LEARNING, Zaheer Abbas, Rosie Zhao, Joseph Modayil, Adam White, Marlos C. Machado

[6] The dormant neuron phenomenon in deep reinforcement learning. Ghada Sokar, Rishabh Agarwal, Pablo Samuel Castro, and Utku Evci.

[7] Wesley Chung, Lynn Cherif, David Meger, and Doina Precup. Parseval regularization for continual reinforcement learning.

[8] Mustafa Shukor, Dana Aubakirova, Francesco Capuano, Pepijn Kooijmans, Steven Palma, Adil Zouitine, Michel Aractingi, Caroline Pascal, Martino Russi, Andres Marafioti, et al. Smolvla: A vision-language-action model for affordable and efficient robotics

[9] π0.5\pi_{0.5} a Vision-Language-Action Model with Open-World Generalization, Physical Intelligence

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary

Dear Area Chair and Reviewers,

Thank you for your hard work and engagement with our paper. Here is a summary of the changes made in response to the reviews.

  1. We explain how the Stiefel orthonormal constraint can improve the conditioning of the Jacobian. We show its implication for the Jacobian norm and describe it in plain English in Section 3.1.

  2. We add experiments with Muon, which orthogonalizes the update without constraining the matrices. Muon also loses plasticity and underperforms Skewon. Given the time limit, we could not run Muon on all the environments, but our initial results support our argument.

  3. We now present the performance of AdamW- and Skewon-based optimization on checkpoints from both chains on a held-out task: PickBall for the SO-100 in simulation.

  4. We clarify our contributions: we are the first to study loss of plasticity in large-scale continual robotic control from pixels, a hard constraint performs better than regularization, and we present a new optimization-based perspective for maintaining plasticity.

  5. We present within-chain results for the LQR experiments that support our Jacobian norm argument.

We hope that you will take these changes into account and update your scores.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.