acceptodds
Under review as a conference paper at ICLR 2027

Continual Learning of Dynamical Systems in Recurrent Neural Networks through Recyclable Unit Gating

Abstract

Dynamical Systems Reconstruction (DSR) aims to infer models from observed time series that reproduce a system's qualitative long-term behavior. Continual DSR (cDSR) requires learning new systems while preserving previously learned dynamics, yet even small parameter updates in recurrent models can qualitatively alter their behavior over long autonomous rollouts. We benchmark established continual learning (CL) methods spanning parameter regularization, replay, and parameter isolation on the fully trainable and interpretable Almost-Linear RNN (AL-RNN). Parameter isolation preserves earlier dynamics most effectively, but excessive task-specific allocations can rapidly exhaust a fixed-size network. We therefore introduce Continually-Recyclable Unit-Gating (CRUG), which conserves capacity through compact allocation and forward transfer. Differentiable gates trained with an -based penalty select task-specific units, while unused units are recycled for subsequent tasks. Directed connections allow later tasks to reuse earlier representations without affecting the dynamics of previously committed units. CRUG achieves the strongest reconstruction–capacity trade-off among the tested methods with zero forgetting and reliably learns a heterogeneous sequence of nonlinear and chaotic systems. Furthermore, we show that forward transfer is more pronounced and useful when tasks share similar underlying dynamics. Lastly, we demonstrate that CRUG's advantages extend beyond autonomous cDSR to sequential cognitive tasks.

Primary area:
learning on time series and dynamical systems
Keywords:
Continual Learning, Dynamical Systems Reconstruction, Recurrent Neural Networks, Piecewise-Linear RNN, Catastrophic Forgetting, Forward Transfer, Sparsity and Gating, Nonlinear Dynamics

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_6N89OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces Continually-Recyclable Unit-Gating, a procedure for training ReLU-like RNNs in a continual learning domain. The method involves explicitly allocating units to specific tasks encountered using learnable gating parameters as an approximate L0 penalty. To avoid catastrophic forgetting, they enforce that units assigned to earlier tasks can provide input to units assigned later, but not vice versa. Using tasks drawn as sequences of synthetic dynamical systems, commonly studied cognitive RNN tasks, and some real data, they show that CRUG achieves competitive performance in DSR continual learning across many domains.

Critical strengths

The paper is well-organized, technically sound, and reasonably easy to follow. The authors demonstrates the efficacy of the proposed method across a fairly comprehensive set of benchmarks and baseline comparisons.

Critical weaknesses

The proposed method seems quite bespoke and overengineered, and by design only works for networks with threshold-linear units. Given the number of moving parts in the setup, I think some ablations would be useful to identifying what about the proposed method is doing the heavy lifting in continual learning performance. For instance, how important was hard gradient masking to prevent updates to previous units? Is it possible that some intermediate level of soft masking might perform even better?

Questions for authors
  • If I am understanding correctly, the gating introduced here is applied symmetrically, in that suppressing a unit's input to other units also suppresses its own updateability. Is there any reason why separate gatings are not used for input and output?
  • Why are encoder and decoder weights left untrained?
  • The networks are trained with continuous gating parameters, but at the end of each task, these are thresholded. How come this thresholding process doesnt cause a mismatch in model predictions during training and after the thresholding that needs to be calibrated?
  • Eq 3: is outer supposed to be max? otherwise gi≥1g_i \geq 1.
4 comments
AuthorsOpenReview ↗

Comment 1:

We thank the reviewer for the positive assessment and the precise technical questions, and we appreciate that the reviewer finds the paper clear, technically sound and comprehensively evaluated. We address each point below.

W1a (threshold-linear units): CRUG does not rely on threshold-linear units. The gates (Eq. 4) and gradient masks (Eq. 6) act on rows and columns of the recurrent parameters, independently of the activation function, so CRUG applies to any RNN whose units interact through weight matrices. To demonstrate this, we applied CRUG to a 160-unit LSTM on the same four-system sequence with the same protocol (new Appx. A.11). Trained sequentially without a continual learning method, the LSTM suffers catastrophic forgetting. With CRUG, it retains all four systems and reaches an overall DstspD_{stsp} of 2.14, even better than interleaved training (3.07), while committing 103 of 160 units.

W1b. Design of CRUG and ablations

The components of CRUG follow from one requirement of DSR: the recurrent dynamics of previously learned systems must remain exactly intact, because small changes to recurrent parameters can qualitatively alter the attractor generated during autonomous simulation, even when one-step predictions remain nearly unchanged (Eisenmann et al., 2023; NeurIPS). To test which components are needed for this, we followed the reviewer’s suggestion and added ablations of each of CRUG’s four components: allocation, commitment, protection and forward transfer (new Experiments section 4.3, Appendix A.13 and A.16, Figs. 12–15).

Hard versus soft masking (gradient protection). As asked by the reviewer, we replaced the hard gradient mask by a soft mask that scales the gradients of committed units by ϵ∈{0.001,0.01,0.1,1}\epsilon\in\lbrace 0.001, 0.01, 0.1, 1\rbrace, with everything else unchanged (Fig. 14). No intermediate level of masking worked: only hard masking (ϵ\epsilon=0) preserved earlier systems. Already at ϵ=0.001\epsilon=0.001, earlier systems deteriorated gradually, and for ϵ\epsilon >= 0.01 they were lost almost immediately once training on the next task started. The following table summarizes all further ablations.

ComponentAblationResult
(i) AllocationPenalty strengths for linear and ReLU unitsTrade-off between reconstruction and committed units
L0L_0 gates replaced by an L1L_1 unit penalty with activity-based pruning (L1-CRUG), all else identicalNo configuration learned the full sequence; configurations that fit the first task commit 84-98% of total capacity (vs. 12% for CRUG)
(ii) CommitmentThree stage evaluations: (1) before thresholding, (2) after thresholding and (3) after the full sequenceNo change across stages
Gate distribution before thresholdingAlmost no gates between 0 and 0.5 (the threshold)
Thresholds 0.1 to 0.9Comparable for 0.1 to 0.7; degraded for 0.9
Absorbed gate values replaced by binary gatesOverall Dstsp≈D_{stsp}\approx 14 vs. 1.43
(iii) ProtectionGradients of committed units scaled by ϵ∈{0.001,0.01,0.1,1}\epsilon\in\lbrace 0.001, 0.01, 0.1, 1\rbraceNo forgetting only for $\epsilon=$0; gradual forgetting at 0.001; immediate forgetting for ϵ\epsilon≥ 0.01
(iv) Forward transferNo transfer vs. unregularized, L0L_0, L1L_1, L2L_2 transferBetter trade-off for related systems; no clear advantage for heterogeneous systems
Transfer connections removed after training, four task groups of graded similaritySignificant degradation of nearly all tasks that received transfer; transfer share increases with task similarity
Reference: separate modelsSmallest independent AL-RNNs that match CRUG’s per-task reconstruction quality206 vs 63; 166 vs 98 and >336 vs. 108.5 units
Independent AL-RNNs with CRUG’s per-task unit countsCRUG better in almost all cases

In brief, the L0L_0 gates were required for compact allocation, since with an L1L_1 penalty the first task alone consumed almost the entire network. Commitment left the trained model unchanged as long as the gate values were absorbed exactly, and forward transfer was used more strongly the more dynamical structure the systems shared.

What does the heavy lifting? Three components are each necessary: hard protection prevents forgetting, exact absorption of the gate values keeps the committed dynamics identical to the trained ones, and the L0L_0 gates keep the allocation compact. Forward transfer is optional. It improves capacity efficiency when systems share dynamical structure, but CRUG also learns sequentially without it (Fig. 3).

Q1. Symmetric gating

The reviewer understands correctly: in Eq. 4, the same gig_i scales both the state update of unit ii (through AiiA_{ii}, Wi,⋅W_{i,\cdot}, Ci,⋅C_{i,\cdot}, and hih_i) and its contribution to other units (through W⋅,iW_{\cdot,i}). This is deliberate, since the gate encodes a single decision, namely, whether unit ii participates in the current task and is committed to it. With separate input and output gates, a unit with an open input gate and a closed output gate would update its state, but this state would reach neither other units nor the readout, so it could not affect the reconstruction. Conversely, a unit with a closed input gate and an open output gate would receive no input from the network. Its state would then evolve independently of the task’s dynamics and, at most, converge to a constant, whose effect on other units their bias terms already provide. A unit therefore contributes only when both gates are open, and the decision to keep it would depend on their product, which is again one gate per unit. Separate gates would also add no expressivity, because the strengths of a unit’s incoming and outgoing connections are already learned freely by the corresponding rows and columns of WW (and by AA, CC, and hh). Whether a unit’s parameters are frozen is governed separately by the gradient masks (Eq. 6), which set the gradients of all parameters updating committed units to zero and leave those of free units unchanged.

Q2. Fixed encoder and decoder

Keeping the encoder and decoders fixed is a deliberate design choice. Our goal is to show that CRUG’s performance arises from how it allocates, protects and reuses capacity in the recurrent dynamics. Fixing the encoder and decoders places all learned structure in the latent dynamics, the component CRUG acts on. Each system’s dynamics must be learned and represented by the recurrent latent model. Trainable encoders or decoders could partly compensate for limitations or interference in the latent dynamics, which would make it harder to attribute the observed retention and reconstruction quality to the proposed method.

CRUG learns and retains all tasks that way, also keeping the number of trainable parameters smaller. Overall though, our goal here was to isolate continual learning in the latent dynamics. We now make this design choice more explicit in Appendix A.1.

Q3. Thresholding of continuous gates

Thresholding leaves the trained model unchanged, for two reasons. First, retained units (gig_i> τg\tau_g) are not rounded to one: their learned gate values are absorbed into the stored weights (Eqs. 10–11), so their effective parameters after commitment are identical to those used during training. Retained units therefore behave after commitment exactly as they did during training. Second, released units (gig_i <= τg\tau_g ) contribute nothing during training: the stretched-and-clipped gates (Eq. 3; Louizos et al., 2018) allow exact zeros, and the capacity penalty drives unneeded gates toward this limit. Since almost all of these gates are already 0 during training, the corresponding units did not contribute to the dynamics before commitment either, and releasing them does not change the model.

We now also verified this empirically. Across all three DSR task sequences, 65–92% of the gates per task equal 0 exactly. Only 4 of 12,295 released gates (over all runs on the three DSR task groups) are nonzero (new Fig. 13B). Accordingly, DstspD_{stsp} and DHD_H remained largely unchanged between the model before and after commitment, which confirms that commitment does not alter the reconstruction (new Fig. 13A).

Thresholds up to 0.7 yield the same or very similar reconstruction quality, so the result does not depend on the exact choice of threshold. Performance degrades only at τg=0.9\tau_g = 0.9, which also releases units with gate values between 0.5 and 0.9, and when the absorbed gate values are replaced by binary gates (both p = 0.002; new Fig. 13D). This shows that the absorption of gate values is what keeps the trained and the committed model consistent, so no calibration step is needed.

Q4. Eq 3

We thank the reviewer for checking this carefully. Eq. 3 is correct as written:

gi=min⁡(1,max⁡(0,σ(log⁡αi)(ζ−γ)+γ)).g_i = \min\left(1, \max\left(0, \sigma(\log\alpha_i)(\zeta-\gamma)+\gamma\right)\right).

The outer operation is a minimum and the inner one a maximum. With (γ,ζ)=(−0.1,1.1)(\gamma,\zeta)=(-0.1, 1.1), the stretched sigmoid sˉi=σ(log⁡αi)(ζ−γ)+γ\bar{s}_i =\sigma(\log\alpha_i)(\zeta - \gamma) + \gamma takes values in [-0.1, 1.1]. The inner maximum clips values below 0, so that qi=max⁡(0,sˉi)q_i = \max(0, \bar{s}_i) lies in [0, 1.1), and the outer minimum clips values above 1, so that gi=min⁡(1,qi)g_i = \min(1, q_i) lies in [0, 1]. The stretch beyond [0, 1] followed by clipping allows gates to take the values 0 and 1 exactly (Louizos et al., 2018). To avoid ambiguity, we now write Eq. 3 as gi=clip[0,1](σ(log⁡αi)(ζ−γ)+γ)g_i = \mathrm{clip}_{[0,1]}\left(\sigma(\log\alpha_i)(\zeta-\gamma)+\gamma\right), consistent with Appx. A.4.

Thanks to the authors for their clarifications and additional experiments; the ablation experiments are particularly helpful. I still have (admittedly subjective) reservations about the elegance of the approach, but nonetheless am happy to maintain my positive score.

We thank the reviewer for their response and positive feedback and appreciate their continued support of our work.

Reviewer_MsXiOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents CRUG, a technique for continual learning in recurrent neural networks through recyclable unit gating. The key ideas are the following. 1) Every latent unit not claimed by an earlier task gets a trainable gate between 0 and 1. The gate multiplies the unit’s update and its outgoing connections, so a gate near 0 removes the unit from the network. 2) When a task finishes, units whose gate exceeds 0.5 are committed. Their parameters are frozen, and later units can never write into them. 3) Units whose gates closed are wiped (weights reset to zero, fresh self-connection) and returned to the free pool for the next task. 4) Later tasks can read from committed units through trainable connections, which lets them reuse earlier representations (forward transfer). Connections in the reverse direction are fixed at zero, so earlier dynamics stay intact.

Critical strengths

(+) The paper proposes a simple method that learns per-task unit allocation, guarantees no interference by construction, and recycles unused capacity. (+) The paper covers synthetic chaotic systems, real gait recordings, and cognitive tasks, plus task-order robustness checks. (+) The paper benchmarks regularization, replay, and parameter-isolation methods on one fully trainable recurrent architecture. (+) The evaluation is broad. The forward-transfer experiment is well motivated.

Critical weaknesses

(-) The contributions of this paper is incremental. The design components of CRUG exist in related works. The paper proposes adaptation to these components. However, the paper’s ablation doesn’t fully justify/validate these adaptations.
For instance, relative to CLNP (one of the existing works), the novelty of CRUG is replacing the L1 penalty on weights with activity-based pruning by learned L0 gates. Authors could show an ablation where CRUG uses L1 pruning instead of gates, keeping everything the same. This would would show whether the gating explains the improvement. The appendix description does show some amount of similarity of CRUG and CNLP, but there are design aspects like regularization that remains different. Therefore, an explicit within-framework ablation is needed.

(-) Comparison with state-of-the-art is weak. HAT is the most similar method and is never run as a baseline. So is Hypernetwork-based continual learning. Furthermore, the baselines use different budget (40 for CRU/CNLP, 25 for others). The seeds selection needs clarification (10 seeds vs 5 seeds at places). For the compared baselines, it is unclear whether they are weak references or genuinely harder settings. For instance, interleaved baseline’s architecture, training budget, and tuning needs must be described. Other related concerns: the reconstruction-capacity trade-off is only compared to CLNP. In the gait experiment, CRUG is compared only with its own per-group variant, not with EWC, replay or CLNP.

(-) The paper does not show if the method can be applied to other recurrent architectures.

(-) The main claim of the paper is not verified empirically. The synthetic benchmarks use only 4 tasks on a 160-unit network. The longest sequences are 9 cognitive tasks and 12 gait subjects (on a 1024-unit network). Committed units are never released, so capacity runs out eventually. There is no scaling study showing how unit usage grows with the number of tasks, or what happens when the free pool runs low.

Questions for authors
  1. Can the authors provide component-wise ablations?
  2. How does CRUG behave over a much longer sequence?
  3. Can you compare CRUG against HAT?
4 comments

We thank the reviewer for the thoughtful review and positive remarks on CRUG’s simplicity, our broad evaluation and the forward-transfer experiment.

W1a. Contribution relative to existing components

CRUG indeed builds on several earlier ideas: unit recycling and one-directional connections from CLNP, learned unit gating from HAT, gating combined with recycling from CCGN, and the L0L_0 gate formulation of Louizos et al. (2018). The main novelty of our work lies in the problem it addresses. Continual learning (CL) in a fully trainable recurrent model that must preserve the long-term behavior of each system has, to our knowledge, not been studied before, and our work provides the first systematic benchmark of existing CL methods on it.

This problem determines how components need to be adapted and combined. Dynamical systems reconstruction (DSR) requires that a trained model, run autonomously, reproduces the attractor geometry and temporal statistics of each system. These properties are highly sensitive to the recurrent parameters: a small change can leave one-step predictions nearly unchanged and still cross a bifurcation, qualitatively altering the attractor (Eisenmann et al., 2023; NeurIPS). A CL method for DSR must therefore keep the recurrent weights exactly as they were trained. Protection of committed units prevents later tasks from changing these weights. CRUG differs in two components from CLNP that keep weights unaltered at selection and commitment. First, CRUG selects units with L0L_0 gates jointly with the reconstruction objective, which penalizes the number of active units without directly shrinking retained weights. Second, it absorbs the learned gate values into the weights, so the committed model equals the trained one. New ablations confirm that both are required (W1b).

These adjustments have large practical consequences. On our primary benchmark, CRUG thereby achieves the lowest reconstruction error of all tested methods, 1.8 to 3.4 times lower than its closest competitors (HAT, CLNP and generative replay), while committing only 63 of 160 units, whereas replay uses the full network and CLNP and HAT about twice as many units or more (Table 1). CRUG is also more capacity-efficient than CLNP and HAT at high performance across two DSR benchmarks, cognitive tasks, and sMNIST (new Fig. 6).

W1b and Q1. Within-framework and component-wise ablations.

CRUG consists of four components: (i) allocation, by learned L0L_0 gates with recycling of unused units; (ii) commitment, retaining units with gates above 0.5 and absorbing their gate values; (iii) gradient protection through gradient masks and zero connections from free to committed units; and (iv) forward transfer from committed to free units. The submission already contained ablations of (i) and (iv). We expanded these and added ablations of all other components, including the requested within-framework ablation, and a comparison with independent per-task models (new Appx. A.13 and A.16):

ComponentAblationResult
(i) AllocationPenalty strengths for linear and ReLU unitsTrade-off between reconstruction and committed units
L0L_0 gates replaced by an L1L_1 penalty with activity-based pruning (L1-CRUG), all else identicalNo configuration learned the full sequence; configurations that fit the first task commit 84-98% of total capacity (vs. 12% for CRUG)
(ii) CommitmentThree stage evaluations: (1) before thresholding, (2) after thresholding and (3) after the full sequenceNo change across stages
Gate distribution before thresholdingAlmost no gates between 0 and 0.5 (the threshold)
Thresholds 0.1 to 0.9Comparable for 0.1 to 0.7; degraded for 0.9
Absorbed gate values replaced by binary gatesOverall Dstsp≈D_{stsp}\approx 14 vs. 1.43
(iii) ProtectionGradients of committed units scaled by ϵ∈{0.001,0.01,0.1,1}\epsilon\in\lbrace 0.001, 0.01, 0.1, 1\rbraceNo forgetting only for $\epsilon=$0; gradual forgetting at 0.001; immediate forgetting for ϵ\epsilon≥ 0.01
(iv) Forward transferNo transfer vs. unregularized, L0L_0, L1L_1, L2L_2 transferBetter trade-off for related systems; no clear advantage for heterogeneous systems
Transfer connections removed after training, four task groups of graded similaritySignificant degradation of nearly all tasks that received transfer; transfer share increases with task similarity
Reference: separate modelsSmallest independent AL-RNNs that match CRUG’s per-task unit counts206 vs 63; 166 vs 98 and >336 vs. 108.5 units
Independent AL-RNNs with CRUG’s per-task unit countsCRUG better in almost all cases

Main results. The L0L_0 gates are required for compact allocation, commitment must leave the trained model unchanged, committed dynamics must be fully protected, and forward transfer is used more strongly the more structure the systems share (Figs. 12-15; independent models: Table 19, Fig. 10).

W2 and Q3. Baselines and budgets.

Comparison with HAT. We now added HAT as a baseline on both DSR benchmarks, the cognitive tasks, and sMNIST (Appx. A.7; Tables 1, 3, 11, 15, 16, and 18; Figs. 2, 6, and 9). HAT enables sequential learning with good performance, but uses far more capacity than CRUG: 89% vs. 39% on the first DSR benchmark, 82% vs. 61% for the second, 65% vs. 58% on the cognitive tasks, and 88% vs. 32% on sMNIST.

Budget: CRUG, HAT and CLNP indeed received a different number of Optuna trials. Our rationale was to give more trials to methods with more hyperparameters: they tune three hyperparameters jointly, whereas EWC, SI, XdG, ER and GR each tune a single one. Relative to the number of hyperparameters, this still favors the simpler methods, since all of their trials explore a single parameter, whereas each of the 40 trials for CRUG, HAT, and CLNP samples one combination of three.

Seeds: The second DSR benchmark used 5 seeds per Optuna trial whereas the first and the cognitive task used 10.

Interleaved baseline. It serves as a joint-training reference, showing how well the same 160-unit AL-RNN can reconstruct the four systems with access to all four throughout training. This provides a point of comparison for the continual setting, where an earlier system’s training data is unavailable once the model moves on. We used the same training setup, without a separate hyperparameter search. We added all missing details to Appx. A.5.

Reconstruction capacity trade-off: A trade-off between reconstruction and capacity is defined only for methods that allocate units to tasks. EWC, SI, ER and GR use all units for every task by construction, and XdG assigns a fixed fraction. We therefore compared CRUG to the two references that do allocate capacity. HAT and CLNP, the closest methods by design, were compared on both DSR benchmarks, the cognitive tasks, and sMNIST (Fig. 6). CRUG achieves competitive performance while using far fewer units. As a second reference, we added independent per-task AL-RNNs. For each task, we determined the smallest independent AL-RNN (trained from scratch without CRUG) that comes within 10% of CRUG’s reconstruction quality. In total, these models require 206, 166 and more than 336 units on the three DSR benchmarks, compared with 63, 98, and 108.5 units committed by CRUG. When each independent model instead receives the number of units CRUG committed to that task, it reconstructs the systems worse than CRUG in almost all cases (new Fig. 10, and Table 19).

W3. Application to other recurrent architectures.

To test whether CRUG generalizes beyond the AL-RNN, we chose an LSTM, a widely used recurrent architecture whose gated memory cells differ substantially from the AL-RNN’s linear and ReLU units. We trained a 160-unit LSTM on the same four-system sequence with the same protocol using CRUG (Appx. A.11).

Like the AL-RNN, the LSTM learns all four tasks under interleaved training but suffers catastrophic forgetting when trained sequentially. CRUG prevents this forgetting and even achieves a better overall DstspD_{stsp} than interleaved training (median 2.14 vs. 3.07) while committing 103 of 160 units. Compared with the AL-RNN configuration reported in Table 1, however, the best LSTM configuration commits significantly more units (103 vs. 63, p < 0.001) and attains a higher overall DstspD_{stsp}, although this difference is not significant (2.14 vs. 1.43, p = 0.25). Thus, CRUG is not tied to the AL-RNN, which remains the more capacity-efficient choice for DSR.

W4 and Q2. Capacity exhaustion.

As tasks are added, capacity will eventually run out, as for any sequential continual learning method. So the question is when and what happens.

Regarding when, CRUG strongly delays exhaustion through compact allocation. After the longest sequences in the submission, 9 cognitive tasks and 12 gait subjects, CRUG had committed only 58.4% and about 50% of its units. CLNP exhausted its capacity in 3 of 10 runs already on the first DSR benchmark, and in 4 of 10 runs on the cognitive tasks.

Regarding what happens when the free pool runs out, we now forced exhaustion by reducing the network to M=60 on the first benchmark and M=90 on the Lorenz family, below the 63 and 108 units CRUG originally committed (new Appx A.15, Fig. 11). When a few units are still available, the degradation is gradual: the third Lorenz task is still learned, with slightly worse reconstruction (DstspD_{stsp}=0.82 compared to 0.32 for the full model). When too few or no units are left, the fourth task cannot be learned. All earlier tasks remain unchanged, so exhaustion shows up as an inability to learn the new task, never as forgetting.

A simple remedy is to let the network grow (e.g. based on a simple performance or unit count criterion): adding 20 linear and 20 ReLU units before the fourth task restored full-model performance (Fig. 11).

Q1. See W1b.

Q2. See W4.

Q3. See W2.

I thank the authors for their detailed rebuttal. I find the paper interesting and the experimental validation thorough, and therefore maintain my positive score.

We thank the reviewer again for their positive feedback, for the time they devoted to our manuscript, and for raising the question of how CRUG scales as tasks accumulate (W4, Q2). Since our initial rebuttal, we have extended the scaling analysis which the reviewer asked for and added the results to the revision (new Appx. A.15, Fig. 16 and 11 and Table 21).

Scaling of unit usage with the number of tasks (W4, Q2)

1. Unit usage grows approximately linearly and predictably. We compared cumulative unit usage for CRUG, CLNP and HAT on the gait dataset across all configurations tested for each method (40 trials, 3 seeds). CRUC and CLNP both show approximately linear growth (with higher seed variance for CLNP), whereas HAT's usage approaches the model's capacity (Fig.16A). The growth pattern is consistent across the tested hyperparameter configurations, although stronger regularization reduces the slope or saturation rate (Fig.16D).

We attribute this difference to the allocation penalties. CRUG’s (L0L_0) penalty counts committed units in absolute terms, so the cost of a task does not depend on how many units remain. Whereas HAT attention regularizer penalizes newly used units relative to the units still free, so each task claims a roughly constant fraction of the remaining pool. Usage therefore approaches M geometrically.

2. The linear growth rate reflects structure tasks share. With the hyperparameters fixed, we ran three Lorenz family streams of increasing heterogeneity. The slope rises with dissimilarity:

  • Similar: ρ = 28→ 30→ 32→ 35→ 38→ 40→ 43→ 45→ 48.
  • Moderate: ρ = 28→ 40→ 45→ 60→ 70→ 80→ 90→ 100→ 110.
  • Dissimilar: ρ = 28 → 313 → 200 → 160 → 99.65 → 166.3 → 126.52 → 218 → 148.5 | Stream | Slope = units/task (per-seed R2) | Transfer input / total input | # Tasks learned (finalDstsp<4.0) | |---|---|---|---| | Similar | 21.2 (0.994) | 0.29 [0.27, 0.32] | 6.5 of 9 (range 6–8) | | Moderate | 27.7 (0.989) | 0.24 [0.21, 0.28] | 6 of 9 (range 5–7) | | Dissimilar | 30.3 (0.987) | 0.16 [0.14, 0.19] | 5 of 9 (range 4–5) |

Thus, the more structure the tasks share, the more each new task reuses earlier units through forward transfer, and the fewer new units it needs and therefore more tasks can fit into a single model of a fixed size.

3. Long sequences reveal how performance changes near exhaustion. We extended the gait task sequence from 12 to 30 subjects (M = 1024, P = 512) and observed three phases.

PhaseSubjectsFree ReLU units when the subject is claimedLinear units / subjectReLU units / subjectTotal units / subjectMedian test Dstsp
Plentiful ReLU1–15512 → 11316.128.344.42.68
ReLU pool shrinking16–2289 → 223.712.436.17.01
No ReLU left23–300–113.9013.915.02

With plentiful free ReLU units (subjects 1–15), performance is stable (median$\tilde{D}_{stsp}= 2.68$ ) and ReLU usage is linear (≈\approx 28 units per subject, R2=0.998R^2 = 0.998). As the ReLU pool shrinks (subjects 16–22, from ≈\approx 90 to ≈\approx 2 free units), the allocator compensates with more linear units (≈\approx 24 instead of 16 per subject); total usage grows somewhat more slowly (≈\approx 36 vs 44 units per subject) and the median performance degrades to 7.01. With no ReLU units left (subjects 23–30), performance degrades to 15.02.

Summary. CRUG’s unit usage grows linearly with the number of tasks. The slope depends on the hyperparameter configuration (pruning and transfer strengths) and on how heterogeneous the task stream is. Because in CRUG growth is linear with low variance across conditions, unlike CLNP, the point of exhaustion can be predicted in advance. This also suggests a practical allocation strategy - estimate the rate at which units are committed from the first few tasks, project when the free pool will run low, and add units as needed. When it is reached, CRUG loses plasticity but does not forget the tasks it has already learned, and adding units restores learning without disturbing committed tasks (Fig 11).

We believe these results address the reviewer’s remaining concern about whether the main claim holds over long task sequences.

Reviewer_45TVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This work explores Dynamical Systems Reconstruction (DSR) which trains an RNN so that its long free-running rollouts reproduce a system’s underlying attractor geometry and temporal statistics of the observed system. This is crucial for nonlinear and chaotic systems, where just short-horizon forecasts do not ensure that you have recovered the system dynamics due to rapidly diverging nearby trajectories.

This paper studies Continual Learning (CL) in the context of RNNs which is an underexplored problem compared with feedforward setup. CL typically suffers catastrophic forgetting (CF) in feedforward multi-task learning, this problem exacerbates in the recurrent networks since the dynamics changes are absorbed in the state transition. Thus, Continual Dynamical System Reconstruction (cDSR) requires even more careful consideration to avoid CF. It aims to learn new regimes while preserving previously learned reconstructed dynamics, along with shared dynamical representation for different regimes.

This work mainly studies the Almost-Linear RNN (AL-RNN) unit which splits an M dimensional hidden states into two different types — P non-linear (ReLU) units and M-P linear units. This is explicitly enforced in the transition function along with a diagonal gain on the hidden state term. This work proposes CONTINUALLY-RECYCLABLE UNIT-GATING (CRUG) which is an elaborate multi-stage mechanism consisting of allocations for different tasks, frozen hidden state units that preserve learned dynamics and free units that enable learning new dynamics and mechanism to update connections between free and frozen units.

  1. Gating: Each free unit gets a deterministic hard-concrete gate (the Louizos et al. parameterization, without sampling) with an L0-style penalty. ReLU units get a larger penalty than linear units.
  2. Commitment: After training on a task, units with gate > 0.5 are frozen, and their gate magnitudes are absorbed into the weights.
  3. Recycling: The remaining units are reinitialized and returned to the free pool.
  4. Forward transfer: Connections from committed to free units are trainable (with L2 regularization). The reverse direction is pinned at zero, as in CLNP.

Zero forgetting therefore holds by construction. At inference, all task subsystems run concurrently in one network, and each task is read out from its own fixed readout units.

Empirical evaluations have been conducted on various benchmark cDSR tasks where CRUG has show better performance at the end of the entire training regime while still recovering the dynamics of earlier tasks.

Critical strengths
  • Simple and easy mechanism that maps hidden state unit gating onto RNN state. One-directional masks ensure that older reconstructed dynamical subsytesm remaining committed in memory, and available for use in later interactions.
  • CRUG allocates RNN hidden state capacity based on task difficulty since there’s a penalty on using less hidden state units while maximing task performance.
  • Exhaustive details on hyper-parameters, task order permutations and other details for reproducibility
  • Strong ablation study on the plasticity and penalty on free unit allocation during a task learning in CL setup.
Critical weaknesses
  • Limited novelty compared to CLNP. Its L1 + activity pruning step is replaced with L0 penalty based gates / masks in CRUG with additional cDSR benchmarks.
  • Missing a baseline which would be effectively a separate AL-RNN per task to understand what is the ceiling on each of these individual tasks. Since forgetting is zero by construction, the real question is whether one CRUG network beats independent per-task AL-RNNs with the same total unit or parameter budget. The gait experiment has a version of this (per-group models), but the synthetic benchmarks don’t.
  • Hyper-parameter tuning via Optuna objectives seem to use the same evaluation trajectories which are reported in the results. It is unclear if there is an additional validation set on which model hyper-parameters are tuned.
  • Inconsistencies in results
    • Table 14 says CRUG uses 58.4% capacity, while Cognitive task capacity in the main text is reported as 30%
    • Appendix A.5 says 20 optuna trials with 5seeds, while Caption in Table 4 says 20 optuna trials with 10seeds
    • Did all baselines used the same number of seeds for Optuna trials including CRUG and CLNG? Sometimes CRUG and CLNG use 40 trials while others use 20 trials
  • Scale of this experiments do not stress test the allocation of the proposed method. Paper is missing the behavior on long sequences which could result in exhausted free capacity and its unclear whether the behavior degrades gracefully or there are some catastrophic failures at that point.
Questions for authors
  • What is the complexity of maintaining the frozen parameters, free hidden state units, and other management infra while performing continual learning?
  • For hyper-parameter search, did you use a separate held-out data from the reported test trajectories?
  • Since there different phases after AL-RNN is trained on a task (gated variant, converting gates to 0-1 output based on 0.5 threshold, and then absorbing these hard boundaries within the weights), do you have results at each of these separate boundaries? Since these are hard decision boundaries, it would be good to understand how these decisions affect the performance (RNN was trained with gates, do you degrade while making the decisions binary?).
  • What is the performance of AL-RNNs when trained independently on each task? i.e., no CL, but pure single task performance.
  • Since there’s a fixed number of units in AL-RNN free pool, what happens when there are more tasks (since in real world setting, you would not have much control in continual learning setup) and eventually your RNNs runs out of free units to allocate? Do you have any graceful degradation in performance?
  • Is there any analysis which show that there’s some shared dynamics recovered while learning multiple task?
4 comments

We thank the reviewer for the careful review, the accurate summary of CRUG, and the positive remarks on its simple mechanism, capacity allocation by task difficulty, reproducibility and ablations.

W1: Limited novelty compared to CLNP

While CRUG does build on CLNP, the problem of continuously learning dynamical systems with their correct long-term behavior has, to our knowledge, not been addressed before, and our work provides the first systematic benchmark of established continual learning methods on it. In dynamical systems reconstruction (DSR), a trained model, run autonomously, must reproduce each system’s attractor geometry and temporal statistics to count as a true surrogate. These properties are highly sensitive to the recurrent parameters. A small change to a recurrent update can leave one-step predictions nearly unchanged yet cross a bifurcation and qualitatively alter the attractor (Eisenmann et al., 2023, NeurIPS). A continual learning method for DSR must therefore preserve the recurrent dynamics of each learned system exactly, both while it learns and when it commits units.

CLNP, developed and evaluated for feedforward architectures, applies an L1L_1 penalty to incoming weights and then prunes units by their activity, allowing some loss in performance. Both weight shrinkage and the removal of units with low nonzero activity can alter the generated dynamics. For this reason, CRUG instead learns a gate for each eligible unit while learning the dynamics. Its L0L_0-based penalty encourages compact allocations without directly shrinking the weights of retained units. At commitment, almost all released gates are already zero, and CRUG absorbs the learned magnitudes of retained gates into the recurrent parameters. CRUG also keeps the parts of CLNP that don’t harm DSR.

While these differences in allocation and commitment may appear incremental, they have substantial consequences for DSR. On our primary benchmark, CLNP reaches DstspD_{\mathrm{stsp}}=2.61 on Van der Pol and 1.98 on Lorenz-63 right after learning each system, whereas CRUG reaches 0.15 and 0.35 (Table 1). Overall, CRUG achieves significantly lower reconstruction errors (DstspD_{\mathrm{stsp}}=1.43 vs. 3.77, p=0.0052, and DHD_H=0.09 vs. 0.17, p=0.0068), commits roughly half as many units (63 vs. 123.5), and completes all ten runs, vs. seven for CLNP. Two new ablations show why this matters. With L1L_1 regularization on units and activity-based pruning, fitting the first task used 84-98% of total capacity (vs. 12% for CRUG), and no configuration learned the full sequence (Fig. 12). Rounding retained gates to one at commitment increased reconstruction error (p=0.002, Fig. 13 and Q3).

W2 and Q4. Independent AL-RNNs per task

We performed the suggested experiments (below), but note that the submission already contained a closely related comparison: CRUG without forward transfer (Sec. 4.2, Fig. 3). Without transfer, each new system is learned by newly allocated units that receive no input from earlier tasks, and the capacity penalty lets each task choose its own size. Each system is therefore reconstructed by a separate subnetwork, like an independent AL-RNN with CRUG per task. On the heterogeneous sequence (Van der Pol, Lorenz-63, Rössler, Chua), transfer provides no clear advantage over this condition (Fig. 3A). In the Lorenz family, where all four systems share the same equations and differ only in ρ\rho, all transfer variants achieve better reconstruction–capacity trade-offs than without transfer (Fig. 3B). A single network with forward transfer therefore uses its capacity more efficiently than separate subnetworks when the systems share dynamical structure.

We now added comparisons with independent AL-RNNs trained without any continual learning mechanisms (new A.13 , Fig. 10, Table 19).

First, we matched performance: for each task, we swept network sizes up to M=160M=160 and identified the smallest AL-RNN whose median DstspD_{\mathrm{stsp}} over ten seeds comes within ±\pm 10% of CRUG’s. In total, independent models require 206 units on the first (heterogeneous) benchmark, 166166 on the second and more than 336 on the Lorenz family, compared with 6363, 9898, and 108.5108.5 for CRUG ($\sim$1.7-3.3 times higher capacity).

Second, we matched capacity: each independent AL-RNN received the number of units that CRUG committed to that task, matching the total size of the final CRUG model. Here, independent models reconstruct the systems much worse than CRUG in almost all cases (overall $D_{\mathrm{stsp}}=$11.43 vs. 1.43 on the first benchmark, 12.35 vs. 0.71 on the second and 7.46 vs. 0.64 on the Lorenz family).

Thus, one CRUG network reconstructs the systems better in almost all cases than independent-per-task AL-RNNs of the same total size, and matches their quality with far fewer units, all while determining this allocation within a single training run instead of a sweep over network sizes.

W3 and Q2. Hyperparameter tuning and test data

The reviewer is correct that the Optuna objective was evaluated on the test segment, which continues the training trajectory of each system by 20,000 steps (Appendix A.2). This segment is compared with 40,000-step autonomous rollouts of the model in terms of their state-space distribution DstspD_{stsp} and power spectrum DHD_H (Appendix A.3). This evaluation is tailored to DSR, where a surrogate model must reproduce the long-term statistics of the observed system. Since typical trajectories from the same basin of attraction converge to the same ergodic limiting distribution, any such trajectory provides an adequate test of these statistics. The evaluation also does not reward agreement with individual test points: for the chaotic systems, model rollouts and test trajectory decorrelate within a few Lyapunov times, and for all systems, the first 10,000 rollout steps are discarded before evaluation. Hyperparameter selection can therefore only exploit the sampling variability of the test segment.

To verify that this variability does not affect our results, we generated a separate test set for every task that was not used for hyperparameter selection. For the dynamical systems, it consists of a further 40,000 time steps following the evaluation segment, with identical integration, burn-in and standardization. For the cognitive tasks, it consists of 400 newly generated test trials per task. For the gait dataset we split the data into 11,400 training, 4,000 validation and 4,000 test time steps. We re-evaluated all trained models on these test sets with the original evaluation protocol. The results changed by less than 1% for cognitive-task accuracies and by less than 2.5 units in DstspD_{stsp}, and the ranking of methods remained unchanged. All tables now report results on the new test data.

W4a. Capacity on the cognitive tasks We thank the reviewer for spotting this. Table 14 (new Table 15) gives the correct values. After the nine cognitive tasks, CRUG committed 58.4% of the available units while CLNP used 94.2% among the six runs that completed the sequence (the other four exhausted their units). We corrected these values in Sec. 4.1.

W4b. Seeds on the second benchmark We thank the reviewer for pointing out this inconsistency. The second benchmark used 5 seeds per trial, as stated in Appendix A.5, and we corrected the caption of Table 4.

W4c. Tuning budgets We used ten seeds per trial on the cognitive and first synthetic benchmarks and five on the second. CRUG and CLNP (and new HAT) indeed received more Optuna trials than the other methods. Our rationale was to give more trials to methods with more hyperparameters. CRUG, HAT and CLNP tune three hyperparameters jointly, whereas EWC, SI, XdG, ER, and GR each tune a single one. Per hyperparameter, this still favors the simpler methods, as each of the 40 trials for CRUG, HAT and CLNP samples a combination of three. We now explain this rationale in A.5 and A.7 and state the trials and seeds of each method in the table captions.

W5 and Q5. Number of tasks and exhausted capacity

The referee is right that as tasks are added, capacity will eventually run out, as for any sequential continual learning method. So the question is when and what happens. Regarding when, we showed that CRUG strongly delays exhaustion through compact allocation. Regarding what happens, we now forced exhaustion by reducing the network to M=60 on the first benchmark (as it originally required 63 units) and M=90 on the Lorenz family (originally requiring 108; new Appx. A.15, Fig. 11). In both benchmarks, the model ran out of capacity on the fourth task, with 10 and 15 free units left. Two cases need to be distinguished here:

  • Few units left. On the Lorenz family, the reduced network already reached its capacity limit during the third task, with only 13.5 free ReLU units , compared with 28.5 ReLU units the full model committed to this task. It still learned this system, with slightly worse reconstruction than the full model (DstspD_{stsp} = 0.82 vs. 0.32 on the third task).
  • Too few units left. On the fourth task, neither reduced model could learn the new system, because not enough free units remained. The recurrent dynamics are then fixed, and no new dynamics can be learned. Exhaustion therefore shows up as an inability to learn the new task, never as forgetting.

As a remedy, the network can grow once the pool runs low. Adding 20 linear and 20 ReLU units before the fourth task restored full-model performance (Fig. 11). New units receive input from committed units and send none back, preserving earlier dynamics. Growth could be triggered when the free pool falls below the largest previous task allocation, when nearly all free-unit gates remain open at the end of training on a task, or the new task’s reconstruction error exceeds a tolerance after training. We discuss capacity exhaustion in Sec. 4.3 and Appx. A.15.

Q1. Cost of maintaining committed and free units

CRUG’s memory and compute costs are modest, in absolute terms and compared with the alternatives. We now quantify them (new Appx A.14, Table 20).

Memory. CRUG maintains a single recurrent network of fixed size MM. It records which units are committed to which task and adds one gate parameter per unit, both requiring O(M)O(M) memory. All tasks share the same recurrent weight matrix, so a new task never requires storing a copy of the network.

Compute. Applying the gates and gradient masks costs O(M2)O(M^2) operations per optimizer step, about 1/3200 of the training step itself (at our batch size and seq. length). Committing and recycling units costs another O(M2)O(M^2), once per task. At inference, the gates are absorbed into the weights, so a step costs the same as in a standard AL-RNN.

Measured cost. We compared CRUG with naive sequential training of the same network (AL-RNN and LSTM) on the first benchmark (M=160, 10 seeds):

ModelMethodParametersTraining time (min)Training FLOPs (101210^{12})Peak GPU memory (MiB)
AL-RNNNaive25,92033.3 ±\pm 2.381.72101.4 ±\pm 0.05
AL-RNNCRUG26,080 (+0.62%)34.9 ±\pm 1.8 (+4.80%)81.75 (+0.04%)101.5 ±\pm 0.05 (+0.10%)
LSTMNaive103,04038.5 ±\pm 1.0323.53103.0 ±\pm 0.1
LSTMCRUG103,200 (+0.16%)40.5$\pm$ 1.0 (+5.19%)323.64 (+0.03%)103.5$\pm$ 0.3 (+0.49%)

Compared with alternatives. Independent per-task AL-RNNs require 1.7 to more than 3 times as many units to match CRUG’s reconstruction quality and a size sweep for every task, whereas CRUG sizes each subnetwork within a single run. CLNP commits about twice as many units on the first benchmark, and experience replay stores the full training trajectories of previous tasks.

Q2. See W3.

Q3. Performance at commitment stages

At the end of each task, units with gates above τg\tau_g = 0.5 are retained, their learned gate values are absorbed into the stored weights, and all other units are released. The stretched and clipped sigmoid of Eq. 3 allows gates to reach 0 and 1 exactly, and the capacity penalty drives them toward these values. We performed three new analyses:

First, we evaluated DstspD_{stsp} and DHD_H for each task and seed at three stages: 1) at the end of training with continuous gates, 2) after thresholding and absorption, and 3) after the full task sequence. Both remained unchanged across stages for all four systems (new Fig. 13A). Commitment thus leaves the trained model and earlier reconstructions unchanged.

Second, we recorded the gate values at the end of each task, before thresholding, for all three DSR task sequences (new Fig. 13B-C). The gate values concentrate at 0 and 1: per task, 65–92% of the gates equal 0, 5–31% equal 1 and 1–11% lie between 0.5 and 1. Only 4 of 12,295 gates (over all runs) take values in (0, 0.5], all below 0.15 (Fig. 13B).

Third, we repeated the first benchmark with thresholds τg∈{0.1,0.3,0.7,0.9}\tau_g \in \lbrace 0.1, 0.3, 0.7, 0.9\rbrace and with binary gates in place of the absorbed gate values (new Fig. 13D). Thresholds between 0.1 and 0.7 did not change reconstruction quality significantly (all p≥ 0.28 for DstspD_{stsp} and DHD_H). Performance degraded only at τg=0.9\tau_g = 0.9 (overall $D_{stsp}\approx $8, p = 0.002). Binary gates degraded reconstruction most strongly (overall $D_{stsp}\approx $14 compared with 1.43).

Thus, commitment preserves the trained dynamics over a wide range of thresholds, and absorbing the gate values is required for it.

Q4. See W2.

Q5. See W5.

Q6. Evidence for shared dynamics

Yes. Sec. 4.2 compares forward transfer on the heterogeneous benchmark with a family of four Lorenz-63 systems (ρ=28,313,200,160\rho = 28, 313, 200, 160), which share the same equations and differ only in ρ\rho. On the Lorenz family, all transfer variants achieve better reconstruction–capacity trade-offs than without transfer, and the transfer weights are larger relative to the within-task weights than on the heterogeneous benchmark (Fig. 3B, C), where transfer gives no clear advantage (Fig. 3A).

To test this more directly, we extended this comparison to a graded series of four task groups (new Appx A.16.4, Fig. 15): the heterogeneous sequence, the original Lorenz family, and two new Lorenz-63 sequences with moderately similar (ρ=28,40,45,60)(\rho=28,40,45, 60) and similar (ρ=28,30,32,35)(\rho =28,30,32,35) parameters. The share of each task’s input that arrives through transfer connections increases gradually with similarity, from 4.2% for the heterogeneous sequence to 15.7%, 20.9% and 24.5% for the dissimilar, moderately similar, and similar Lorenz sequences (Fig. 15E). Lesioning these connections after training significantly degraded reconstruction of nearly all tasks that received transfer (Fig. 15 A-D). Later tasks thus draw on dynamics encoded by earlier units, and the more so the more dynamical structure the tasks share.

We thank the reviewer again for their time and look forward to hearing from them. Since our initial rebuttal, we have extended the scaling analysis as requested (W5, Q5) and added the gait baselines; see the revision (scaling: new Appx. A.15, Figs. 11, 16, Table 21; gait baselines: Appx. A.9, Tables 12, 14).

1. Scaling of unit usage with the number of tasks (W5, Q5)

I) Unit usage grows approximately linearly and predictably. On the gait dataset, we compared cumulative unit usage of CRUG, CLNP and HAT across all tested configurations (40 trials, 3 seeds). CRUG and CLNP grow approximately linearly (CLNP with higher seed variance), whereas HAT's usage approaches model capacity (Fig. 16A). This pattern holds across configurations; stronger regularization only lowers the slope or saturation rate (Fig. 16D).

We attribute this to the allocation penalties. CRUG’s (L0L_0) penalty counts committed units in absolute terms, so a task's cost does not depend on how many units remain. HAT's attention regularizer penalizes new units relative to those still free, so each task claims a roughly constant fraction of the remaining pool, and usage approaches M geometrically.

II) The linear growth rate reflects structure tasks share. With fixed hyperparameters, we ran three Lorenz streams of increasing heterogeneity:

  • Similar: ρ = 28→ 30→ 32→ 35→ 38→ 40→ 43→ 45→ 48.
  • Moderate: ρ = 28→ 40→ 45→ 60→ 70→ 80→ 90→ 100→ 110.
  • Dissimilar: ρ = 28 → 313 → 200 → 160 → 99.65 → 166.3 → 126.52 → 218 → 148.5 | Stream | Slope = units/task (per-seed R2) | Transfer input / total input | # Tasks learned (finalDstsp<4.0) | |---|---|---|---| | Similar | 21.2 (0.994) | 0.29 [0.27, 0.32] | 6.5 of 9 (range 6–8) | | Moderate | 27.7 (0.989) | 0.24 [0.21, 0.28] | 6 of 9 (range 5–7) | | Dissimilar | 30.3 (0.987) | 0.16 [0.14, 0.19] | 5 of 9 (range 4–5) |

The more structure tasks share, the more each new task reuses earlier units via forward transfer, so it needs fewer new units and more tasks fit into a fixed-size model.

III) Long sequences reveal how performance changes near exhaustion. We extended the gait sequence from 12 to 30 subjects (M = 1024, P = 512) and observed three phases:

PhaseSubjectsFree ReLU units when the subject is claimedLinear units / subjectReLU units / subjectTotal units / subjectMedian test Dstsp
Plentiful ReLU1–15512 → 11316.128.344.42.68
ReLU pool shrinking16–2289 → 223.712.436.17.01
No ReLU left23–300–113.9013.915.02

With plentiful ReLU units, performance is stable and ReLU usage is linear (R2=0.998R^2 = 0.998). As the ReLU pool shrinks, the allocator compensates with more linear units, total usage grows more slowly, and performance degrades; once no ReLU units remain, it degrades further.

IV) Summary. CRUG's unit usage grows linearly with the number of tasks, with a slope set by the hyperparameters (pruning and transfer strength) and the heterogeneity of the task stream. Because this growth has low variance across conditions, unlike CLNP, exhaustion can be predicted in advance. This suggests a practical strategy: estimate the commitment rate from the first few tasks, project when the free pool will run low, and add units as needed. At exhaustion, CRUG loses plasticity but does not forget learned tasks, and adding units restores learning without disturbing them (Fig. 11).

2. Gait baselines

CRUG versus other baselines. We tuned CRUG (with L2 transfer regularization) and all baselines with Optuna, using 40 trials and three seeds per trial each, selecting the best configuration on the validation set and reporting test-set results (Due to the character limit, only median values are shown here; full statistics with IQRs are reported in Table 14(a).):

MethodDstspDHCapacity (%)
Naive50.930.170–
Interleaved2.910.034–
EWC8.510.075–
SI5.940.064–
XdG37.920.187–
ER3.840.029–
GR4.440.031–
CLNP6.700.06759.2
HAT3.780.03399.9
CRUG2.660.04650.2

As in previous benchmarks, the three adaptive unit-isolation methods and the replay methods perform best, with CRUG offering the best trade-off between performance and committed units. CRUG learned all 12 subjects sequentially and reproduced their subject-specific gait dynamics (Fig. 8), showing that its advantage extends to real-world recordings: a single RNN retains multiple subject-specific dynamical models.

We believe these experiments directly address the reviewer's remaining concern by confirming that our main claim holds over long task sequences, and we would be grateful if the reviewer could take them into account in their final assessment.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Overall Response

We thank all reviewers for their careful and constructive reviews. Reviewers found CRUG simple and its mechanism clear (45TV, MsXi), the paper technically sound and well organized (6N89), and the evaluation broad (MsXi, 6N89). The main shared requests were

  • (i) ablations identifying which components of CRUG matter,
  • (ii) a comparison with independent per-task AL-RNNs,
  • (iii) the behavior of CRUG when its capacity is exhausted, and
  • (iv) evidence that CRUG applies beyond the AL-RNN.

Further questions concerned the novelty relative to CLNP and HAT, tuning budgets, computational cost, additional benchmarks, and shared dynamics across tasks. We addressed all of them with new analyses, summarized below and added to the revised manuscript (all changes marked in blue).

RequestRefs.New analysisMain result
Ablations: AllocationMsXi, 6N89L0L_0 gates replaced by an L1L_1 penalty on units with activity-based pruning, all else identical (A.16.1)No configuration learned the full sequence; the first task alone took 84-98% of total capacity (vs. CRUG: 12%)
Ablations: Commitment45TV, 6N89Performance before and after thresholding and absorption, gate distributions, thresholds 0.1–0.9, binary gates (A.16.2)No change through commitment; thresholds 0.1–0.7 equivalent; binary gates raise the overall DstspD_{stsp} from 1.43 to ≈\approx 14
Ablations: Gradient ProtectionMsXi, 6N89Gradients of committed units scaled by ϵ\epsilon instead of zero (A.16.3)Forgetting already at $\epsilon=$0.001; only hard masking preserves earlier systems
Ablations: Shared dynamics45TV, MsXiCRUG trained on four task groups of increasing similarity, from heterogeneous systems to Lorenz-63 systems with nearly identical ρ\rho, followed by transfer lesions (A.16.4)The share of input that tasks receive through transfer connections increases gradually with task similarity; removing these connections degrades nearly all tasks that received transfer
Independent per-task models45TV, MsXi1. Matched-performance: Size sweep to find minimum needed capacity; 2. Matched-capacity performance comparison (A.13)Independent models need 1.7 to more than 3 times as many units (206 vs. 63, 166 vs. 98, >336 vs. 108.5); at matched capacity, CRUG is better in almost all cases
Capacity exhaustion45TV, MsXiReduced networks that run out of units; network growth (A.15)No forgetting of earlier tasks; with few remaining units, the new task is still partially learned; with no free units, it cannot be learned by construction; adding units on the fly restores performance
Other architectures and tasksMsXi, 6N89CRUG on an LSTM (A.11); Split MNIST (A.12)LSTM: no forgetting, overall DstspD_{stsp}=2.14 vs. 3.07 for interleaved training; Split MNIST: 99.25% accuracy with 31.7% of units
Computational cost45TVTraining time, FLOPs and peak memory vs. naive training (A.14)<+5.2% training time; FLOPs and memory unchanged within 0.5%
HATMsXiAdding HAT as a baseline to both DSR benchmarks, cognitive tasks and Split MNIST (A.7; Tables: 1,3,11,15,16,18; Figures: 2, 6, 9)Performs on par with or close to CRUG on all benchmarks, but needs much more capacity (89/39%, 82/61%, 65/58%, 88/32% for DSR-1, DSR-2, cognitive, Split MNIST)

Together, these results show that compact L0L_0 allocation, exact commitment and hard gradient protection are each required to learn sequences of dynamical systems without forgetting, since small changes to recurrent parameters can alter the reconstructed attractors. A single CRUG network is more capacity-efficient than separate models, uses shared structure across tasks when it exists, and responds to capacity exhaustion by failing to learn new tasks while preserving all earlier ones. The framework also transfers to other recurrent architectures.

We further clarified the novelty relative to CLNP, which lies mainly in the problem of continually reconstructing dynamical systems with their long-term behavior and in the adaptations this problem requires (45TV W1, MsXi W1a); the rationale for tuning budgets and seeds (45TV W4, MsXi W2); the use of invariant statistics for evaluation (45TV W3); and the design of the gates and the fixed encoder and decoders (6N89 Q1–Q4). We also corrected the inconsistencies the reviewers pointed out. We hope to have clarified the reviewers’ concerns and remain open for further questions and criticism.

AuthorsOpenReview ↗

21 Additions at a Glance for the Final Assessment

Dear Reviewers and AC,

We thank the reviewers for their thorough examination and insightful questions, which helped strengthen the paper. To facilitate the final assessment, we list here each addition and location in the revised PDF. In total, the revision adds a new main-text section (4.3), 6 appendix sections (A.11-A.16), 9 figures (Figs. 6, 9-16) and 6 tables (Tabs. 11, 17-21), and extends Tabs. 1, 3, 12 and 14-16.

Scaling, capacity and cost

#AdditionAsked byWhereKey result
1Capacity exhaustion and network growth45TV (W5, Q5), MsXi (W4, Q2)Sec. 4.3; A.15; Fig. 11Exhaustion limits plasticity for new tasks, never causes forgetting; adding units restores learning
2Unit usage vs. number of tasks: (1) CRUG vs. CLNP, HAT on gait extended from 12 to 30 subjects; (2) three Lorenz streams of 9 tasks of varying similarity45TV (W5, Q5), MsXi (W4, Q2)A.15; Fig. 16; Tab. 21Usage grows linearly with a slope set by task similarity, so exhaustion can be predicted and units added in advance
3Computational cost on AL-RNNs and LSTMs45TV (Q1)Sec. 4.3; A.14; Tab. 20CRUG adds +5% time; <0.5% FLOPs and memory

Component-wise ablations

#AdditionAsked byWhereKey result
4L0 gates replaced by L1 activity penalty + pruningMsXi (W1b, Q1), 6N89 (W1b)Sec. 4.3; A.16.1; Fig. 12L1 cannot drive activity to exactly zero; pruning low-activity units degrades DSR. No configuration learns the full sequence (task 1 alone takes 84-98% of units)
5Gate thresholding g: (1) gate distributions, (2) g[0.1,0.9], (3) binary gates, (4) 3 evaluation stages,45TV (Q3), 6N89 (W1b, Q3)Sec. 4.3; A.16.2; Fig. 13Gates are exactly 0 or clearly open. g0.7: no change; g=0.9 and binary gates degrade Dstsp (14 vs. 1.43), confirming DSR's sensitivity to small parameter changes
6Soft gradient masking ([0.001,1])6N89 (W1b)Sec. 4.3; A.16.3; Fig. 14Forgetting starts at =0.001, so hard masking is essential
7Transfer lesions in 4 task streams of varying similarity45TV (Q6), MsXi (W1b, Q1)Sec. 4.2; A.16.4; Fig. 15Transfer grows with similarity; lesioning transfer weights degrades nearly all receiving tasks

New baselines, benchmarks, architectures

#AdditionAsked byWhereKey result
8HAT on all benchmarksMsXi (W2, Q3)Sec. 4.1; A.7; Tabs. 1, 3, 11, 12, 14-16, 18; Figs. 2, 6, 9On par with CRUG but uses more capacity everywhere (e.g., 142 vs. 63 units on DSR-1)
9All CL baselines on gait (previously CRUG only)MsXi (W2, Q3), (Note: added after Reviewer MsXi's final comment (16 Nov 2026), so not yet reflected in their assessment.)Sec. 4.1; A.9; Tabs. 12, 14aCRUG has the best performance-capacity trade-off
10Split MNISTMsXi (W2, Q3)Sec. 4.1; A.12; Fig. 9; Tab. 18On par with the best baselines (ER, HAT, CLNP) while using less capacity
11CRUG on LSTMsMsXi (W3), 6N89 (W1a)Sec. 4.1; A.11; Tab. 17Learns continually without forgetting
12Independent per-task AL-RNNs, matched (1) performance, (2) capacity45TV (W2, Q4), MsXi (W2, Q3)Sec. 4.3; A.13; Fig. 10; Tab. 19Separate models perform worse at equal size; need 1.7-3.3 more units to match CRUG
13Pareto fronts for HAT, CLNP, CRUGMsXi (W2, Q3)A.7; Fig. 6CRUG has the best front on all 4 benchmarks

Evaluation, clarifications, corrections

#AdditionAsked byWhereKey result
14Re-evaluation on separate validation and test sets45TV (W3, Q2)A.2, A.9, A.10Results changed by <1%
15Optuna trials and seeds stated and justified45TV (W4b, W4c), MsXi (W2, Q3)A.5, A.7Same seeds for all methods per benchmark; more trials for CRUG, CLNP and HAT, which tune 3 hyperparameters vs. 1 for the other baselines
16Interleaved baseline describedMsXi (W2)A.5Identical to the CL methods in every respect, except data from all tasks are interleaved; no separate hyperparameter search
17Fixed encoder/decoders6N89 (Q2)A.1Ensures that all learning happens in the latent dynamics, isolating CRUG's effect
18Symmetric input/output gating6N89 (Q1)RebuttalA unit contributes only if both input and output gates are open; separate gates add no expressivity
19Novelty vs. CLNP/HAT45TV (W1), MsXi (W1a)RebuttalTo our knowledge, the first systematic benchmark of CL methods on DSR. CRUG's novel integration of existing components outperforms all baselines with superior capacity efficiency across classification, input-output mapping and continual DSR
20Cognitive-task capacity corrected (58.4%)45TV (W4a)Sec. 4.1; Tab. 15-
21Notation of Eq. 36N89 (Q4)Sec. 2.2; A.4Correct; rewritten in clip notation

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.