acceptodds
Under review as a conference paper at ICLR 2027

Sleep-Inspired Two-Stage Continual Learning for Spiking Neural Networks

Abstract

Continual learning systems are often plagued by catastrophic forgetting across sequential tasks, whereas the human brain retains prior knowledge through neural plasticity and memory consolidation. Spiking neural networks (SNNs), which emulate the brain’s communication and neuronal dynamics, provide a promising substrate for brain-inspired continual learning. However, a pronounced stability and plasticity dilemma is still encountered in continual learning with SNNs. Neuronal dynamics are altered by incoming tasks, while current and historical samples are usually treated symmetrically by conventional replay. To overcome these limitations, a sleep inspired framework is proposed. Rapid daytime adaptation is separated from nighttime knowledge consolidation. Three complementary mechanisms are integrated, including sample-wise coefficients derived from membrane potential and spike statistics, source-dependent optimization for current and replay samples, and class-balanced consolidation via old class response distillation. Experiments on MNIST, DVS128 Gesture, CIFAR-10, and CIFAR-100 under class incremental and task incremental settings show consistent improvements in final accuracy over experience replay. These results demonstrate that temporally separating acquisition and consolidation can effectively coordinate stability and plasticity in SNN continual learning.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Spiking Neural Networks, Continual Learning, Sleep Inspired Consolidation, Activity Aware Gating, Asymmetric Replay

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_toAuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a sleep-inspired framework for continual learning in spiking neural networks. Daytime learning combines source-dependent replay, old-class distillation, and classifier-weight regularization controlled by detached neuronal statistics. Nighttime learning consolidates knowledge through approximately class-balanced memory replay and distillation. Experiments cover MNIST, DVS128 Gesture, CIFAR-10, and CIFAR-100 under class-incremental and task-incremental settings, using a fixed memory capacity of 500 samples.

Critical strengths

The paper addresses the important problem of catastrophic forgetting in spiking neural networks. Its sleep-inspired separation of adaptation and consolidation provides an intuitive framework for managing the stability–plasticity trade-off, making the research direction relevant to both continual learning and brain-inspired computing.

Critical weaknesses
  1. The principal comparisons do not isolate the effect of the proposed learning algorithm. Section 4.1 uses artificial neural networks for the MNIST and CIFAR baselines but different spiking architectures for the proposed method; the DVS128 Gesture architecture is also described as “improved.” Consequently, the sequential curves described as matched comparisons still confound learning strategy with architecture and potentially optimization. The heterogeneous protocols acknowledged for Tables 1–2 further limit comparative conclusions. Evaluation against ER, DER++, and relevant SNN baselines using the same architecture, memory policy, augmentation, and comparable training budgets is necessary to substantiate the main performance claims.

  2. The controller’s objective may encourage boundary-seeking behavior rather than meaningful sample-dependent adaptation. In Section 3.2, detached neuronal statistics produce coefficients whose minibatch mean enters only the regularization objective. For fixed classifier weights, this objective is linear in the mean coefficient, so its preferred value lies toward one of the permitted extremes unless the two penalties balance exactly. Furthermore, every sample receives the same sign of direct coefficient gradient, determined by global classifier-weight drift, while the controller receives no direct classification or distillation supervision. This does not establish that useful sample-specific plasticity is learned.

Questions for authors
  1. The method simultaneously introduces a controller, selective weight anchoring, source-dependent Mixup, label smoothing, distillation, balanced replay, and an additional consolidation stage. Without controlled ablations, the observed improvements could arise primarily from familiar replay or distillation components. Particularly informative comparisons would remove each major component and compare nighttime consolidation with the same number of balanced replay updates interleaved throughout daytime learning.

  2. The state vector combines membrane-potential statistics and spike rate, which have different scales, yet their relative scaling before cosine-distance computation is not explained. Min–max normalization also makes surprise depend on minibatch composition and can amplify very small differences. Moreover, classifier expansion changes the population over which these statistics are aggregated, potentially introducing changes unrelated to forgetting. Sensitivity analyses and measurements relating surprise to subsequent forgetting would help establish whether the signal captures meaningful novelty.

  3. The experimental description is insufficient for reproduction. The manuscript should specify the exact “improved” architectures, neuronal leak and reset settings, event preprocessing, exemplar selection and removal rules, replay batch sizes, controller optimization, and learning-rate schedules. Nighttime epoch counts are stated for CIFAR but remain unclear for MNIST and DVS128 Gesture.

  4. The scope of the SNN-specific contribution should be clarified. Section 3.1 states that the feature extractor is evaluated once and only the spiking classification head is unfolded over time. The manuscript should therefore describe the architecture precisely and distinguish benefits attributable to spiking dynamics from those attributable to the training schedule. A corresponding nonspiking-head control would be especially informative.

  5. Section 3.3 divides the batch-averaged soft cross-entropy by the number of old classes, but this loss does not generally grow linearly with class count. The normalization may therefore progressively weaken preservation as the task sequence grows. Additionally, matching distributions normalized only over old classes preserves their relative responses without directly controlling competition between old and new classes. Sensitivity experiments and old-versus-new class accuracy breakdowns would clarify how these choices affect retention and calibration.

2 comments
AuthorsOpenReview ↗

Authors' Response

Thank you very much for taking the time to review and for your support We try our best to address your questions as follows.

A1: The components are organized around acquisition and consolidation goals, not as isolated innovations. Source-related processing distinguishes current samples from past memories, while the controller modulates daytime retention intensity; nighttime switches to balanced memory consolidation. This functional division is directly verifiable via Eqs. (5)–(10). Controls address distinct issues: source-symmetric processing tests discrimination; fixed/global coefficients test input conditioning; ER with identical distillation tests routine retention; equal-budget interleaving tests temporal organization. State modulation and weight retention are separable (fixed coefficients still retain anchors). Interleaved updates act on acquisition-stage models and require redefining memory entry timing, thus not equivalent to post-acquisition consolidation. We will clarify these conditions and separate existing-system results from individual component effects.

A2: "Surprise" denotes a neuron's deviation from the consolidated reference state (Eqs. 2–4, Sec. 3.2); it does not represent semantic novelty or forgetting probability. Membrane potential mean/dispersion and firing rate provide complementary activity information; cosine distance removes uniform scaling, and min-max normalizes range. These operations serve distinct purposes—cosine does not eliminate component-wise scale differences, and normalization differs from pre-scaling. Batch composition, narrow distance ranges, and head expansion may influence this metric. We treat surprise as an auxiliary input in a bounded controller, combined with raw state, rather than sole determinant of predictions or forgetting. We will clarify statistical units, stability terms, and reference update rules; Sec. 3.3 explains reference updates via equilibrium memory statistics (momentum 0.9).

A3: Sections 3.1–3.3 and 4.1 describe the memory/head configuration, controller architecture, stage objectives, and CIFAR training settings. The revision and supplementary materials will further detail layer architecture, LIF dynamics, event preprocessing, memory management, replay, controller optimization, learning-rate scheduling, and night rounds. We will also release the full implementation and reproduction instructions (data splits, seeds, scripts, result aggregation), while including all necessary settings in the paper/supplement so that reproduction does not depend on future code releases.

A4: Section 3.1 describes a feature extractor that performs a single computation and a classification head that executes 12 steps. Phase scheduling, replay, and distillation are not unique to SNNs; what is directly related to SNNs is the dynamics of the classification head and the interface between membrane potential and firing rate states. In the spiking architecture studied in this paper, observable membrane potentials and firing rates are used to link neural dynamics and training regulation, which constitutes its SNN-specific implementation. The training scheduling is portable and does not require that each optimization tool be applicable only to SNNs. Non-spiking head controls allow for further analysis of the additional effects of spiking dynamics; we will clearly define the architectural scope of our findings, avoiding confusion between mechanism portability and SNN-specific effects, and refraining from drawing conclusions about hardware energy efficiency based on these results.

A5: Dividing by the number of old classes may indeed weaken long-sequence retention; we will correct the original paper's overly strong motivation. Contextually, Eq. (8) preserves intra-old-class relative responses, while daytime full-class supervision and nighttime class-balanced hard-label training provide direct old-new competition. Thus, softmax translation invariance within old classes does not imply a lack of cross-class competition in the full objective. Our method outperforms ER on Class-IL/Task-IL benchmarks, showing current weights function effectively under existing conditions. However, overall accuracy/forgetting does not equal a dedicated calibration test; we limit our claims accordingly and will investigate weight sensitivity and old-new performance on longer sequences in future work.

I appreciate the authors' responses and their clarifications. However, my main concerns remain insufficiently addressed. In particular, the lack of architecture-matched comparisons still makes it difficult to isolate the contribution of the proposed learning strategy. I therefore maintain my original rating.

Reviewer_gQ8LOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a sleep-inspired two-stage continual learning framework for spiking neural networks. The daytime stage combines source-aware asymmetric replay, old-class response distillation, and a controller-conditioned regularizer; the nighttime stage performs hard-label classification and distillation on a class-balanced memory of 500 samples, with ten extra epochs per task on CIFAR-10/100. The authors evaluate four benchmarks under class-IL and task-IL, report higher final accuracy than the implemented experience-replay (ER) baseline, and analyse the forgetting profiles of the proposed method. Validation rests mainly on the sequential comparison against ER, together with the cross-study Tables 1 and 2.

Critical strengths
  1. A concrete integration of acquisition and consolidation for continual learning in SNNs. The two stages use distinct objectives and coordinate replay, distillation and neuronal-state regularization. This provides an implementable design whose incremental value is worth evaluating.
  2. Promising results for the complete recipe. The method has the highest listed Task-IL accuracy in three of four columns, including 85.32% on CIFAR-100. These results support the potential practical value of the combined method, while the cross-study protocol differences limit controlled ranking claims.
Critical weaknesses
  1. The primary CIFAR comparison does not isolate the proposed training mechanisms. On CIFAR-10/100 the baselines use a ResNet-18 ANN, whereas the proposed method uses a ResNet-18-based spiking model together with distillation, asymmetric replay, controller regularization and ten nighttime epochs per task. A comparison on the same predictive network with a comparable update budget is needed to determine the contribution of the new training mechanisms. This is what determines my assessment — the issue is not a lack of leaderboard performance, but the lack of attribution.
  2. Neither core mechanism is ablated. There is no comparison, at a fixed total update budget, between a separate nighttime stage and distributing the same consolidation updates through daytime training; and the controller is not compared against a fixed coefficient or a single learned global coefficient. Without these, whether the two-stage schedule and the state-dependent controller actually contribute cannot be determined.
Questions for authors
  1. p.8 reports a CIFAR-10 final accuracy of 71.88 with ER at 43.04, whereas Table 1 reports 71.28 and 44.79. Which pair corresponds to the comparison reported in the text? The two give margins of 28.84 and 26.49 points.
  2. Is the feature extractor identical to the baseline's? How does the classifier differ? Which layer-level changes, if any, does "improved" in Section 4.1 refer to?
  3. On the identical predictive network and a comparable update budget, what margin remains for ER and for ER plus distillation? This determines whether the main conclusion holds.
  4. With the total update budget fixed, does a separate nighttime stage outperform interleaving the same consolidation updates into daytime training?
  5. Eq. (5) enters the loss only through the batch mean of the coefficients. What does a per-sample coefficient add over a single learned global coefficient, and what is the within-batch variance of the coefficients during training?
2 comments
AuthorsOpenReview ↗

Authors' Response

Thank you very much for taking the time to review and for your support We try our best to address your questions as follows.

Q1: p.8 reports a CIFAR-10 final accuracy of 71.88 with ER at 43.04, whereas Table 1 reports 71.28 and 44.79. Which pair corresponds to the comparison reported in the text? The two give margins of 28.84 and 26.49 points.

A1: The main text values (71.88/43.04; 28.84 pp) and Table 1 values (71.28/44.79; 26.49 pp) reflect different statistical methods—single run versus multi-run average—and cannot be cited interchangeably. To ensure consistency, we will standardize the text and figures to the single-run raw data with explicit labeling. Multi-run averages and dispersions will be reported separately, avoiding appending "±" directly to single-run values. All differences are expressed in percentage points.

Q2: Is the feature extractor identical to the baseline's? How does the classifier differ? Which layer-level changes, if any, does "improved" in Section 4.1 refer to?

A2: Both methods share an identical ResNet-18 feature extractor; the difference lies in the classification head (non-spiking baseline vs. 12-step LIF spiking in ours), yielding distinct prediction networks.

"Improved" refers to adding the state condition controller (Sec. 3.2), not modifying the feature extractor. This MLP generates bounded coefficients from membrane potential statistics, firing rates, and a surprise signal to adjust old-class weight retention during daytime training, without scaling features/outputs or participating in inference.

Q3: On the identical predictive network and a comparable update budget, what margin remains for ER and for ER plus distillation? This determines whether the main conclusion holds.

A3:The distinction between ER, ER with old-class distillation, and our method is not merely whether teacher responses are retained. ER replays historical samples; distillation further constrains old-class responses; our method also alters current and replay data processing and switches to class-balanced memory consolidation after task acquisition. Conventional distillation thus does not automatically yield source-relevant optimization or phase switching—the two key mechanistic differences.

For the requested values, comparisons must align the full prediction network, update count, and sample volume; differences from existing systems should not be labeled as remaining gain under these conditions. Our accuracy/forgetting results illustrate framework performance in the current setting. We will clarify applicable conditions and focus on the two-stage mechanism and asymmetric data processing, rather than treating general distillation or optimizers as sources of innovation.

Q4: With the total update budget fixed, does a separate nighttime stage outperform interleaving the same consolidation updates into daytime training?

A4: Nighttime consolidation has clear triggers (daytime adaptation done, current class learned, memory updated). Daytime insertion acts on an intermediate model, so equal steps do not guarantee equivalence. Sequential and interleaved updates differ in parameters and memory protocol, and thus are not interchangeable without aligning memory updates, sampling, teachers, objectives, and budgets. This justifies our “acquire-then-consolidate” design, though it is not a universal superiority theorem.

Q5: Eq. (5) enters the loss only through the batch mean of the coefficients. What does a per-sample coefficient add over a single learned global coefficient, and what is the within-batch variance of the coefficients during training?

A5: The controller performs input-conditional, batch-level adjustment: Eq. (4) computes sample-wise coefficients from neuronal states, and Eq. (5) uses their batch mean to regularize shared old-class weights. “Sample-wise” thus denotes coefficient generation, not per-sample loss weighting. Unlike a learnable global scalar that is input-independent under fixed parameters, the batch mean varies with neuronal state and batch composition; mean aggregation also prevents a single sample from dominating shared-weight maintenance. These are structural/functional distinctions whose benefits require control comparisons. Although no significant within-batch variance anomalies were observed, this does not replace quantitative analysis; we will report variance, batch-mean trajectories, and boundary-approaching proportions, clarifying that variance describes coefficient distribution whereas the batch mean is what enters Eq. (5).

Thank you for the clarifications. However, the response still doesn't include a comparison on the same predictive network with a comparable update budget, or ablations isolating the two-stage schedule and the controller. Without those results, the main claims remain unsupported, so I'm keeping my score.

Reviewer_2AfGOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper explores the idea of sleep-based learning for deep neural networks to aid class-incremental learning, which is a good connection to neuroscience, through an architecture that decouples learning into a rapid daytime adaptation phase and a nighttime consolidation phase. The authors show empirical results on CIFAR10 and CIFAR-100.

Critical strengths
  1. Biologically Inspired Temporal Separation of rapid acquisition (daytime adaptation) from memory stabilisation (night consolidation).

  2. This approach demonstrates significant accuracy gains over standard experience replay across MNIST, CIFAR-10, CIFAR-100 and neuromorphic (DVS128 Gesture) datasets.

Critical weaknesses
  1. The entire methodology seems weak since the authors mention a lot about sleep and daytime adaptations over and over but never really justify it or bridge the connection between actual neuroscience and the deep learning method written.

  2. The main paper, except the appendix, doesn't contain any tables for comparison. Table 1 and Table 2 don't show or compare with the SOTA SNN methods such as [1], [2], [3]. Also, we note that some of the values in Tables 1 and 2 are filled with "-", which leads to an unfair comparison.

  3. We would like to understand why exactly ResNet18 is used as the backbone, although there are an enormous number of spiking transformers being introduced. We believe that the vision transformers may be better at capturing features for continual learning. We would like to know whether the method is invariant to the backbone; i.e., extend the experiments to different backbones to analyse its sensitivity to the backbone.

  4. The authors claim to be an SNN-based method that utilises spike states for continual learning; we'd like to understand whether this method can be scaled for large-scale event-based datasets [4].

  5. The authors don't provide any description of the controller mentioned in the paper, which helps in the adaptation.

  6. The motivation of the paper remains unclear, as they don't describe or show any analysis (ablations) on the importance of the different adaptation phases

  7. There is no forgetting, memory, latency or even energy analysis given in the paper.


References

[1] Ni, Wenyao, et al. "Alade-snn: Adaptive logit alignment in dynamically expandable spiking neural networks for class incremental learning." Proceedings of the AAAI conference on artificial intelligence. Vol. 39. No. 18. 2025

[2] Han, Bing, et al. "Enhancing efficient continual learning with dynamic structure development of spiking neural networks." In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence (IJCAI '23). Article 334, 2993–3001. https://doi.org/10.24963/ijcai.2023/334

[3] Shen, Jiangrong, et al. "Robust selective activation with randomized temporal k-winner-take-all in spiking neural networks for continual learning." International Conference on Learning Representations. Vol. 2026. 2026.

[4] Qian, Yuanbin, et al. "Ucf-crime-dvs: A novel event-based dataset for video anomaly detection with spiking neural networks." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 39. No. 6. 2025.

Questions for authors
  1. Clarity on the motivation for why sleep is required in continual learning, since most recent papers utilise a sleep mechanism in OCL rather than CIL.

  2. Aren't mixup and label smoothing common feature engineering tricks, but it is unclear whether they play a role as a specific mechanism.

  3. What are the Forgetting, memory, energy, latency or wall-clock analyses?

  4. Is this method scalable with respect to different backbones?

  5. Also, can you compare your method with recent continual learning methods (both spiking and non-spiking) on higher-resolution image datasets such as ImageNet / or its comparable variants?

  6. Does it extend to more complex vision tasks such as object detection or segmentation, or, say, large foundation models?

These are the major questions. For the ones not mentioned here, please refer to the weakness section.

2 comments
AuthorsOpenReview ↗

Authors' Response

Thank you very much for taking the time to review and for your support We try our best to address your questions as follows.

Q1: Clarity on the motivation for why sleep is required in continual learning, since most recent papers utilise a sleep mechanism in OCL rather than CIL.

A1: This paper introduces a separate consolidation phase to resolve weight discrepancies and preserve historical knowledge after acquiring new classes. Since CIL (incremental categories) and OCL (online constraints) can overlap, sleep-inspired consolidation is not limited to OCL; in our CIL setting, it balances current task learning with full-category consolidation. As detailed in Sections 3.1 and 3.3, once daytime adaptation concludes at a known task boundary, the class-index memory is updated. Only this class-balanced memory is then used for hard-label classification and old-class distillation, while acquisition-phase enhancements and controller regularization are removed. The task boundary provides clear timing for consolidation. We investigate this specific phasing arrangement without claiming that all CIL requires sleep, nor do we use the presence of task boundaries as a defining distinction between OCL and CIL.

Q2: Aren't mixup and label smoothing common feature engineering tricks, but it is unclear whether they play a role as a specific mechanism.

A2: The novelty lies not in Mixup or label smoothing per se, but in their source- and phase-dependent application. Specifically, early stages enhance only current samples (playback retains hard labels), later stages share refinement, and these operations are removed during consolidation. This grants new classes fitting flexibility while preserving the fixed supervisory targets of scarce old samples. Thus, our contribution is the orchestration of these operations rather than the operations themselves. We will explicitly clarify this design in Section 3.3, avoiding claims of these general operations as independent innovations.

Q3: What are the Forgetting, memory, energy, latency or wall-clock analyses? A3: Forgetting (Sec. 4.2, Fig. 3), memory configuration (500 samples; Secs. 3.1, 4.1), and CIFAR training schedules (Sec. 4.1) address performance, constraints, and scheduling, respectively. For runtime, latency, and power, we distinguish training from deployment under standardized hardware and conditions. We do not equate sample capacity with total storage, nor do we claim hardware efficiency advantages for the SNN architecture.

Q4: Is this method scalable with respect to different backbones?

A4: Day/night scheduling and source-related replay are structurally decoupled from ResNet-18, enabling integration with compatible backbones. However, the state controller's reliance on pulse statistics requires dimension matching and re-optimization for new backbones. We thus emphasize structural portability with bounded claims: our results strictly apply to the validated architecture, and portability does not equate to cross-backbone performance invariance.

Q5: Also, can you compare your method with recent continual learning methods (both spiking and non-spiking) on higher-resolution image datasets such as ImageNet / or its comparable variants?

A5: We select MNIST, DVS128 Gesture, CIFAR-10, and CIFAR-100 to evaluate phase scheduling and source-dependent learning across static/event inputs and varying category scales. ImageNet evaluations involve confounding variables (resolution, features, pretraining, and memory budgets); thus, concatenating heterogeneous protocols cannot isolate our mechanisms. We therefore focus on controlled comparisons and clarify our experimental scope rather than substituting heterogeneous literature results.

Algorithmically, our source-dependent objectives and scheduling are resolution-agnostic, extensible to high-resolution inputs via front-end adjustments. However, we do not infer ImageNet performance from this. We discuss this extension while grounding our conclusions in the four benchmarks.

Q6: Does it extend to more complex vision tasks such as object detection or segmentation, or, say, large foundation models? A6: We thank the reviewer and plan to extend phased acquisition and consolidation to detection, segmentation, and foundation model adaptation. Future work will tailor memory mechanisms for region/pixel-level supervision and design efficient phased scheduling for foundation models. This maintains the core principle of separating new knowledge acquisition from historical integration, while adapting implementations to varying output structures and training costs.

Reviewer_2AfGOpenReview ↗

Thanks but my questions not addressed yet

I would like to retain my review comments and rating, as the authors have not fully addressed the weakness as of now.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.