- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes a wake–sleep approach to continual learning. The model periodically adds parameters and distills knowledge from fast-updating modules into slower ones. It then generates synthetic data and learns which data helps improve itself. Experiments cover continual learning, long-context understanding, reasoning, and knowledge incorporation.
Critical strengths
Transferring temporary knowledge into slower modules is an interesting direction. The paper gives a concrete way to combine expansion, distillation, and synthetic replay.
The evaluation covers several tasks, and the ablations help show where the gains come from. Consolidation and dreaming seem to provide complementary benefits.
Critical weaknesses
It is hard to tell how much improvement comes from the proposed mechanism versus more parameters and more training. OPSD plus expansion is a useful baseline, but the remaining gains are often small, with no uncertainty reported. The continual-learning setup also needs more detail.
The theoretical claims seem stronger than what is shown. The SVD result applies to a simplified surrogate, and the conditioning results require assumptions about the covariance and task-gradient direction that are not checked empirically.
Questions for authors
How does Sleep compare with baselines given the same parameter capacity and total compute, including generation and inner-loop training?
What are the task order, consolidation schedules, and parameter counts in the continual-learning experiments? Can you show retention after each task?
Is the on-policy distribution held fixed for the SVD result? Do the assumptions behind the conditioning results hold in practice?
Do the gains hold across seeds? Several improvements over OPSD plus expansion are small, and ARC uses only eight evaluation tasks.
3 comments
Total compute, including generation and the inner loop. We now report GPU-hours end-to-end and give the strongest baseline Sleep's full budget. The budget counts teacher/student generation and semantic-judge calls; the baseline may spend the extra budget on additional OPSD training and generation.
| Qwen3-8B (3 seeds) | GPU-h | AIME-24 | AIME-25 | HMMT-25 |
|---|---|---|---|---|
| OPSD + Expansion (as in paper) | 25 | 77.7 ± 1.0 | 68.0 ± 1.1 | 45.5 ± 1.3 |
| OPSD + Expansion (compute-matched to Sleep) | 40 | 77.9 ± 1.1 | 68.2 ± 1.2 | 45.6 ± 1.4 |
| Sleep | 40 | 78.8 ± 1.0 | 68.7 ± 1.1 | 45.7 ± 1.3 |
Dreaming's cost includes generation and inner-loop training. In the sequential knowledge-incorporation study (response to Reviewer 6ZHJ, W2), Dreaming only (SEAL-style) gets the same total budget as full Sleep and reaches 29.3 vs. 40.0 final average accuracy. For the remaining baselines, GRPO already takes 5× more steps than Sleep (Table 2), and App. D.5 shows that SFT needs 3.6–4.8× Sleep's wall-clock to reach the same accuracy.
Q2. What are the task order, consolidation schedules, and parameter counts in the continual-learning experiments? Can you show retention after each task?
Response: Thank you for the suggestion. To make the setup fully explicit, the revision adds:
| CLINC | Banking | DBpedia | |
|---|---|---|---|
| # tasks × classes per task | 10 × 15 | 7 × 11 | 7 × 10 |
| Task order | fixed random permutation, identical for all methods | ||
| Protocol | Momeni et al. (2025): one pass per task, no replay of earlier data | ||
| # CMS levels | 3 levels | ||
| Total / active / added parameters | 3.21B / unchanged / 55.1M | ||
| Metric | accuracy over all classes seen so far |
Retention after each task. The final metric covers all classes seen so far, so Fig. 3 already penalizes forgetting. HOPE + Sleep improves over the identical HOPE backbone without the sleep phase in all 6 dataset × backbone settings. (We will make the naming in Sec. 5.1 consistent so that "HOPE" always refers to the no-sleep baseline.)
The revision adds the full accuracy matrix (accuracy on task after training on task ). From it we report final average accuracy and forgetting, :
| Llama-3B | CLINC / | Banking / | DBpedia / |
|---|---|---|---|
| HOPE (no sleep) | 69.0 / 25 | 63.5 / 28 | 42.2 / 50 |
| HOPE + Sleep | 76.9 / 18 | 69.0 / 23 | 47.4 / 46 |
We also add a multi-round sequential knowledge-incorporation study for full Sleep, consolidation only and Dreaming only. It tracks new learning and retention after every sleep cycle, including Dreaming (response to Reviewer 6ZHJ, W2). Full Sleep has forgetting 8.0, vs. 18.0 for Dreaming only.
Q3. Is the on-policy distribution held fixed for the SVD result? Do the assumptions behind the conditioning results hold in practice?
Response:
On-policy distribution. Yes. Theorem 5 treats as fixed. In the GKD distillation term of Algorithm 3, student rollouts are drawn under a stop-gradient (line 6), executing a semi-gradient step with held fixed (while line 15 uses policy gradient on the discrete LTI reward, which Theorem 5 models via the fixed- local surrogate). It does not hold over the whole trajectory: as change, the student's samples, and hence , change too. The consistent statement for the trained adapter is therefore self-referential. At any fixed point that is second-order stationary for the frozen-distribution surrogate at its own equilibrium covariance , after the change of variables in App. A.4.2 this problem is a standard low-rank approximation whose local minima are global (Baldi & Hornik, 1989), so the closed form applies self-consistently at . The rank-allocation interpretation then applies with taken at convergence. We will state this explicitly.
Do the conditioning assumptions hold? We measured them in the SQuAD dreaming phase, following the protocol in our response to Reviewer nwwj (Q3). Conditioning is reported on the top- eigenspace, with bootstrap error bars:
| Assumption / prediction | Measured |
|---|---|
| PSD and better conditioned than (top- subspace) | = 0.00; 15 vs. 100 |
| ratio 0.90 | |
| Task gradient aligned with the top eigen-direction | = 0.11; = 0.40 |
| Spectral gap | 0.50 |
| 0.88 vs. 0.50 |
Response:
Seeds. Thank you for the suggestion. Note that OPSD + Expansion is our own strengthened baseline, which already includes Sleep's expansion; its margins (+1.3 / +0.8 / +0.2) therefore isolate the KS objective alone, and we now report them with uncertainty. The 3-seed results are in the Q1 table: Sleep's mean exceeds OPSD + Expansion on all three benchmarks (by 1.1 / 0.7 / 0.2), and the paired per-seed difference is positive in 6 of 9 seed × benchmark pairs. We also report bootstrap CIs (over problems and samples) for every row of Tables 1, 3 and 4.
Against the external baseline OPSD, the margin is +2.6 / +1.6 / +1.0, and Sleep beats OPSD in all 6 model × benchmark cells of Table 3.
ARC. Thank you for raising this. We follow the SEAL protocol: 11 training and 8 held-out tasks, with 5 dreams applied per test task, so each success rate rests on only 40 adaptations. Treating these as independent, the 95% Wilson intervals are [65.2, 89.5] for Sleep (32/40) and [57.2, 83.9] for SEAL (29/40). Because the adaptations are nested within 8 tasks, the true intervals are wider still. Table 5 therefore does not establish a significant difference from SEAL; the only clear separation is from TTT (4/40, [4.0, 23.1]). We will state this in the paper, report per-task outcomes, and present ARC as protocol-matched supporting evidence that complements the main results on math, SQuAD and continual learning.
We hope these answers and results address the reviewer's concerns. If they do, we would be grateful if the reviewer would consider raising their score.
We thank the reviewer for the concise and pointed review, and for noting that consolidation and Dreaming provide complementary benefits. The questions are the right ones to ask. Below we answer each directly, distinguishing what the current paper already shows from what we add. Where a point overlaps with another review, we give a self-contained summary here.
Response: We address this in three points; details are in Q1, Q2 and Q4.
Response: Thank you for this comment. We will make the scope of each result explicit in the abstract, Sec. 4 and the conclusion. The underlying results are also more general than a first reading may suggest:
The SVD result. rsem and rabs are binary or thresholded per sample, but at each prefix position (on 1⊥) Eq. (4) optimizes their expectation under the student, which is a smooth function of the logits. At Z⋆, the combined KL-regularized objective therefore has a PSD quadratic local model by Taylor's theorem. Theorem 5 holds for any context-independent PSD metric with Q~≻0 (i.e., col(Wu)∩null(Q)={0} and any target. With context-dependent curvature, it holds for the averaged metric in Qˉ~ (retaining exact Q(h) in C) under the standard K-FAC approximation. For the KL term, Lemma 9 shows that KL is locally equal to a quadratic in the Fisher metric (requiring 1∈/col(Wu)), so the upper bound is not needed. The closed form explains the update but does not compute it: A,B are trained on the exact Eq. (4). We will present Thm 1 as "a second-order model of the KS objective admits a closed-form truncated-SVD solution".
The conditioning results. We will move every condition (including the inherited Thm 6(iii) conditions and O(ε3) remainder in Thm 8) into the main-text Theorems 2–3, together with two minor technical refinements (an explicit κ≥π/(π−2) condition in Eq. 54, and labeling the post-filtering constant as an approximation). Two conditions are milder than they may look:
Whether the assumptions hold in practice is measured in Q3.
Response:
Parameters. The expansion adds the 5 low-rank (dimension-64) expert blocks of App. D: 94.4 M parameters, 1.15% of the backbone. The active parameter count is unchanged. All Table 3 baselines train rank-64 LoRA adapters (174.6 M trainable parameters) with the same learning rate and batch size (Table 2). OPSD + Expansion has exactly Sleep's parameters.