- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes AMBER, an age-aware replay-memory framework for continual fine-tuning of large language models. Built on a fast/slow LoRA architecture, it coordinates three buffer operations through task-relative sample age: admitting reservoir, middle-loss-quantile and slow-adapter-generated samples; replacing unprotected historical entries with generated traces according to an age-dependent probability while preserving real-data anchors; and sampling replay entries with exponentially decaying age weights. Training additionally uses a one-sided hidden-state consistency loss between the adapters. On a fixed eight-task TRACE-8 sequence with Llama-2-7B, the paper reports improved overall performance (0.465 versus 0.384 for I-LoRA and 0.408 for FOREVER), with accompanying forgetting, transfer, ablation and buffer-composition analyses.
Critical strengths
-
An understandable and interesting memory-management mechanism with potential wider applicability. Rather than treating writing, buffer maintenance, and replay sampling as entirely isolated decisions, AMBER organizes them in a coherent lifecycle with a common task-age reference. Its protected-real-anchor mechanism also provides an intuitive safeguard against overreliance on synthetic rehearsal. I find the mechanisms relatively easy to understand, practically interesting, and potentially adaptable to other continual-learning settings, even though that broader applicability has not yet been experimentally established.
-
Relatively substantial empirical assessment and promising end-to-end results. Within the eight-task TRACE-8 setting, the paper provides comparisons against several replay and non-replay methods, transfer and forgetting metrics, component ablations, trajectory plots, and buffer-composition analysis. Its reported final overall performance of 0.465 is well above I-LoRA (0.384) and FOREVER (0.408). Despite the limitations of a single model, task order and run, I regard this as a meaningful empirical evaluation that supports a cautiously positive recommendation for the overall system, without proving the incremental effect of every proposed mechanism.
Critical weaknesses
- The specific effect of the core age-evolution mechanism has not been adequately isolated under matched training protocols. Although the complete AMBER system reports a substantial improvement over I-LoRA, Appendix B.2 (Table 8) lists gradient accumulation 4 for AMBER and 8 for I-LoRA. The approaches also differ in their consistency loss, slow-adapter handling and replay generation. These changes could affect the effective batch size, number of optimizer updates and computation budget. Meanwhile, Table 2 shows only 0.001–0.006 OP decreases when individual components are removed. Such small ablations do not establish that the components are useless, since their effects may interact, but neither do the end-to-end comparisons establish how much of the improvement is specifically due to age-aware buffer evolution. A matched training-protocol comparison with a simpler static/uniform replay policy would strengthen the paper's main innovation claim. I view this as a substantive but potentially addressable evaluation limitation, rather than evidence that the proposed method does not work.
Additional Methodological Consideration (AI-assisted; optional, not a second decision-driving weakness)
The link between task age, historical interference, and robust retention is not directly validated. AMBER defines age solely by distance in the task sequence and uses fixed exponential priors for replacement and sampling. In the writing stage, reservoir admission, loss-quantile selection, and generation are not explicitly weighted by age; new samples are assigned age zero for subsequent processing, so the claim that age directly governs all three stages should be qualified. The paper does not demonstrate that this proxy tracks actual forgetting or cross-task interference, or that its half-life schedule outperforms simple count-matched replacement and task-balanced sampling strategies. The final buffer contains only 11.6% generated traces, and single-component ablations produce small OP changes. Furthermore, the nonnegative BWT claim rests on a single point estimate of +0.002, with no run-to-run uncertainty, while AMBER's forgetting score (0.032) is slightly worse than FOREVER's (0.030). The empirical results support the full system more strongly than the proposed explanation of why age-aware evolution works.
Questions for authors
Controlled contribution: With the same fast/slow adapter settings, gradient accumulation, optimizer update budget, consistency loss and replay budget, could the authors compare the full AMBER design with static reservoir/uniform replay? How much of the improvement remains specifically attributable to the age-aware buffer mechanism? Alternative explanation: If available, could the authors compare age-based replacement and sampling with simple random or task-balanced alternatives using matched buffer size and replacement/replay counts? Such evidence would help establish whether the age variable contributes beyond a generally effective replay configuration. Robustness and reproducibility: Do the authors have additional seeds or task orders showing whether the small ablation differences and BWT=+0.002 are stable? Could they also clarify the relevant replay hyperparameters and optimizer-step counts? These details would increase confidence in the results without changing the fact that the existing evaluation covers a meaningful eight-task setting.
2 comments
Response to Reviewer 4o56
Q1 Controlled contribution This is the L1 experiment under aligned fast and slow adapter configuration, gradient accumulation, optimizer update budget, consistency loss and replay budget. It matches AMBER in GPU count, effective batch size, optimizer steps per task, precision, LoRA configuration, EMA, consistency term form, task-boundary reload and buffer capacity, and changes only the content and update rule of the buffer. OP is 0.371 against 0.465, a gap of 0.094, which can be regarded as the overall contribution of the buffer mechanism.
Q2 Alternative explanation The full age ablation fixes buffer capacity and replacement count, 86 per GPU and 172 across two GPUs, keeps the replay count identical to the complete method, and replaces age-based sampling and replacement with uniform, that is a random strategy, giving an OP of 0.455. Task-balanced sampling was not run owing to time constraints and is listed as future work.
Q3 Robustness and reproducibility OP for the repeated runs and the reversed order is 0.465, 0.462 and 0.437, all above FOREVER at 0.408 and I-LoRA at 0.384. The two original-order runs differ by 0.003 and BWT is positive in both.
Replay hyperparameters and optimizer steps are as follows.
| Item | Value |
|---|---|
| Buffer capacity | 500 per GPU, 1000 across two GPUs |
| Replay samples per micro batch | 1 |
| Replay samples per optimizer step | 8 |
| Optimizer steps per epoch per GPU | 625 |
| Optimizer steps per task | 1875 |
| Optimizer steps over the sequence | 15000 |
| Generation frequency | once every 10 optimizer steps |
| Generation prefix length | 20 tokens |
| Maximum new tokens | 50 |
| Half life | 50 tasks |
| Per-boundary replacement probability | |
| Replay sampling weight | exponential decay in age, set to 0.5 |
| Consistency coefficient | 1.0 |
| Boundary sample quantile band | 40th to 60th percentile |
| Protected anchors | 5 per task, 40 slots in total, about 8% of capacity |
This table will be placed in the appendix. Each task contains 5000 training samples and runs 3 epochs at effective batch size 8, giving 1875 optimizer steps per task and 15000 steps over eight tasks.
Response to Reviewer 4o56
Thank you for your thorough review and for recognizing this mechanism and its potential for wider applicability. Your comments have been a great encouragement to us, and we accept that the role of age should be stated more narrowly.
W1 Whether the temporal evolution mechanism is independently validated Our protocol runs 8 TRACE-5000 tasks with base model Llama-2-7B, LoRA rank 8, alpha 16, dropout 0.05, target modules q_proj and v_proj, max length 1024, learning rate 10−4, 3 epochs, linear warmup ratio 0.2, bf16, 2 GPUs, per-device batch size 1, gradient accumulation 4, effective batch size 8, replay buffer 500 per GPU and 1000 in total, one replay sample per micro batch and 8 per optimizer step. We added three experiments on this protocol.
Table 8 reports 8 for the gradient accumulation of I-LoRA while AMBER is 4. Several entries in that column are incorrect. In the final configuration AMBER, I-LoRA, SSR and FOREVER all train on 2 GPUs with accumulation 4, and all methods use an effective batch of 8. The correction changes neither the numbers in the main table nor the conclusions.
The first is the L1 control, which replaces only the buffer with static real samples and uniform replay, stores no generated traces, samples in place uniformly without age weights, performs no half-life replacement, no loss-quantile admission and sets no protected anchors, leaving all other parameters untouched. The second is the full age ablation, whose replacement count is matched to the replacement frequency of the complete method, with 86 replacements per GPU across boundaries and 172 across two GPUs, and whose sampling and replacement are changed to uniform, independent of age. A third run reverses the whole task order with seed 42.
The gain of the system as a whole is thus validated. Age is a component accounting for about one tenth of it and forms the overall gain of 0.094 together with buffer composition and replay allocation. Interaction effects exist between modules, so the difference of a single module cannot be read as zero contribution.
W2 Scope of age and interpretation of BWT We accept the reviewer's suggestion on presentation. The writing stage marks samples by task position without explicitly introducing age weights, while maintenance and sampling use age weights. The paper will state that age acts on maintenance and sampling and no longer present it as running through all three stages.
BWT is a relative quantity, defined as
BWT=T−11∑j=1T−1Δj,Δj=RT,j−Rj,j
where Rj,j is the score right after task j is learned and RT,j is the score at the end of the sequence. The reference point is Rj,j. The full method learns new tasks more thoroughly, so its reference point is higher and the same late-stage drift is recorded as a lower BWT. Removing a module lowers that reference point and makes BWT look better, but these ablations also lose OP and Fgt together, which shows that less knowledge can degrade rather than more being retained. The paper frames this as a stability and plasticity trade-off and prioritizes overall performance over minimizing forgetting. Judged by the OP of this method and the L1 control, we do not read small BWT changes as evidence that a module is ineffective.
Results of the repeated runs are as follows.
BWT is positive in both original-order repeated runs, consistent with the paper's non-negative backward transfer. After changing the task order we claim only the system-level gain in OP and do not claim the sign of BWT.
Generated traces account for 11.6% of the buffer and Fgt is 0.032, slightly above 0.030 for FOREVER. We report both as they are and interpret them within the same stability and plasticity trade-off.