acceptodds
Under review as a conference paper at ICLR 2027

AMBER: Age-Aware Memory Evolution for Continual Adaptation of Large Language Models

Abstract

Continual fine-tuning adapts a large language model to a sequence of tasks. However, fitting a new task tends to overwrite knowledge from earlier tasks whose data is no longer available. Experience replay addresses this difficulty by retaining a small buffer of historical samples and training them along with the current task, but existing methods decide separately which samples enter the buffer and how often each is replayed. Because the buffer has a fixed capacity and is largely frozen once written, adaptation reduces to adjusting the replay frequency, and the stored content cannot keep pace as the gradient relationships between earlier and current tasks drift during training. To tackle this, we propose AMBER, a buffer management framework that manages the Age-aware Memory Buffer with Evolving Replay. The age is defined as the temporal distance between a sample and the current task. Using task temporal distance to encode each sample's historical position and its conflict with the current task, AMBER jointly regulates buffer writing, maintenance, and sampling for dynamic memory evolution. Writing draws candidates from a reservoir stream, a loss-quantile band, and slow-adapter generation. Maintenance replaces unprotected entries with generated traces at an age-dependent rate while protected anchors retain a real-data backbone. Sampling weights replay by age. On an eight-task sequence with Llama-2-7B, AMBER improves overall performance by 21.1% over the dual-adapter baseline and is the only compared method that does not degrade backward transfer. These results indicate that one interpretable age signal can govern buffer composition and replay scheduling together within a fixed storage budget, improving the stability and plasticity trade-off that continual learning must balance.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Large Language Models, continual learning, parameter-efficient fine-tuning, experience replay, Catastrophic Forgetting

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_4o56OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes AMBER, an age-aware replay-memory framework for continual fine-tuning of large language models. Built on a fast/slow LoRA architecture, it coordinates three buffer operations through task-relative sample age: admitting reservoir, middle-loss-quantile and slow-adapter-generated samples; replacing unprotected historical entries with generated traces according to an age-dependent probability while preserving real-data anchors; and sampling replay entries with exponentially decaying age weights. Training additionally uses a one-sided hidden-state consistency loss between the adapters. On a fixed eight-task TRACE-8 sequence with Llama-2-7B, the paper reports improved overall performance (0.465 versus 0.384 for I-LoRA and 0.408 for FOREVER), with accompanying forgetting, transfer, ablation and buffer-composition analyses.

Critical strengths
  1. An understandable and interesting memory-management mechanism with potential wider applicability. Rather than treating writing, buffer maintenance, and replay sampling as entirely isolated decisions, AMBER organizes them in a coherent lifecycle with a common task-age reference. Its protected-real-anchor mechanism also provides an intuitive safeguard against overreliance on synthetic rehearsal. I find the mechanisms relatively easy to understand, practically interesting, and potentially adaptable to other continual-learning settings, even though that broader applicability has not yet been experimentally established.

  2. Relatively substantial empirical assessment and promising end-to-end results. Within the eight-task TRACE-8 setting, the paper provides comparisons against several replay and non-replay methods, transfer and forgetting metrics, component ablations, trajectory plots, and buffer-composition analysis. Its reported final overall performance of 0.465 is well above I-LoRA (0.384) and FOREVER (0.408). Despite the limitations of a single model, task order and run, I regard this as a meaningful empirical evaluation that supports a cautiously positive recommendation for the overall system, without proving the incremental effect of every proposed mechanism.

Critical weaknesses
  1. The specific effect of the core age-evolution mechanism has not been adequately isolated under matched training protocols. Although the complete AMBER system reports a substantial improvement over I-LoRA, Appendix B.2 (Table 8) lists gradient accumulation 4 for AMBER and 8 for I-LoRA. The approaches also differ in their consistency loss, slow-adapter handling and replay generation. These changes could affect the effective batch size, number of optimizer updates and computation budget. Meanwhile, Table 2 shows only 0.001–0.006 OP decreases when individual components are removed. Such small ablations do not establish that the components are useless, since their effects may interact, but neither do the end-to-end comparisons establish how much of the improvement is specifically due to age-aware buffer evolution. A matched training-protocol comparison with a simpler static/uniform replay policy would strengthen the paper's main innovation claim. I view this as a substantive but potentially addressable evaluation limitation, rather than evidence that the proposed method does not work.

Additional Methodological Consideration (AI-assisted; optional, not a second decision-driving weakness)

The link between task age, historical interference, and robust retention is not directly validated. AMBER defines age solely by distance in the task sequence and uses fixed exponential priors for replacement and sampling. In the writing stage, reservoir admission, loss-quantile selection, and generation are not explicitly weighted by age; new samples are assigned age zero for subsequent processing, so the claim that age directly governs all three stages should be qualified. The paper does not demonstrate that this proxy tracks actual forgetting or cross-task interference, or that its half-life schedule outperforms simple count-matched replacement and task-balanced sampling strategies. The final buffer contains only 11.6% generated traces, and single-component ablations produce small OP changes. Furthermore, the nonnegative BWT claim rests on a single point estimate of +0.002, with no run-to-run uncertainty, while AMBER's forgetting score (0.032) is slightly worse than FOREVER's (0.030). The empirical results support the full system more strongly than the proposed explanation of why age-aware evolution works.

Questions for authors

Controlled contribution: With the same fast/slow adapter settings, gradient accumulation, optimizer update budget, consistency loss and replay budget, could the authors compare the full AMBER design with static reservoir/uniform replay? How much of the improvement remains specifically attributable to the age-aware buffer mechanism? Alternative explanation: If available, could the authors compare age-based replacement and sampling with simple random or task-balanced alternatives using matched buffer size and replacement/replay counts? Such evidence would help establish whether the age variable contributes beyond a generally effective replay configuration. Robustness and reproducibility: Do the authors have additional seeds or task orders showing whether the small ablation differences and BWT=+0.002 are stable? Could they also clarify the relevant replay hyperparameters and optimizer-step counts? These details would increase confidence in the results without changing the fact that the existing evaluation covers a meaningful eight-task setting.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 4o56

Thank you for your thorough review and for recognizing this mechanism and its potential for wider applicability. Your comments have been a great encouragement to us, and we accept that the role of age should be stated more narrowly.

W1 Whether the temporal evolution mechanism is independently validated Our protocol runs 8 TRACE-5000 tasks with base model Llama-2-7B, LoRA rank 8, alpha 16, dropout 0.05, target modules q_proj and v_proj, max length 1024, learning rate 10−410^{-4}, 3 epochs, linear warmup ratio 0.2, bf16, 2 GPUs, per-device batch size 1, gradient accumulation 4, effective batch size 8, replay buffer 500 per GPU and 1000 in total, one replay sample per micro batch and 8 per optimizer step. We added three experiments on this protocol.

Table 8 reports 8 for the gradient accumulation of I-LoRA while AMBER is 4. Several entries in that column are incorrect. In the final configuration AMBER, I-LoRA, SSR and FOREVER all train on 2 GPUs with accumulation 4, and all methods use an effective batch of 8. The correction changes neither the numbers in the main table nor the conclusions.

The first is the L1 control, which replaces only the buffer with static real samples and uniform replay, stores no generated traces, samples in place uniformly without age weights, performs no half-life replacement, no loss-quantile admission and sets no protected anchors, leaving all other parameters untouched. The second is the full age ablation, whose replacement count is matched to the replacement frequency of the complete method, with 86 replacements per GPU across boundaries and 172 across two GPUs, and whose sampling and replacement are changed to uniform, independent of age. A third run reverses the whole task order with seed 42.

MethodOPFgtFWT
AMBER0.4650.0320.130
L1 static reservoir0.3710.1140.089
Full age ablation0.4550.0290.117
Reversed AMBER0.4370.0490.127

The gain of the system as a whole is thus validated. Age is a component accounting for about one tenth of it and forms the overall gain of 0.094 together with buffer composition and replay allocation. Interaction effects exist between modules, so the difference of a single module cannot be read as zero contribution.

W2 Scope of age and interpretation of BWT We accept the reviewer's suggestion on presentation. The writing stage marks samples by task position without explicitly introducing age weights, while maintenance and sampling use age weights. The paper will state that age acts on maintenance and sampling and no longer present it as running through all three stages.

BWT is a relative quantity, defined as

BWT=1T−1∑j=1T−1Δj,Δj=RT,j−Rj,j\mathrm{BWT} = \frac{1}{T-1}\sum_{j=1}^{T-1}\Delta_j, \quad \Delta_j = R_{T,j}-R_{j,j}

where Rj,jR_{j,j} is the score right after task jj is learned and RT,jR_{T,j} is the score at the end of the sequence. The reference point is Rj,jR_{j,j}. The full method learns new tasks more thoroughly, so its reference point is higher and the same late-stage drift is recorded as a lower BWT. Removing a module lowers that reference point and makes BWT look better, but these ablations also lose OP and Fgt together, which shows that less knowledge can degrade rather than more being retained. The paper frames this as a stability and plasticity trade-off and prioritizes overall performance over minimizing forgetting. Judged by the OP of this method and the L1 control, we do not read small BWT changes as evidence that a module is ineffective.

Results of the repeated runs are as follows.

SettingOPFgtFWT
Seed 42, original order0.4650.0320.130
Seed 3675, original order0.4620.0180.120
Seed 42, reversed order0.4370.0490.127

BWT is positive in both original-order repeated runs, consistent with the paper's non-negative backward transfer. After changing the task order we claim only the system-level gain in OP and do not claim the sign of BWT.

Generated traces account for 11.6% of the buffer and Fgt is 0.032, slightly above 0.030 for FOREVER. We report both as they are and interpret them within the same stability and plasticity trade-off.

AuthorsOpenReview ↗

Response to Reviewer 4o56

Q1 Controlled contribution This is the L1 experiment under aligned fast and slow adapter configuration, gradient accumulation, optimizer update budget, consistency loss and replay budget. It matches AMBER in GPU count, effective batch size, optimizer steps per task, precision, LoRA configuration, EMA, consistency term form, task-boundary reload and buffer capacity, and changes only the content and update rule of the buffer. OP is 0.371 against 0.465, a gap of 0.094, which can be regarded as the overall contribution of the buffer mechanism.

Q2 Alternative explanation The full age ablation fixes buffer capacity and replacement count, 86 per GPU and 172 across two GPUs, keeps the replay count identical to the complete method, and replaces age-based sampling and replacement with uniform, that is a random strategy, giving an OP of 0.455. Task-balanced sampling was not run owing to time constraints and is listed as future work.

Q3 Robustness and reproducibility OP for the repeated runs and the reversed order is 0.465, 0.462 and 0.437, all above FOREVER at 0.408 and I-LoRA at 0.384. The two original-order runs differ by 0.003 and BWT is positive in both.

Replay hyperparameters and optimizer steps are as follows.

ItemValue
Buffer capacity500 per GPU, 1000 across two GPUs
Replay samples per micro batch1
Replay samples per optimizer step8
Optimizer steps per epoch per GPU625
Optimizer steps per task1875
Optimizer steps over the sequence15000
Generation frequencyonce every 10 optimizer steps
Generation prefix length20 tokens
Maximum new tokens50
Half life50 tasks
Per-boundary replacement probability1−2−a/501-2^{-a/50}
Replay sampling weightexponential decay in age, λ\lambda set to 0.5
Consistency coefficient λcons\lambda_{\mathrm{cons}}1.0
Boundary sample quantile band40th to 60th percentile
Protected anchors5 per task, 40 slots in total, about 8% of capacity

This table will be placed in the appendix. Each task contains 5000 training samples and runs 3 epochs at effective batch size 8, giving 1875 optimizer steps per task and 15000 steps over eight tasks.

Reviewer_RwWJOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

AMBER manages a fixed-capacity replay buffer for continual LLM fine-tuning using reservoir and loss-based admission, generated replay, age-dependent replacement, protected real examples, and age-weighted sampling. It builds on a fast/slow LoRA architecture and adds asymmetric consistency regularization. On an eight-task sequence with Llama-2-7B, it reports final average performance of 0.465, compared with 0.384 for I-LoRA and 0.408 for FOREVER.

Critical strengths
  • The problem is practically relevant, and the combination of content renewal with replay scheduling is a reasonable design for bounded-memory adaptation.
  • The reported improvement of the complete system is substantial. The evaluation spans heterogeneous tasks, and the paper provides component ablations, task-performance matrices, buffer statistics, and pseudocode.
  • The authors explicitly acknowledge single-run evaluation and the untested sensitivity to memory capacity and age schedules. They also correctly describe the method as a retention-adaptation trade-off rather than eliminating forgetting.
Critical weaknesses
  1. The comparison set omits closely related recent methods from experiments. FOREVER is a relevant 2026 baseline, so the issue is not a complete absence of recent work. However, OPR (Chen et al., 2026) and MSSR (Lu et al., 2026) are discussed but not evaluated. OPR is particularly important because it regenerates responses to historical prompts, directly challenging the broad distinction between existing static supervision and AMBER's evolving content. MSSR tests adaptive memory-based sampling and scheduling. SuRe (Hazard et al., 2025), also cited, combines selective replay with fast/slow LoRA consolidation and is another relevant comparator. Prioritized comparisons under a common protocol, or specific explanations of incompatibility, are needed; citing these methods alone does not establish the claimed advance.

  2. The headline comparison does not isolate buffer evolution. Table 8 uses gradient accumulation of four for AMBER versus eight for I-LoRA, SSR, and FOREVER. AMBER also changes the consistency objective and reloads the slow adapter at task boundaries. Thus, attributing its 21.1% relative improvement over I-LoRA specifically to buffer management is not justified by this comparison. A matched dual-adapter baseline should hold the objective, reset policy, effective batch size, and replay exposure constant while changing buffer operations. The 500-entry storage budget also does not establish equal training compute: generation and teacher passes require measured overhead, and the numerical replay batch size is not specified in the reported configuration.

  3. The central novelty and coupling claim need stronger isolation. The individual ingredients are established strategies, making the contribution primarily their age-based coordination. Yet writing records source-task metadata without an explicit age-dependent admission rule; age directly governs maintenance and sampling. Moreover, Table 2 provides leave-one-component-out results rather than testing whether shared-age coordination outperforms independently controlled operations. Age-independent replacement matched for replacement counts, together with a maintenance-by-sampling factorial control, would distinguish the proposed coupling from a collection of useful heuristics. Task distance is also assumed to represent conflict without directly testing that relationship.

  4. Experimental coverage and statistical support are insufficient for the breadth of the claims. All results use one backbone, one task order, one AMBER buffer capacity, and one run per configuration. This is particularly limiting for an age-based method, whose behavior may depend on task order. Component-removal differences are only 0.001-0.006 in overall performance, and the headline non-negative BWT is +0.002. Multiple seeds, reordered sequences, another modern backbone, and sensitivity to memory size and the two age parameters would substantially strengthen the evidence. Removing age weighting or generative replacement improves forgetting, so the ablations support a trade-off rather than establishing every component's necessity for retention.

Questions for authors
  1. How does AMBER compare with OPR and MSSR under matched storage and replay-compute budgets?
  2. How much improvement remains when optimization settings, consistency regularization, and teacher resets are matched to a buffer-only control?
  3. Are the small component gains and near-zero BWT reproducible across seeds and task orders?
2 comments
AuthorsOpenReview ↗

Response to Reviewer RwWJ

Thank you for your constructive review and for recognizing the practical value of the problem and the design that couples content update with replay scheduling under a limited memory budget. Your questions on attribution, on missing comparisons and on reproducibility guided the experiments below.

W1 Missing experimental comparison with OPR, MSSR and SuRe Our protocol runs 8 TRACE-5000 tasks with base model Llama-2-7B, LoRA rank 8, alpha 16, dropout 0.05, target modules q_proj and v_proj, max length 1024, learning rate 10−410^{-4}, 3 epochs, linear warmup ratio 0.2, bf16, 2 GPUs, per-device batch size 1, gradient accumulation 4, effective batch size 8, replay buffer 500 per GPU and 1000 in total, one replay sample per micro batch and 8 per optimizer step. OPR was ported on this protocol following the authors' public implementation, without extra tuning, and rebuilds the whole replay buffer after each task. The buffer for the final task is generated after the seventh task and holds 71 samples per historical task over seven tasks, 994 samples across two GPUs.

MethodOPFgtFWT
I-LoRA0.3840.1040.062
FOREVER0.4080.0300.069
AMBER0.4650.0320.130
OPR port0.3710.1170.096
L1 static reservoir0.3710.1140.089

OPR reaches 0.371, below AMBER at 0.465 and FOREVER at 0.408. AMBER holds 500 per GPU and OPR 994 in total, so the two buffers are of the same order, and the difference lies in how content is updated, with AMBER refreshing gradually by age while OPR rebuilds fully.

The code for MSSR and SuRe is not open source, so neither can be run under the same protocol. Owing to time constraints both reproductions are listed as future work, and the paper will state this rather than mention them only through citations.

W2 Core comparison and storage budget The training budget and replay capacity of each method are as follows.

MethodGPUsAccum.Effective batchBuffer per GPUTotal capacity
AMBER2485001000
I-LoRA2485001000
OPR248497994
SSR248grows with tasks3154
FOREVER2482% per task700

AMBER and I-LoRA both hold 1000 and OPR 994, all of the same order, while SSR at 3154 is larger than ours and FOREVER at 700 is smaller. No method is systematically favored or penalized along the capacity dimension.

Table 8 reports 8 for the gradient accumulation of I-LoRA while AMBER is 4. Several entries in that column are incorrect. In the final configuration AMBER, I-LoRA, SSR and FOREVER all train on 2 GPUs with accumulation 4, and all methods use an effective batch of 8. The correction changes neither the numbers in the main table nor the conclusions.

To isolate the buffer variable we ran the L1 control on the same protocol, replacing only the buffer with static real samples and uniform replay. The buffer holds real samples only, contains no generated traces, samples in place uniformly without age weights, performs no half-life replacement, no loss-quantile admission and sets no protected anchors. GPU count, effective batch size, optimizer steps per task, precision, LoRA configuration, EMA, task-boundary reload and buffer capacity are identical.

MethodOPFgtFWT
AMBER0.4650.0320.130
L1 static reservoir0.3710.1140.089

OP is 0.371 against 0.465 for AMBER, a gap of 0.094.

Compute is set by the number of replay samples. Both methods replay the same number, one sample per micro batch, and with 2 GPUs at gradient accumulation 4 this gives 8 replay samples per optimizer step.

AMBER has an additional generation overhead, run once every 10 optimizer steps with one sequence per GPU, a 20-token prefix and at most 50 new tokens, executed in a no-grad forward pass of the slow adapter. Within 10 optimizer steps the training side processes 160 sequences through forward and backward, that is 10 steps times 16 sequences per step, while the generation side processes 2 sequences of at most 70 tokens through forward only. Generation therefore accounts for about one percent of total compute.

W3 A replacement-count matched control independent of age This control is the full age ablation. Its replacement count is matched to the replacement frequency of the complete method, with 86 replacements per GPU across boundaries and 172 across two GPUs, and sampling and replacement are changed to uniform, independent of age.

MethodOPFgtFWT
AMBER0.4650.0320.130
Full age ablation0.4550.0290.117

The ablation reaches 0.455, a gap of 0.010 from the main result of 0.465, and this 0.010 equals 10.6% of the 0.094 gain over L1. Age therefore participates in the overall gain as an important component.

AuthorsOpenReview ↗

Response to Reviewer RwWJ

W4 Coverage We added a repeated run and a reversed-order run. The first changes only the seed and the second reverses the whole task order while keeping seed 42, with all other settings as in the main result.

SettingOPvs FOREVER 0.408vs I-LoRA 0.384
Seed 42, original order0.465+0.057+0.081
Seed 3675, original order0.462+0.054+0.078
Seed 42, reversed order0.437+0.029+0.053

All three settings exceed FOREVER at 0.408 and I-LoRA at 0.384. A second base model and sensitivity analyses over capacity and the two age hyperparameters were not run owing to time constraints and are listed as future work.

Q Three follow-up questions

Q1 Comparison with OPR and MSSR under matched storage budget and replay compute. OPR was ported on the same protocol, with a buffer of 994 samples at the final task against 500 per GPU and 1000 in total for AMBER, so the two are of the same order. Replay compute is also the same, one sample per micro batch and 8 per optimizer step. The result is 0.371 for OPR against 0.465 for AMBER, a gap of 0.094, as shown in the table in W2.1. The code for MSSR is not open source, so its reproduction is listed as future work.

Q2 How much gain remains when optimization parameters, consistency regularization and teacher adapter handling are aligned and only the buffer varies. We ran the L1 control for this comparison. It matches AMBER in GPU count, effective batch size, optimizer steps per task, precision, LoRA configuration, EMA coefficient and update rule, consistency term form, task-boundary reload and buffer capacity, and changes only the buffer to static real samples plus uniform replay. OP is 0.371 against 0.465, a gap of 0.094.

Q3 Whether small gains and a near-zero BWT reproduce across seeds and task orders. A rerun with another randomly drawn seed gives 0.462, a difference of 0.003 from the main result of 0.465. Fully reversing the task order gives 0.437, still above FOREVER at 0.408 and I-LoRA at 0.384. BWT is positive in both repeated runs, consistent with the non-negative backward transfer reported in the paper, so the paper's conclusion on BWT does not change.

Reviewer_xbsvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

AMBER tackles buffer management for continual instruction tuning and reports substantial aggregate gains over baselines like I-LoRA and FOREVER. However, the experimental design fails to properly isolate the proposed contributions. Additionally, the ablation evidence is somewhat weak to support the paper's core claims.

Critical strengths
  1. The age-based replacement and sampling rules are straightforward, and the protected-anchor mechanism explicitly addresses the risk of replacing all real supervision with synthetic data. The paper also distinguishes per-scan replacement probability from cumulative survival probability, avoiding an overly literal interpretation of “half-life.”
Critical weaknesses
  1. The comparison against I-LoRA does not actually isolate buffer management. AMBER changes several variables: effective batch size (gradient accumulation of 4 instead of 8), consistency objective (one-sided penalty vs. symmetric MSE), adapter reset policy, and replay allocations. Without a strictly controlled baseline that alters only the buffer strategy, the performance improvements cannot be reliably attributed to the proposed evolutionary design.

  2. The ablation study fails to prove that the proposed components are necessary or effective. Removing individual features (like generative replay or half-life replacement) drops overall performance by just 0.001 to 0.006. Because the experiments rely on a single run, these tiny margins are indistinguishable from random noise. Paradoxically, the data show that removing these components actually improves backward transfer.

  3. Using task age as a proxy for interference isn't well-supported. Because the method is evaluated on a single, fixed sequence, "age" is completely entangled with task identity and order. Furthermore, with the chosen half-life (Thalf=50T_{half}=50), the oldest entries face only a ~9% replacement probability per scan, making it unclear if this specific scheduling mechanism actually drives the results.

Questions for authors
  1. What numerical values are used for λ\lambda, λcons\lambda_{\mathrm{cons}}, brepb_{\mathrm{rep}}, and the consistency-layer set? How are competing buffer candidates admitted?
3 comments
AuthorsOpenReview ↗

Response to Reviewer xbsv

Thank you for your careful reading and for highlighting the distinction between per-scan and cumulative replacement probability. Your comments on isolating the buffer variable and on the statistical resolution of the ablations led us to add a matched same-backbone control and a repeated run.

W1 Whether the buffer variable is isolated We checked all training scripts. The protocol runs 8 TRACE-5000 tasks with base model Llama-2-7B, LoRA rank 8, alpha 16, dropout 0.05, target modules q_proj and v_proj, max length 1024, learning rate 10−410^{-4}, 3 epochs, linear warmup ratio 0.2, bf16, 2 GPUs, per-device batch size 1, gradient accumulation 4, effective batch size 8, replay buffer 500 per GPU and 1000 in total, one replay sample per micro batch and 8 per optimizer step. All methods use an effective batch of 8 and the same GPU count, so the differences lie in the buffer and the memory policy.

MethodGPUsAccum.Effective batchBuffer per GPUTotal capacity
AMBER2485001000
I-LoRA2485001000
OPR248497994
SSR248grows with tasks3154
FOREVER2482% per task700

The capacities of SSR and FOREVER are totals at the final task.

Table 8 reports 8 for the gradient accumulation of I-LoRA while AMBER is 4. Several entries in that column are incorrect. In the final configuration AMBER, I-LoRA, SSR and FOREVER all train on 2 GPUs with accumulation 4, and all methods use an effective batch of 8. The correction changes neither the numbers in the main table nor the conclusions.

We added experiment L1, a control on the same protocol that replaces only the buffer with static real samples and uniform replay. The buffer holds real samples only, contains no generated traces, samples in place uniformly without age weights, performs no half-life replacement, no loss-quantile admission and sets no protected anchors. GPU count, effective batch size, optimizer steps per task, precision, LoRA configuration, EMA coefficient and update rule, task-boundary reload and buffer capacity all match AMBER.

MethodOPFgtFWT
AMBER0.4650.0320.130
L1 static reservoir0.3710.1140.089

The gap is 0.094, while the two repeated runs differ by only 0.003, and the gain over L1 is 25.3%. This magnitude attributes the gain to the buffer mechanism itself.

W2 Whether the ablation can resolve module contributions The reviewer notes that single-module gaps in Table 2 are only 0.001 to 0.006, which a single run cannot separate from noise. We reran the same protocol with random seed 3675 and changed nothing else.

SeedOPvs FOREVER 0.408vs I-LoRA 0.384
420.465+0.057+0.081
36750.462+0.054+0.078

Both runs exceed the two baselines, the lower one by 0.054, and they differ by only 0.003. The direction is consistent and neither run falls below the baselines.

The new results do not change the paper's conclusions. The claim that the method works as a system rests on the L1 control and the repeated run. On the observation that some ablations show a higher BWT, this follows directly from how BWT is computed. BWT is the final score minus the score reached right after each task was learned, averaged over tasks. The full method learns new tasks more thoroughly, so its reference point is higher and the same late-stage drift is recorded as a lower BWT. Removing a module lowers that reference point and makes BWT look better, but these ablations also lose OP and Fgt together, which shows that less knowledge can degrade rather than more being retained. The paper frames this as a stability and plasticity trade-off and prioritizes overall performance over minimizing forgetting, so we do not read small BWT changes as evidence that a module is ineffective.

AuthorsOpenReview ↗

Response to Reviewer xbsv

W3 Coupling between age and task order Age serves in the method as a single temporal reference for a sample's historical position. The write-in stage records the source task index for each new sample, the maintenance stage sets the half-life replacement probability from age, and the sampling stage allocates replay frequency through exponential age weights. The abstract further describes this distance as encoding a sample's historical position and its potential conflict with the current task, and the title of 3.2.2 uses the phrase global conflicts, but these are motivation-level generalizations. What the paper defines and tests is the effect of a task-level age signal on buffer scheduling, not a validated per-sample measure of interference. The following examines this schedule through replacement probability, task order and the age ablation.

Each unprotected sample is drawn once at each task boundary. A sample of age aa is replaced with probability

pa=1−2−a/T1/2,T1/2=50p_a = 1 - 2^{-a/T_{1/2}}, \quad T_{1/2}=50

The probabilities for ages 1 to 7 are as follows.

Age1234567
Per-boundary probability1.38%2.73%4.07%5.39%6.70%7.98%9.25%

The 9% cited by the reviewer is the per-boundary probability of the oldest stratum. For a sample born in the first task, the probability of at least one replacement across seven boundaries is

Pcum=1−∏a=172−a/T1/2=1−2−28/50=0.322P_{\mathrm{cum}} = 1 - \prod_{a=1}^{7} 2^{-a/T_{1/2}} = 1 - 2^{-28/50} = 0.322

This is a per-sample and per-step quantity, not a one-shot replacement ratio over the buffer. Replacement does not delete samples either. A slot keeps its prefix and task label while the slow adapter regenerates its content, so 32.2% is a content refresh rate.

We reran the sequence with the task order fully reversed, keeping seed 42 and all other settings.

SettingOPFgtFWT
Seed 42, original order0.4650.0320.130
Seed 42, reversed order0.4370.0490.127
FOREVER0.4080.0300.069
I-LoRA0.3840.1040.062

Reversing the order lowers OP by only 0.028 and remains above FOREVER at 0.408 and I-LoRA at 0.384. The system-level gain does not depend on a particular order.

Finally the place of age in the total gain. The full age ablation matches the replacement frequency of the complete method, with 86 replacements per GPU across boundaries and 172 across two GPUs, and changes sampling and replacement to uniform, fully independent of age. OP is 0.455, a gap of 0.010 from the main result, and this 0.010 equals 10.6% of the 0.094 gain over L1. Together with buffer composition and replay allocation it forms the overall gain of 0.094.

Q Hyperparameters and admission λ\lambda is 0.5 and weights replay sampling exponentially, and λcons\lambda_{\mathrm{cons}} is 1.0 and scales the consistency term in the total loss.

The consistency term acts on the embedding output and the outputs of 32 decoder layers, 33 tensors in total, with a one-sided penalty that fires only when the fast adapter falls below the slow adapter, using the slow adapter as reference and equivalently taking the larger of the two.

Admission has four channels. Every training sample enters the buffer through reservoir sampling. At the end of each training epoch, samples in the 40 to 60 loss quantile band become boundary samples. Every 10 optimizer steps the slow adapter generates one trace from a 20-token prefix with at most 50 new tokens. Five real non-generated samples per task become protected anchors, 40 slots in total and about 8% of buffer capacity, redrawn at every refresh, kept from replacement within the round and blocking reservoir writes to their slots.

Thank you for your rebuttal. I have carefully read your response, but I have decided to maintain my original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.