- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
The paper studies which historical examples to replay when sequentially training the Weaver in a MemGen-style latent-memory model with a frozen reasoner. It first shows that static latent rankings identify influential event-slots without reliably distinguishing helpful from harmful effects, and that these effects depend on the injection event. The proposed selector nominates four event-slots by norm, independently zeroes each selected contextualized hidden state before projection, and admits an example if teacher-forced loss changes satisfy helpfulness and harm-avoidance thresholds. It then samples original, unmasked examples under source, pattern, and budget constraints. On a four-task sequence, the method improves final macro accuracy by 2.19 percentage points over per-source quantity-matched random replay across three seeds. Component ablations and single-run checks with another task order and Qwen3-4B provide additional support.
Critical strengths
- The paper makes a useful connection between interventions on latent memory and replay selection. I find the distinction between identifying an influential vector and determining the direction of its effect helpful. The event-conditioned, bilateral admission rule turns this observation into a concrete training procedure.
- The main comparison separates the benefit of replay from the benefit of selecting particular examples. Matching actual per-source replay counts, training-example counts, and optimization settings makes Table 2 informative about selection quality. The gains over random replay are positive for all three seeds (, , and points). Table 3 further strengthens this result: bilateral admission without Pattern Cap still improves over random replay by 1.41 points, while removing harm avoidance reduces the full method's mean score by points.
Critical weaknesses
-
The current evidence primarily establishes an advantage over quantity-matched random replay and internal variants. Although Table 7 includes alternative selection strategies, it uses a separate experimental protocol and does not report Causal replay under that protocol, leaving a direct comparison unresolved. Moreover, EWC and LwF provide useful classical continual-learning references, but neither is a replay selection method. They therefore do not substitute for comparisons with competitive, directly relevant replay or data selection approaches under a shared protocol. These gaps make it difficult to assess the advance over existing work and limit my assessment of the contribution’s significance.
-
Section 4.5 states that each replay stage requires five gradient-free forward passes per historical example. The method retains complete historical candidate pools and re-scores them at subsequent checkpoints. For equally sized tasks and fixed candidate-testing budget k, this yields quadratic cumulative scoring growth with task count, even when the replay training budget is fixed. The provided PDF does not report measured scoring time, end-to-end training time, or performance under a matched total compute budget. Furthermore, the main experiments match training-example counts but not token counts. The results therefore support effectiveness under a matched example budget, but do not establish whether the average 2.19-percentage-point gain over random replay justifies the scoring and storage costs, particularly for longer task sequences.
Questions for authors
-
Comparison request: Can the authors compare Similarity with Causal replay under the same per-source replay counts and training protocol, or report Causal replay under the full-pool protocol of Table 7? Comparisons with relevant loss-, interference-, or diversity-based selectors under the same protocol would also help establish the contribution beyond random selection. Please account for selection costs when assessing these alternatives.
-
Robustness request: The main multi-seed evidence comes from a single four-task sequence within the MemGen architecture. The alternative task order and Qwen3-4B results in Table 4 each use a single run. Could the authors provide multi-seed results for these two settings, including per-task performance and paired gains over quantity-matched random replay with uncertainty estimates? This would help determine whether the improvements are stable across task orders and backbones.
-
Could the authors report the computational budget and measured runtime at each stage, separating historical-example scoring and selection from subsequent training, together with scoring-token counts and memory/storage costs? Please clarify whether the quantity-matched Random branch also performs checkpoint rescoring, as suggested by line 364, and distinguish costs required by the selector from costs introduced by the experimental matching protocol. A comparison under a matched total compute budget would clarify the cost–benefit tradeoff. A longer task stream or a bounded/subsampled historical-pool evaluation would also materially inform the scalability assessment.
-
Section 4.3 explains why independent measurements implement the chosen per-candidate harm constraint, but does not establish the downstream value of this granularity. Can the authors compare the individual-slot rule with a block-level rule that jointly intervenes on all eight slots at a single injection event, at comparable scoring cost, and examine whether slot interactions change admission decisions and final replay performance?
-
Figure 1 examines different candidate-testing budgets using diagnostic discovery rates, but does not report the sensitivity of final replay performance to . Could the authors clarify the validation evidence behind choosing , including the search range and selection criterion, and report admission rates, final performance, and scoring overhead for alternative values of ?
-
Lines 279–281 zero the contextualized state before projection, whereas lines 287–289 describe the intervention as candidate removal. Please specify projection bias, attention-mask and position handling, cache handling, and prediction-target alignment: depending on the implementation, zero replacement can retain a projected bias or attention/position effects and need not completely remove the slot's influence. Can the authors assess effect-sign agreement, admission overlap, and a representative downstream replay outcome under another reasonable intervention baseline, such as mean replacement or graded attenuation? The helpful/harmful interpretation should remain explicitly relative to the tested baseline.
-
The source constraints impose upper caps rather than minimum replay allocations. What happens when no example from a historical task passes admission, or when the remaining budget is exhausted before a source is processed? Please report source-level zero-replay frequencies and subsequent task retention, and assess whether a minimum allocation or fallback policy is needed. This concerns a possibility allowed by the algorithm, rather than an assumption that it occurred in the reported runs.
-
How is the stage training size chosen, and how are the old/new and historical-source mixture proportions controlled under unequal task sizes and checkpoint-dependent admission rates? Please report source-level exposure and retention, particularly for GPQA. A feasible fixed/normalized mixture or reweighting control would clarify whether the allocation preserves small tasks; please specify whether sampling is with replacement. Varying proportions alone do not establish that the allocation is harmful, but their effects are not isolated by the current comparisons.
-
Section 4.1 mentions deduplication and availability filtering without specifying their executable rules in the provided PDF. Please provide dataset versions and splits, deduplication fields and matching criteria, availability conditions, counts or item identifiers before and after each step, the AQuA8000 sampling procedure, and the held-in validation split. If these details are supplied elsewhere, please identify the exact files.
10 comments
Response to Reviewer WepN (2/5) — Q2–Q3: Robustness and Compute
Q2. Multi-seed order and backbone checks
Both settings now use seeds 42–44. In the tables below, values are mean ± sample SD; differences are given with two-sided 95% paired-t intervals, n=3, df=2.
Permuted order: GPQA → KodCode → GSM8K → AQuA.
| Task | No replay | Random | Causal | Paired gain over Random |
|---|---|---|---|---|
| GSM8K | 10.66 ± 0.04 | 50.67 ± 0.09 | 51.58 ± 0.12 | +0.91 [0.41, 1.41] |
| KodCode | 10.15 ± 0.15 | 49.43 ± 0.13 | 51.00 ± 0.18 | +1.57 [1.38, 1.76] |
| GPQA | 29.29 ± 0.51 | 27.27 ± 0.00 | 26.94 ± 0.77 | −0.34 [−2.25, 1.58] |
| AQuA | 33.33 ± 0.99 | 37.93 ± 1.49 | 43.57 ± 0.45 | +5.64 [2.21, 9.08] |
| Macro | 20.86 ± 0.37 | 41.33 ± 0.39 | 43.27 ± 0.09 | +1.95 [0.80, 3.10] |
Qwen3-4B, main order.
| Task | No replay | Random | Causal | Paired gain over Random |
|---|---|---|---|---|
| GSM8K | 63.41 ± 0.34 | 63.74 ± 0.12 | 65.73 ± 0.08 | +2.00 [1.52, 2.47] |
| KodCode | 17.18 ± 0.25 | 53.82 ± 0.18 | 63.30 ± 0.20 | +9.48 [8.55, 10.42] |
| GPQA | 26.60 ± 1.05 | 27.44 ± 1.91 | 28.62 ± 0.77 | +1.18 [−5.01, 7.37] |
| AQuA | 63.78 ± 0.79 | 58.53 ± 0.99 | 64.70 ± 1.14 | +6.17 [1.15, 11.19] |
| Macro | 42.74 ± 0.09 | 50.88 ± 0.67 | 55.59 ± 0.26 | +4.71 [2.42, 6.99] |
The seed-wise Macro gains in the two settings are 1.62/2.47/1.75 and 4.38/4.00/5.75, respectively, positive on every seed. In both settings, the paired intervals for GSM8K, KodCode, and AQuA exclude zero; GPQA is the only task whose interval includes zero. The single-seed result on the larger Qwen3.5-9B backbone (Macro +2.45) is reported in our response to sULK Q1.
Q3. Cost accounting, Random rescoring, and restricted pools
On the 2×H800 seed-43 chain, S2/S3/S4 scoring takes 2.46/5.24/5.36 GPU-hours and processes 7.27M/20.21M/20.73M tokens; the subsequent training takes 2.14/0.07/2.39 GPU-hours. Scoring totals 171,955 forwards and 13.06 GPU-hours; training adds 4.60. Scoring is gradient-free, with batch size one and no key–value cache reused across passes. The stored records contain candidate identifiers, losses/effects, and signed patterns rather than hidden-state tensors. Scoring therefore needs less GPU memory than training, its records are all scalars, and selection is a single CPU pass over them; we report GPU-hours as the primary cost.
Random does not re-score. Q-matched Random receives Full's actual per-source counts; Proportional Random receives the same total historical count, allocated by source size. Both sample uniformly within sources. We have corrected "each branch re-scores" at line 364 of the submitted version: scoring applies only to Full and to the intervention-based ablations in Table 3 (gate only, w/o harm, gate + rank), which diverge after S2 and therefore score at their own checkpoints. Deployment requires scoring only along Full's chain, i.e., the 13.06 GPU-hours above; re-scoring for the ablation branches is an experimental cost and is not counted as part of the method's cost.
At matched total compute, more Random epochs do not close the gap. Eight-epoch Random takes 18.40 GPU-hours, 0.74 more than Causal k=4, and reaches 43.48 ± 0.40 Macro; Causal k=4 takes 17.66 and reaches 45.88 ± 0.71, a paired gain of +2.40 [0.70, 4.10] pp. On token matching: eight-epoch Random trains on four times the tokens of two-epoch Random without improving Macro (43.48 vs 43.69), so the gap does not stem from the total number of training tokens.
Restricted scoring retains the selection advantage. Scoring only half/one third of S4 history lowers the total cost to 14.98/14.09 GPU-hours; Causal then reaches 42.51/39.20 Macro versus 39.88/36.92 for matched Random (seed 43), a relative gain of +2.63/+2.28 pp (+2.68 with full scoring); the absolute Macro of both methods decreases similarly as the replay set shrinks. For equal-size tasks, the cumulative cost of full re-scoring grows quadratically with the number of tasks; scoring only a fixed M candidates per stage gives (k+1)M(T−1) cumulative forwards, which grows linearly with the number of tasks. These experiments support that the selection advantage is retained when only a subset of history is scored; combined with this analytical result, restricted scoring offers a promising, cost-controlled path toward longer task streams.
Response to Reviewer WepN (3/5) — Q4–Q6: Intervention Granularity, k, and Baseline
Q4. Individual slots versus an event-level block
A block-level intervention can enforce its own block-level harm constraint. Our independent-slot rule instead checks each nominated event-slot individually, so positive and negative effects within the same event can be detected separately. The diagnostic in Section 3 shows that repair and damage rates differ across slots of the same event (Figure 2a,b), so a block-level result can aggregate opposite effects.
Removing harm avoidance in Table 3 lowers Macro by 1.09 pp, showing that the harm constraint contributes to downstream replay; block-level masking cannot apply this constraint slot by slot. A comparable-cost replay control between block-level and slot-level rules is not yet complete, so we have restricted the wording in Section 4.3 to the difference between the constraints the two rules can enforce.
Q5. Final replay sensitivity to the candidate budget k
The additional end-to-end results show an accuracy–cost tradeoff. Each historical example requires one baseline forward plus k intervention forwards:
| k | Forwards/example | Scoring GPU-h | Total GPU-h | Macro, seeds 42–44 |
|---|---|---|---|---|
| 2 | 3 | 7.84 | 12.44 | 44.54 ± 0.53 |
| 4, default | 5 | 13.06 | 17.66 | 45.88 ± 0.71 |
| 8 | 9 | 23.51 | 28.11 | 47.16 ± 0.52 |
Increasing k from 2 to 4 costs 5.22 additional GPU-hours and raises Macro by 1.34 points; increasing it from 4 to 8 costs 10.45 additional GPU-hours and raises Macro by 1.28 points. This corresponds to roughly 0.26 and 0.12 points per additional GPU-hour. k=4 is a compromise operating point; k=8 is better when compute permits. All three values exceed every tested Random configuration (at most 43.84).
Admission rates (%, admitted / scored, seed 43; G/K/P = GSM8K/KodCode/GPQA; ≥ marks a count truncated at the per-source cap):
| k | S2, G | S3, G / K | S4, G / K / P |
|---|---|---|---|
| 2 | 25.1 | 0.77 / 0.73 | ≥29.7 / 26.5 / 52.5 |
| 4, default | 25.3 | 0.79 / 0.77 | ≥29.7 / 26.2 / 53.4 |
| 8 | 25.8 | 0.79 / ≥0.80 | ≥29.7 / 27.1 / 55.2 |
Admission rates change little with k. S2 shares the Stage-1 checkpoint across k, so its rise from 25.1% to 25.8% reflects k alone; at S3 and S4 the incoming checkpoints also depend on earlier replay.
How k was chosen. We need to correct one statement in Section 5.1: k ∈ {2, 4, 8} and the remaining selector settings were not chosen on a separate held-in validation split, but during development by weighing scoring cost against Macro, and the Figure 1 diagnostic also uses test questions. All three k values in the table exceed every Random configuration, so the conclusion does not depend on the specific k chosen. Once chosen, the settings were fixed across stages, seeds, and branches and reused unchanged, without retuning, for the permuted order, Qwen3-4B, and Qwen3.5-9B, where the gains are +1.95 [0.80, 3.10], +4.71 [2.42, 6.99], and +2.45 (single seed), respectively.
Q6. What exactly does zero replacement intervene on?
The intervention replaces content relative to a specified baseline; it does not delete the slot. The implementation is _intervene in memgen/model/modeling_memgen.py. Specifically:
- Projection: the contextualized Weaver state is zeroed before the biased Weaver→reasoner projection, so the reasoner receives the projection bias at that position.
- Mask and position: attention masks and positions are unchanged; the slot remains present and can still be attended to.
- Cache: every scoring forward is an independent, full teacher-forced pass over the whole sequence (
score_exampleinreplay/scoring.py), and no key–value cache is carried across passes. Effects may propagate through later injection events. - Targets: logits at latent positions are excluded, and the losses before and after intervention use the same reference-completion tokens and labels.
We have replaced "removing the candidate" at lines 287–289 of the submitted version with "zeroing the candidate" and added:
Helpfulness and harm are both relative to this zero-replacement reference: the reasoner still receives the projection bias, and positions and attention masks are unchanged.
The intervention is used only for admission: the replayed example is the original, unmasked one, and the rule's value is tested by downstream replay results (Tables 2 and 3). Mean replacement and graded attenuation have not yet been compared as alternative references; following the revised text above, helpfulness and harm are explicitly defined relative to the zero-replacement reference.
Response to Reviewer WepN (1/5) — Q1: Direct Selector Comparison
Thank you for the detailed questions and for recognizing the distinction between identifying influential memory states and determining the direction of their effects. This response is posted in five parts: Part 1 addresses Q1, Part 2 Q2–Q3, Part 3 Q4–Q6, Part 4 Q7, and Part 5 Q8–Q9.
We add both alternatives requested in Q1. First, under the same per-source counts at S4, all selectors start from Full's S3 checkpoint of the same seed, match Full's per-source replay counts
m_4on that seed (seed 43:(1,765, 1,651, 104)), and share the training-set size and training recipe (seeds 42–44). S4 is the final stage, so the effect appears directly in the final results without interference from later-stage divergence; it is also the only stage that replays all three historical sources (S3 has only two historical sources and replays only 99 examples). Selection cost is counted as forwards over the 13,944 historical examples at S4; Similarity additionally embeds the 8,000 current-task prompts:13,944forwards54.38 ± 0.0954.60 ± 0.1523.91 ± 0.5843.04 ± 0.2343.98 ± 0.12+1.90 [0.12, 3.68]27,888forwards + one virtual update54.44 ± 0.2754.48 ± 0.0824.41 ± 0.7743.18 ± 0.6044.13 ± 0.35+1.76 [0.84, 2.67]21,944prompt-only forwards, frozen reasoner51.07 ± 0.3653.17 ± 0.2526.43 ± 0.5841.86 ± 0.6043.13 ± 0.19+2.75 [0.79, 4.71]69,720forwards,5.36GPU-h54.54 ± 0.5355.32 ± 0.5527.61 ± 1.0546.06 ± 1.0445.88 ± 0.71Causal replay is above all three selectors on every seed. Second, under the full-pool protocol, the supplementary seed-43 results are:
53.1553.7527.2737.8042.9951.0252.9028.7943.7044.1055.1956.0530.8146.0647.03This merged-pool protocol, with approximately 4,000 examples, differs from the per-source-matched S4 comparison; within this protocol, the Macro gains are
+2.93pp over Similarity and+4.04pp over Random (single seed). Diversity-based selection (GSS) requires per-example gradients and is not included in this round; in our method, set-level diversity is enforced by Pattern Cap, which the ablation in Table 3 shows contributes a+0.78pp gain. We agree that EWC and LwF are not replay-selection methods: the revision explicitly positions them as classical continual-learning references in the caption of Table 6, and the direct comparison with replay selectors is given by the new tables above.