acceptodds
Under review as a conference paper at ICLR 2027

CausalMem: Counterfactually Validated Memory for Training-Free Visual Reasoning Agents

Abstract

Autonomous visual agents should improve as they accumulate experience, yet updating large vision-language models after every interaction is impractical. External memory offers a training-free route to self-evolution, but naive memory accumulation can amplify visual shortcuts: an agent may remember background, style, layout, or co-occurring objects rather than the evidence that actually determines the answer. We introduce , a training-free memory framework that writes visual memories only after counterfactual validation. CausalMem alternates between wake-time reasoning and sleep-time consolidation. During wake, a frozen VLM solves new image reasoning tasks by retrieving validated semantic rules, hard negatives, and anchor episodes. During sleep, the agent proposes causal and nuisance visual factors, edits stored images as noisy interventions, and re-answers the edited cases. A rule is promoted to long-term memory only if the answer is invariant to nuisance edits and sensitive to edits of the hypothesized causal evidence. We formalize this process as noisy interventional memory validation and derive a finite-sample bound on the false acceptance of spurious rules. Across seven visual reasoning benchmarks, CausalMem improves Qwen2.5-VL-7B and Qwen3-VL-8B by 4.18 and 3.29 average points over vanilla inference without updating model parameters. Component ablations, stream-order and inductive-split controls, cross-benchmark transfer, and experiments with other editor and VLM families support that, on the tested settings, the gains come from counterfactually validated memory rather than passive retrieval, suggesting that self-evolving visual agents must learn not only how to recall experience, but which rules deserve to be remembered.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
visual reasoning agents, agent memory, counterfactual validation, vision-language models, shortcut learning, training-free adaptation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_qBJ9OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

CausalMem is a training-free memory framework for frozen visual reasoning VLMs that writes memory only after counterfactual validation: in a wake phase the frozen VLM answers while retrieving validated semantic rules, hard negatives, and anchor episodes; in a sleep phase (every 10 tasks) it proposes causal/nuisance factors, edits stored images with a frozen image editor (environment edit vs object edit), re-answers the edits, and promotes a rule only if it is invariant to nuisance edits and sensitive to causal edits, subject to fidelity filtering (tau_q=0.70) and a conflict check. A finite-sample bound (Prop. 1: Pr[accept] <= exp(-2n(tau-p_s)^2)+neta) formalizes how edit fidelity and validation diversity control false acceptance. On seven benchmarks, CausalMem improves frozen Qwen2.5-VL-7B by +4.18 and Qwen3-VL-8B by +3.29 average points over vanilla (best in 6/7 and 5/7 columns), with the largest gains on fine-grained visual search (V +8.90/+8.38); ablations show passive episodic RAG is 9.40 pts below vanilla, whereas counterfactual validation turns memory into a net gain (+15.65 over passive RAG).

Critical strengths
  1. The core idea - treat "which rules deserve to be remembered" as an interventional falsification problem rather than a retrieval problem - is a genuinely novel and well-articulated framing for agent memory; the shortcut-amplification failure mode it targets is real and rarely addressed. 2. Evaluation discipline is strong: transductive online protocol with no label leakage, stream-order and inductive-split controls, cross-benchmark transfer, alternative editor and VLM families, plus the striking ablation that passive RAG hurts while validated memory helps.
Critical weaknesses
  1. The evaluation protocol is transductive and online (memory updated during evaluation, same fixed stream for all methods) - the gains could in principle depend on stream order and the specific 10-task sleep cadence; the inductive-split control numbers were not visible in the read portion, so the robustness claim needs verification. 2. Cost: sleep-time editing + re-answering runs per hypothesis (K_e=K_o=4, M=3) adds real compute; the paper mentions cost measurements, but the numbers were not in the read portion - this matters for a "training-free" claim that is really "inference-time-costly."
Questions for authors
  1. How much does memory growth matter: does performance saturate as S grows, and how does the pruning budget B interact with the 0.80 retrieval cutoff? 2. What fraction of proposed hypotheses are actually accepted (selectivity), and how often does a rule get falsified after promotion (churn)? 3. In the inductive-split control, are the gains maintained when memory is frozen before evaluation - and is the gap to transductive evaluation quantified?
3 comments
AuthorsOpenReview ↗

Response to Reviewer qBJ9 (1/3)

Dear Reviewer qBJ9,
Thank you for the careful reading and the positive assessment of the framing and of the evaluation protocol. The two weaknesses concern numbers that are in the submission but outside the read portion (the cost table in Appendix F). We reproduce both below with new measurements, and the Rebuttal Revision PDF places them in the main text so they are visible without the appendix.

W1.\textcolor{green}{\text{W1.}} Gains may depend on stream order and on the 10-task sleep cadence; the inductive-split numbers need verification

The inductive rows are the submission's Table 2 (p. 9).

Table C. Protocol controls (Qwen2.5-VL-7B; from Tables 1 and 2).

SettingV*HallusionBench
Vanilla73.8266.14
Transductive, default order (Table 1)82.72 (+8.90)71.82 (+5.68)
Transductive, 5 random orders, mean ± std81.67 ± 1.17 (+7.85)71.88 ± 0.38 (+5.74)
Transductive, worst of the 5 orders80.10 (+6.28)71.50 (+5.36)
Inductive A↔B: memory frozen, wake only81.15 (+7.33)71.71 (+5.57)
  • Stream order. CausalMem is above vanilla for every one of the five random permutations. The worst order still gains +6.28 / +5.36, and the default order of Table 1 lies within one standard deviation of the permutation mean on both benchmarks.
  • Inductive split. Memory is built without labels on one random half, frozen, and the disjoint half is answered with wake only; both directions are averaged. The gains are maintained: +7.33 on V* and +5.57 on HallusionBench.
  • Gap to transductive evaluation. The gap is 1.57 points on V* and 0.11 on HallusionBench, so the frozen memory keeps 82% and 98% of the transductive gain. This holds although the inductive memory is built from only half of the data and never sees an evaluated image.
  • Sleep cadence. In the inductive split, all consolidation is finished before evaluation starts, so the cadence cannot interact with the evaluated stream; the gains persist. The same holds for the transfer results of Table 2 (MMStar→V*: +5.24 with a frozen memory). In the online protocol, the interval only determines how long an episode waits before its rule becomes available (at most 10 tasks here). It is also not critical to the result: on V* with Qwen3-VL-8B, intervals of 5, 10 (default), 20 and 50 tasks give 84.82, 86.39, 85.86 and 83.25, all far above vanilla (78.01). The default is at the optimum: very frequent consolidation spends edits on hypotheses that few episodes support, while very infrequent consolidation delays a rule's first use.

W2.\textcolor{green}{\text{W2.}} Cost: sleep-time editing and re-answering add real compute to a "training-free" method

The numbers are in Table 5 (Appendix F). We moved them into the main text of the Rebuttal Revision PDF:

Table D. Cost per sample on V* (Qwen3-VL-8B; Table 5).

VanillaCausalMem wakeCausalMem sleep
VLM calls1122.23
Editor calls001.98
Extra prompt tokens02,23463,460
Time0.71 s1.32 s20.97 s VLM + 130.87 s editor

Four points put these numbers in context.

1. The user-facing cost is one VLM call. Wake takes 1.32 s against 0.71 s (1.86×) because of 2,234 extra prompt tokens. This context does not grow with the stream, since pruning keeps the active store near constant (Fig. 4b).

2. The editor dominates the sleep time and is replaceable. The editor accounts for 130.87 of the 151.84 s (86%). With InstructPix2Pix instead of Qwen-Image-Edit (Table 2), the whole sleep phase, VLM and editor together, takes 6.39 s per sample, and accuracy is 84.82 on V* and 61.47 on MM-Vet (86.39 and 62.39 with Qwen-Image-Edit).

3. Comparison with two verified-memory baselines. During the rebuttal we ran HtT (arXiv:2310.07064) and M$^2$Note (arXiv:2607.00685) under the protocol of Table 1, label-free, and timed them in the same environment. Time is model-call time per question for the entire pipeline (answering plus memory update).

AuthorsOpenReview ↗

Response to Reviewer qBJ9 (2/3)

Table E. Accuracy and cost on V* (Qwen3-VL-8B).

MethodAcc. (%)User-facing latency / questionTotal time / questionExtra prompt tokens / question
Vanilla78.010.71 s0.71 s0
M$^2$Note80.103.68 s†3.68 s14.0k
HtT79.068.46 s†8.46 s14.3k
CausalMem, InstructPix2Pix84.821.32 s7.71 s (1.32 wake + 6.39 sleep)16.8k
CausalMem, Qwen-Image-Edit86.391.32 s153.16 s (1.32 wake + 151.84 sleep)65.7k

†HtT and M$^2$Note ran their update steps inline with answering in our measurement, so their per-question cost appears in both time columns; CausalMem's sleep is asynchronous and batched offline.

With the lightweight editor, CausalMem takes less time per question than HtT and is 5.76 points more accurate; against M$^2$Note it uses 2.1× the time for +4.72 points. Qwen-Image-Edit adds a further 1.57 points for about 20× the time, so editor quality is a knob with diminishing returns. CausalMem uses 16.8k extra prompt tokens per question with InstructPix2Pix (65.7k with Qwen-Image-Edit), against about 14k for the baselines. Because sleep is asynchronous and batched offline, the quantity on the user-facing critical path is the wake latency: 1.32 s, against 3.68 s for M$^2$Note and 8.46 s for HtT under the same measurement (V*, Qwen3-VL-8B). The trade-off plots are added to the experiments section of the Rebuttal Revision PDF and in our anonymous repository: total pipeline\textcolor{blue}{\text{total pipeline}} and user-facing latency\textcolor{blue}{\text{user-facing latency}} (click to the anonymous repository). Across all seven benchmarks and both backbones, these two baselines change the vanilla average by −1.99 to +0.65 points, against +3.29 / +4.18 for CausalMem, and CausalMem is ahead of both in all 14 benchmark–backbone cells. This is further support for the observation the reviewer highlighted: what makes the memory useful is the validation, not the retrieval.

4. The sleep cost is amortized. It is paid once per consolidated episode and not per query. The inductive split (Table C) keeps 82–98% of the gain with no sleep on any evaluated query, and one shared memory built from 300 unlabeled samples raises pooled accuracy over the full V*, MMStar and MMBench sets from 78.69 to 80.90 (Table 2).

In the Rebuttal Revision PDF the cost table moves into the main text, together with the cost of the two new verified-memory baselines and the trade-off plots, and the consolidation cost is stated next to the training-free claim in the introduction.

Q1.\textcolor{green}{\text{Q1.}} Does performance saturate as S grows, and how does the pruning budget B interact with the 0.80 retrieval cutoff?

Growth and saturation. More memory is not better. Without pruning, the store keeps growing (559 rules on MMStar, Fig. 4b), and accuracy on the V*/MM-Vet average drops by 4.55 points (69.84 vs. 74.39, Fig. 3a), because redundant and conflicting entries compete for the prompt. With pruning, S itself saturates: the active MMStar store ends at 97 rules (82.6% pruned, Fig. 4b), and the advantage over vanilla persists to the end of the stream (+8.3 points on the V* moving average, Fig. 3b). Returns also diminish in the number of consolidated episodes: a memory built from half of the data keeps 82–98% of the gain (Table C), and a memory built from 300 samples already lifts pooled accuracy on three full benchmarks by 2.21 points.

BB and the cutoff act at different stages. BB is a store-level budget enforced during sleep; it bounds how many items exist (97 rules on MMStar at B=12000B=12000 tokens). The cutoff and the top-k limits act per query at wake time: at most 4 rules, 2 hard negatives and 2 anchors enter a prompt, which is why the wake context averages 2,234 tokens although BB is 12,000. BB therefore controls the size of the candidate pool and not the length of the prompt. The cutoff is a cosine distance of 0.80, i.e., a similarity of at least 0.20. It is a loose relevance floor: it matters mainly when the store has few relevant rules (early in the stream, or for queries outside the stored scopes), where it keeps unrelated rules out of the prompt. Among the items that pass it, selection is made by the ranking score of Eq. 3. The interaction is thus one-directional: a larger BB lets more low-value rules pass the cutoff and compete for the four slots.

Table F. BB × cutoff on V* (Qwen3-VL-8B), accuracy (%).

BB \ cutoff (cosine distance)0.700.80 (default)0.90
4,000 tokens78.0181.1580.10
12,000 tokens (default)85.3486.3984.29
No pruning78.5379.5878.01
AuthorsOpenReview ↗

Response to Reviewer qBJ9 (3/3)

The default sits at the grid optimum, and each direction alone costs 1.05–5.24 points (budget 12k → 4k: −5.24; cutoff 0.80 → 0.90: −2.10; 0.80 → 0.70: −1.05). The cutoff matters more when the budget is tight: the spread across cutoffs is 3.14 points at 4,000 tokens against 2.10 at 12,000, since a small store needs a permissive floor to be used at all. Removing the budget hurts at every cutoff, including a tighter one (best unpruned cell 79.58, against 86.39 at the default), which mirrors the −4.55-point no-pruning ablation on the two-task average (Fig. 3a).

Q2.\textcolor{green}{\text{Q2.}} What fraction of proposed hypotheses is accepted, and how often is a rule falsified after promotion?

Selectivity. The gate accepts 45.5% of the candidates on V* (87 accepted, 104 rejected) and 37.3% on MMStar (559 accepted, 941 rejected); see Fig. 4a. On V*, 94.23% (98/104) of the rejected hypotheses match one of five shortcut categories aligned with MM-SpuBench, mostly background/scene (79/104); this keyword audit is a coarse proxy and is described with its caveats in Appendix F.

Churn. After promotion, a rule can leave the active set in three ways: it fails a later sleep re-validation (a contradiction event, which also lowers its LCB), it loses a conflict to a rule with a higher LCB and becomes dormant, or it is evicted by the budget as redundant or low-value. The breakdown by cause is:

PromotedFailed a later re-validationDormant after a conflictEvicted by budget / redundancyActive at the end
V*87216069
MMStar559135739297

Falsification after promotion is rare (2 of 87 on V*, 13 of 559 on MMStar), and a rule that loses a conflict simply becomes dormant (16 and 57). Nearly all removals on MMStar are budget evictions of redundant or low-value rules (392), which is how the active store stays bounded as the stream grows; on V* the store never reaches the budget.

Q3.\textcolor{green}{\text{Q3.}} In the inductive split, are the gains maintained when memory is frozen, and is the gap to transductive evaluation quantified?

Yes to both; the numbers are in Table C above. With memory built on one half, frozen, and the other half answered with wake only (Qwen2.5-VL-7B, both directions averaged), CausalMem reaches 81.15 on V* and 71.71 on HallusionBench, against 73.82 and 66.14 for vanilla: gains of +7.33 and +5.57. The transductive results on the same benchmarks are 82.72 and 71.82 (+8.90 and +5.68). The gap is therefore 1.57 points on V* and 0.11 points on HallusionBench, i.e., the frozen memory keeps 82% and 98% of the transductive gain while seeing only half of the data and none of the evaluated images. The larger gap on V* is consistent with its small size (191 questions), where halving the data available for consolidation matters more. In the submission we report this gap explicitly in the main text (Sec. 4.4).


Summary of revisions for Reviewer qBJ9. (i) The transductive–inductive gap and the worst-order result are stated in the main text of the submission (Sec. 4.4). (ii) The cost table and the trade-off plots move to the main text of the Rebuttal Revision PDF, together with the cost of the two new verified-memory baselines. (iii) Selectivity and churn statistics are reported in Sec. 4.5. We thank the reviewer again and are happy to provide further details.

Reviewer_xRMPOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

CausalMem is a training-free wake–sleep framework for visual reasoning agents. It uses image edits to test answer invariance to nuisance factors and sensitivity to hypothesized causal evidence, then stores semantic rules for later retrieval alongside hard negatives and anchor episodes.

Critical strengths
  • This framework addresses a relevant problem: accumulated visual memories can reinforce shortcuts. Testing candidate memories before reuse provides a concrete approach to selective memory writing without updating model parameters.
  • The reported improvements span seven benchmarks and two main backbones. Stream-order controls, frozen-memory inductive evaluation, cross-benchmark transfer, and alternative editor and VLM families provide meaningful evidence of the complete system’s usefulness.
Critical weaknesses
  • The necessity test can accept an answer change that does not represent a valid counterfactual outcome. In Figure 5, deleting the suitcase and book leaves an empty trunk, yet “the suitcase is beneath the book” is marked as a successful test. None of the answer choices describes the edited image, despite the verifier’s answerability requirement. This example undermines the interpretation that passing the gate validates the claimed causal evidence.
  • The experiments do not isolate the benefit of counterfactual validation from semantic abstraction, pruning, and additional inference. Passive episodic RAG differs substantially from the full memory system, while ReasoningBank already demonstrates abstraction into reusable reasoning strategies. Removing either edit family supports its usefulness within CausalMem, but does not establish an advantage over non-counterfactual semantic memory under comparable computation.
  • Quality-aware pruning can penalize a memory that corrects an earlier error. Replay gain rewards agreement with stored model answers, so changing a replay prediction from an incorrect stored answer to a correct answer contributes negatively to retention. This criterion does not distinguish genuine correction from damage to a correct prediction, leaving its alignment with reliable self-evolution unestablished.
  • Conflict management can classify compatible rules as conflicting when their scopes match but their evidence descriptions are sufficiently dissimilar. The rule with the lower reliability bound can then be rejected or made dormant without establishing that the rules make contradictory predictions. Preservation of valid, complementary knowledge during consolidation is therefore not established.
Questions for authors

See Weakness

6 comments
AuthorsOpenReview ↗

Response to Reviewer xRMP (1/4)

Dear Reviewer xRMP,
Thank you for the comments. We answer the four weaknesses in order.

W1.\textcolor{green}{\text{W1.}} The necessity test can accept an answer change that is not a valid counterfactual outcome (Fig. 5)

The gate measures dependence on the claimed evidence, not correctness on the edited image. By Eq. 6, Nec^(h)\widehat{\mathrm{Nec}}(h) is the fidelity-weighted fraction of evidence edits on which the frozen VLM no longer gives its original answer AA; the term is 1−δQ(f(Ijobj,Q),A)1-\delta_Q\big(f(I^{\mathrm{obj}}_j,Q),A\big). In Fig. 5 the original answer is AA ("The suitcase is on the book"). After the suitcase and the magazine are removed, the model stops answering AA. This is the outcome that a rule naming "the suitcase and the magazine" as evidence must produce. Had the model kept answering AA on an empty trunk, its answer would have been driven by something other than the claimed evidence, and the hypothesis would have been rejected.

The forced-choice response D is never used as knowledge. It is not treated as a label and it is not shown at wake time: the wake prompt contains rule texts, hard negatives and anchor episodes (Sec. 3.3), and Eq. 8 writes to hard-negative memory only environment-edited images whose answer stayed unchanged. An evidence edit contributes one pass/fail outcome to the gate and to the reliability counts. Whether option D describes the edited image therefore has no influence on what enters memory.

Removal is an intended valid intervention and is consistent with Assumption 1 and with the answerability check. Assumption 1 asks a faithful evidence edit to alter z+z^+ "enough to change the true answer when z+z^+ is necessary". Once the two objects are gone, the true answer to the question is no longer AA, because no listed relation holds. The true answer has changed, and this is all the necessity test requires. The answerability check of the verifier concerns the edited image: it must remain a coherent scene on which the question can still be posed, which screens out corrupted or implausible edits. It cannot mean that one of the original options must still be true, since an evidence edit is by construction meant to invalidate the original answer. Eq. 5 allows both a minimal alteration (Fig. 2, curly to straight) and a removal (Fig. 5), and deleting or masking the critical object is a standard necessity intervention in counterfactual VQA (e.g., Chen et al., 2020, cited in the paper).

What passing the gate means. The paper frames the gate as falsification: a rule is written "only after the agent has tried, and failed, to falsify it", and z±z^± are "the model's claims about OO and CC, tested next". Proposition 1 bounds the acceptance of a spurious hypothesis, one whose claimed evidence is in fact nuisance. For such a hypothesis, removing the claimed evidence leaves the answer unchanged and the necessity gate fails. This is the failure the gate is built to catch, and removal is the most direct way to expose it. AA passed gate means "not falsified by Ke+KoK_e + K_o interventions"; the LCB then keeps rules with few checks conservative. We do not claim that the gate certifies a complete causal mechanism, and Sec. 2 states that we do not propose a causal discovery method.

Evidence that the test does useful work. Removing the evidence edits lowers accuracy by 2.52 points under ground-truth evaluation (71.87 vs. 74.39, Fig. 3a), and the write gate rejects 54.5% of the candidates on V* and 62.7% on MMStar (Fig. 4a).

W2.\textcolor{green}{\text{W2.}} The experiments do not isolate counterfactual validation from semantic abstraction, pruning and additional inference

The ablations already separate these factors inside CausalMem. The variants "w/o env. edits" and "w/o obj. edits" keep semantic abstraction, retrieval and pruning unchanged and alter only the validation; they lose 3.77 and 2.52 points (the hard negatives that environment edits also supply account for 0.50 of the former, Table 4). Pruning is ablated separately (−4.55), and so is the wake context (Table 4). Passive RAG is not our isolating control; it illustrates the failure mode (9.40 points below vanilla).

New experiments: non-counterfactual semantic memory, with its cost measured. The remaining question is whether a semantic memory that abstracts, verifies and spends extra inference, but does not intervene, would do equally well. We ran two such methods on all seven benchmarks and both backbones (28 new runs), under the protocol of Table 1 (same frozen VLM, empty initial memory, single pass, no labels, no weight updates):

AuthorsOpenReview ↗

Response to Reviewer xRMP (2/4)

  • HtT (arXiv:2310.07064) abstracts reusable rules and keeps those with occurrence ≥ 2 and confidence ≥ 0.7. We use the upstream RuleLibrary; since the original method verifies rules against ground-truth answers, confidence is computed against the strict majority of 3 independent memory-free answers to stay label-free. The library is capped at 500 rules / 12,000 tokens, the same budget as ours.
  • M$^2$Note (arXiv:2607.00685) abstracts self-judged mistakes into subject–guidance notes, retrieves them by multimodal RAG, and commits a notebook update only if re-answering the whole batch does not lower accuracy.

Table A. Non-counterfactual semantic memories under the protocol of Table 1 (official scores, %). Δ Avg. is relative to vanilla and computed from unrounded averages.

BackboneMethodV*MMStarMMBenchRealWorldHallusionMM-VetMMMUAvg.Δ Avg.
Qwen3-VL-8BVanilla78.0162.5384.2667.7171.6158.2660.4468.97–
HtT79.0660.8079.7667.4569.6159.6358.8967.89−1.09
M$^2$Note80.1062.6084.0168.1072.6658.2661.6769.63+0.65
CausalMem86.3963.8088.3968.3773.2962.3963.2272.26+3.29
Qwen2.5-VL-7BVanilla73.8262.1381.1164.4466.1460.5555.1166.19–
HtT74.3557.5370.6463.2766.9861.4755.1164.19−1.99
M$^2$Note74.8761.9380.2664.3168.2461.9354.7866.62+0.43
CausalMem82.7263.8785.1067.8471.8263.7657.4470.36+4.18

CausalMem is ahead of both methods in all 14 cells, by +2.64 / +3.75 points on average over M$^2$Note and +4.38 / +6.17 over HtT. Both baselines abstract, verify and spend additional inference; what they lack is the intervention. Their average change over vanilla is between −1.99 and +0.65, against +3.29 / +4.18 for CausalMem.

Comparable computation. We measured model-call time per question for the entire pipeline (answering plus memory update) on V* with Qwen3-VL-8B.

Table B. Accuracy and cost on V* (Qwen3-VL-8B).

MethodAcc. (%)User-facing latency / questionTotal time / questionExtra prompt tokens / question
Vanilla78.010.71 s0.71 s0
M$^2$Note80.103.68 s†3.68 s14.0k
HtT79.068.46 s†8.46 s14.3k
CausalMem, InstructPix2Pix84.821.32 s7.71 s (1.32 wake + 6.39 sleep)16.8k
CausalMem, Qwen-Image-Edit86.391.32 s153.16 s (1.32 wake + 151.84 sleep)65.7k

†HtT and M$^2$Note ran their update steps inline with answering in our measurement, so their per-question cost appears in both time columns; CausalMem's sleep is asynchronous and batched offline.

Sleep is asynchronous and batched offline. The quantity on the user-facing critical path is wake latency, which is 1.32 s: lower than M$^2$Note (3.68 s) and HtT (8.46 s) under the same measurement (V*, Qwen3-VL-8B). On the total pipeline, with InstructPix2Pix as the editor, CausalMem takes less time per question than HtT (7.71 s vs. 8.46 s) and is 5.76 points more accurate; against M$^2$Note it uses 2.1× the total time for +4.72 points. CausalMem's extra prompt tokens are spent offline during sleep; its wake prompt adds 2.2k tokens. The advantage over non-counterfactual semantic memory therefore holds at comparable wall-clock cost, and additional inference alone does not explain it: HtT spends more time than CausalMem with InstructPix2Pix and gains 1.05 points on V*.

The two trade-off curves are added to the experiments section of the Rebuttal Revision PDF and our anonymous repository: total pipeline\textcolor{blue}{\text{total pipeline}} and user-facing latency\textcolor{blue}{\text{user-facing latency}} (click to the anonymous repository). Each shows vanilla, HtT, M$^2$Note and CausalMem with the two editors; the halo area of a marker is proportional to its extra prompt tokens per question.

Internal control: rules without validation. Accepting every hypothesized rule without the gate, with hypothesis generation, memory format, retrieval and pruning unchanged, gives 64.60 on the V*/MM-Vet average (Qwen3-VL-8B), against 74.39 for the full system and 68.14 for vanilla. The generator, the rule format and the wake retrieval alone are 3.54 points below vanilla; the gate is what turns the memory into a net gain.

AuthorsOpenReview ↗

Response to Reviewer xRMP (3/4)

On ReasoningBank. We cite and discuss ReasoningBank in Sec. 2. ReasoningBank distills strategies from trajectories that an LLM judges as successes or failures and applies no test on the input; M$^2$Note belongs to the same family, is designed for VLMs, and adds a stronger check (batch re-execution with rollback). To our knowledge, none of these memories intervenes on the visual input to test a candidate item before writing it, and Table A shows what that step is worth.

W3.\textcolor{green}{\text{W3.}} Quality-aware pruning can penalize a memory that corrects an earlier error

gain(m)\text{gain}(m) measures replay agreement and is only one component of the retention score in Eq. 14, which also includes reliability from counterfactual validation.

1. The role of the term. By Appendix C, the replay set Rm\mathcal{R}_m holds up to 32 earlier episodes drawn by reservoir sampling, and it excludes the source and validation episodes of m. These episodes were answered without m and are, for the most part, unrelated to it. On such episodes the desired behavior of a new rule is to leave the answer alone. gain(m)\text{gain}(m) is the difference between two deterministic runs that differ only by mm (Eq. 15). It is therefore zero whenever m leaves a replayed answer unchanged, whether that answer is right or wrong, and it is negative only when m flips answers on past episodes. It is a label-free non-interference term, not an estimate of correctness. The evidence that m is valid enters the retention value through LCB(πm)\text{LCB}(π_m), which comes from the counterfactual checks.

2. Without labels, a single flip cannot be classified as a correction or as damage. That would require the ground truth, which the protocol excludes (Sec. 3.1), and the same holds for any label-free criterion. In the two verified-memory baselines we added (W2), self-agreement is in fact the primary acceptance signal: the majority of three memory-free answers in our label-free HtT, and a self-judged reward in M$^2$Note. In CausalMem it is a secondary pruning term next to interventional evidence. Since stored answers are correct in the majority of cases (vanilla accuracy is 55–84% across our benchmarks) and a flip away from a wrong multiple-choice answer need not land on the right option, penalizing flips on unrelated past episodes is the conservative choice.

3. The magnitude is small. One flipped replay episode changes gain(m)\text{gain}(m) by 1/32≈0.031/32≈0.03 at the full reservoir size. The reliability term has weight 1.0, and a rule that passed all eight edits has LCB≈0.77\text{LCB}≈0.77 (10th percentile of Beta(9,1)\text{Beta}(9, 1)). A rule that corrects one or two past errors thus loses 0.03–0.06 in retention value and is not pruned for that reason. To be pruned through this term, a rule has to flip a large share of unrelated past answers. That is the signature of an over-broad rule, which the term is meant to remove.

4. The criterion is aligned with ground truth empirically. All accuracies in the paper are computed with ground-truth labels, which the pruning criterion never sees. Removing pruning lowers accuracy by 4.55 points (74.39 to 69.84, Fig. 3a). A criterion that systematically discarded error-correcting rules would not be expected to produce this gain.

We also measured the concern directly, with labels and for analysis only. Of the 39 answer flips caused by candidate rules in replay on V* and MMStar, 71.79% changed a correct stored answer into an incorrect one and 28.21% corrected an incorrect one: the flips are dominated by damage, so penalizing them is the conservative choice on this statistic. Removing the gain(m)\text{gain}(m) term from Eq. 14 lowers accuracy by 3.83 points.

W4.\textcolor{green}{\text{W4.}} Conflict management can classify compatible rules as conflicting

Eq. 2 compares the evidence specified by rules within a matched scope and gives priority to rules with stronger counterfactual support.

1. What a rule asserts. Every rule has the same form: a scope σ, the evidence z+ to inspect, and the nuisance z− to ignore (Sec. 3.2 and the rule format in Appendix E). Two rules with the same scope and dissimilar evidence (cosine below 0.50) give different instructions about what determines the answer for the same class of questions, and each may list the other's evidence among the factors to ignore. The paper's example is hat color versus background color for a hat-color question. Since retrieval is keyed on scope and question similarity, both rules would be retrieved for the same queries. This is what the conflict term prevents, whether or not the two rules happen to predict different answers on a particular image.

Reviewer_rBp2OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose CausalMem, a training-free memory framework for frozen VLMs that writes a visual “rule” to long-term memory only after testing it with counterfactual image edits. In a periodic sleep phase, the VLM proposes the causal evidence and the nuisance factors behind one of its past answers. An image editor then produces nuisance edits and evidence edits, and the rule is accepted only if the answer stays the same under the nuisance edits, changes under the evidence edits, and doesn’t conflict with a stronger rule. At inference time, accepted rules, edited hard negatives, and anchor episodes are retrieved into the prompt.

Critical strengths
  1. The problem of unfiltered memory shortcuts are well motivated by the authors with clear examples.
  2. It is intuitive to breakdown the aspects of good memory system to casual evidence and correlative context. And operating on the casual evidence when necessary is very intuitive.
  3. For the experiments that the authors showed across seven benchmarks and two backbones, the gains are very consistent.
Critical weaknesses
  1. The baseline methods that are included in this paper are not sufficient to test the paper's central claim. One key argument that the authors had was that the counterfactual validation was driving the cost instead of having the memory. However, all headline baselines in table 1 are methods that do not have memory structures, such as chain of thought and DeepEyes. This hides the effect of the methods in confounders. The only memory-active baseline is a passive RAG, which was unfiltered. The authors should consider showing improvements over cheap memory validation baselines such as HtT and M2Note, since these are already cheaper than the proposed method and has been proven effective.
  2. This brings to my second concern which is around the cost of such system. The sleeping stage seems to require more than a dozen LLM calls with extra editors, table 5 says it would take 152 seconds per sample. This is a huge time cost compared with a vanilla inference that takes less than a second, so bottom line, such higher time cost needs to be justified with strictly higher performance, let alone the extra wake time inference cost. And then there's also the concern if the marginal gain is worth the extra cost. Over the strong baselines, the average gain is only 0.49 ~ 0.98 on the two models in some cases, and DeepEyesV2 actually wins with some benchmarks. Therefore, the authors should consider including a compute-matched comparison baseline, along with an accuracy compute trade off curve to really justify the need for their proposed system.
  3. There are are a number of related works remains unaddressed or discussed. Aside from the HtT (Hypotheses-to-Theories, arXiv:2310.07064) and M2Note (arXiv:2607.00685) work that was mentioned above, the idea of having counterfactual image editing for probing visual models has been established, such as LANCE (Neurips 2023). In more adjacent realms, falsification-based hypothesis validation also exists, provided by POPPER (ICML 2025). EDV (arXiv:2606.24428) is also another memory management system that works in the spirit with the proposed work and deserves a discussion.
Questions for authors
  1. Can you incorporate a basic and cheap memory-verification method baseline as comparison in the headline metrics?
  2. Can you add a compute-matched baseline for a fair comparison with the proposed method under the same resources? A accuracy-vs-latency trade off curve or some other mechanisms demonstrating the necessity for the extra time cost would strongly enhance the paper.
  3. Can you add discussion of the related works mentioned above and how they differ from the proposed method?
6 comments
AuthorsOpenReview ↗

Response to Reviewer rBp2 (1/4)

Dear Reviewer rBp2,
Thank you for the constructive review. All three requests were concrete, and we acted on each one:

  • Q1 (baselines). HtT and M$^2$Note are now run on all seven benchmarks with both backbones (28 new runs). CausalMem is ahead of both in all 14 benchmark–backbone cells.
  • Q2 (cost). We measured the cost of both baselines and built the accuracy–latency trade-off. With a lightweight editor, the full CausalMem pipeline takes less wall-clock time than HtT and is 5.76 points more accurate.
  • Q3 (related work). HtT, M$^2$Note, EDV, LANCE and POPPER are now discussed and contrasted with our method.

W1 / Q1.\textcolor{green}{\text{W1 / Q1.}} Baselines do not test the central claim; add cheap memory-validation baselines such as HtT and M$^2$Note

We agree. Table 1 compared CausalMem with methods that change the reasoning procedure, and the only memory baseline was unfiltered RAG. The claim that the gain comes from counterfactual validation needs a comparison with memories that are validated without interventions. HtT and M$^2$Note are exactly that, so we ran both.

Protocol. Both baselines follow the protocol of Table 1: the same frozen VLM, an empty memory at the start, a single pass over each benchmark stream, no access to labels, and no weight updates.

  • HtT. We use the upstream RuleLibrary implementation. After every 10 questions (the cadence of our sleep phase), the VLM induces up to 3 reusable rules per seen question. The original HtT is text-only and verifies rules against ground-truth training answers. To stay label-free, we compute rule confidence against a pseudo-reference: the strict majority of 3 independent memory-free answers. Rules with occurrence ≥ 2 and confidence ≥ 0.7 enter the library, which is capped at 500 rules / 12,000 tokens (the same budget as our B).
  • M$^2$Note. The official repository could not be accessed during the rebuttal period, so we re-implemented the method from Algorithm 1 and Appendix A of its paper with the default settings: batches of 16, top-1 multimodal retrieval with threshold 0.4 (Qwen3-VL-Embedding), label-free self-judging, and batch-level post-verification with rollback. Each question is scored on its first answer; the in-batch re-answers only verify the notebook and never replace a scored answer.

Table A. Memory-validation baselines under the protocol of Table 1 (official scores, %). Δ Avg. is relative to vanilla and computed from unrounded averages.

BackboneMethodV*MMStarMMBenchRealWorldHallusionMM-VetMMMUAvg.Δ Avg.
Qwen3-VL-8BVanilla78.0162.5384.2667.7171.6158.2660.4468.97–
HtT79.0660.8079.7667.4569.6159.6358.8967.89−1.09
M$^2$Note80.1062.6084.0168.1072.6658.2661.6769.63+0.65
CausalMem86.3963.8088.3968.3773.2962.3963.2272.26+3.29
Qwen2.5-VL-7BVanilla73.8262.1381.1164.4466.1460.5555.1166.19–
HtT74.3557.5370.6463.2766.9861.4755.1164.19−1.99
M$^2$Note74.8761.9380.2664.3168.2461.9354.7866.62+0.43
CausalMem82.7263.8785.1067.8471.8263.7657.4470.36+4.18

What the table shows.

  1. CausalMem is ahead of both baselines in all 14 cells. The average margin is +2.64 / +3.75 points over M$^2$Note and +4.38 / +6.17 over HtT (Qwen3-VL-8B / Qwen2.5-VL-7B).
  2. Validation without interventions gives small or negative gains. M$^2$Note improves the average by +0.65 / +0.43 and is below vanilla in 5 of 14 cells. HtT lowers the average by 1.09 / 1.99 and is below vanilla in 8 of 14 cells. CausalMem is above vanilla in all 14.
  3. The failure pattern of HtT matches the shortcut-amplification argument. Its largest drops are on MMBench (−4.50 and −10.47), which is also where its occurrence/confidence filter admits the most rules: 4.2k–5.4k rule tokens are injected per question there, against at most 1.1k on every other benchmark. A filter based on how often a rule co-occurs with self-consistent answers cannot tell a shortcut from the evidence, and this is the gap the interventional gate is designed to close.

Both baselines are added to Table 1 of the Rebuttal Revision PDF. Together with the existing ablations, they support the central claim directly: what matters is not having a memory, or having a verified memory, but verifying it by intervention.

AuthorsOpenReview ↗

Response to Reviewer rBp2 (2/4)

W2 / Q2.\textcolor{green}{\text{W2 / Q2.}} The sleep stage costs 152 s per sample; add a compute-matched baseline and an accuracy–compute trade-off

(a) Where the 152 s goes. On V* with Qwen3-VL-8B (Table 5), sleep takes 151.84 s per sample: 130.87 s for the image editor and 20.97 s for the VLM. The editor (Qwen-Image-Edit, 1.98 calls per sample) therefore accounts for 86% of the sleep time. It is a replaceable component. With InstructPix2Pix (Table 2), the whole sleep phase, VLM and editor together, takes 6.39 s per sample, and accuracy stays at 84.82 on V* and 61.47 on MM-Vet (86.39 and 62.39 with Qwen-Image-Edit). Wake remains a single VLM call (1.32 s against 0.71 s for vanilla), with a bounded context of 2,234 extra tokens that does not grow with the stream (Fig. 4b).

(b) Measured cost of the baselines, and a compute-matched comparison. We timed HtT and M$^2$Note in the same environment (V*, Qwen3-VL-8B). Time is model-call time per question for the entire pipeline, i.e., answering plus memory update.

Table B. Accuracy and cost on V* (Qwen3-VL-8B).

MethodAcc. (%)Δ vs. vanillaUser-facing latency / questionTotal time / questionExtra prompt tokens / question
Vanilla78.01–0.71 s0.71 s0
M$^2$Note80.10+2.093.68 s†3.68 s14.0k (+2.0k embedding tokens)
HtT79.06+1.058.46 s†8.46 s14.3k
CausalMem, InstructPix2Pix84.82+6.811.32 s7.71 s (1.32 wake + 6.39 sleep)16.8k (2.2k wake + 14.6k sleep)
CausalMem, Qwen-Image-Edit86.39+8.381.32 s153.16 s (1.32 wake + 151.84 sleep)65.7k (2.2k wake + 63.5k sleep)

†HtT and M$^2$Note ran their update steps inline with answering in our measurement, so their per-question cost appears in both time columns; CausalMem's sleep is asynchronous and batched offline.

  • The user-facing critical path. Sleep is asynchronous and batched offline. The quantity on the user-facing critical path is wake latency, which is 1.32 s: lower than M$^2$Note (3.68 s) and HtT (8.46 s) under the same measurement (V*, Qwen3-VL-8B)†. Wake is one VLM call with a bounded context (2,234 extra tokens) that does not grow with the stream.
  • Matched wall-clock, total pipeline. CausalMem with InstructPix2Pix needs 7.71 s per question in total, less than HtT (8.46 s), and is 5.76 points more accurate (84.82 vs. 79.06). Against M$^2$Note it uses 2.1× the total time for +4.72 points. Measured as accuracy gained per extra second over vanilla, CausalMem with InstructPix2Pix is the most efficient of the three memory methods: 0.97 points/s, against 0.70 for M$^2$Note and 0.14 for HtT.
  • The frontier. Vanilla, M$^2$Note, CausalMem (InstructPix2Pix) and CausalMem (Qwen-Image-Edit) form the accuracy–latency frontier; HtT is dominated. Moving from InstructPix2Pix to Qwen-Image-Edit buys a further +1.57 points for about 20× the time, so the editor is a quality knob with diminishing returns, and the 152 s configuration is the high-accuracy end of the curve, not a requirement of the method.
  • The token budget is comparable too. With InstructPix2Pix, CausalMem spends 16.8k extra prompt tokens per question, between HtT (14.3k) and M$^2$Note (14.0k + 2.0k embedding tokens); Qwen-Image-Edit alone accounts for 65.7k. The tokens that reach the frozen VLM at wake time, i.e., the part the agent pays per query, are 2.2k in both configurations.
  • The cheap baselines are not free. Averaged over the seven benchmarks, HtT takes 11.94 / 11.10 s per question and M$^2$Note 6.84 / 4.01 s (Qwen3-VL-8B / Qwen2.5-VL-7B), against 1.39 / 0.97 s for vanilla.

Trade-off curves. The two plots are added to the experiments section of the Rebuttal Revision PDF and our anonymous repository: total pipeline\textcolor{blue}{\text{total pipeline}} and user-facing latency\textcolor{blue}{\text{user-facing latency}} (click to the anonymous repository). Each shows vanilla, HtT, M$^2$Note and CausalMem with the two editors; the halo area of a marker is proportional to its extra prompt tokens per question. In the second plot CausalMem sits at its wake latency, 1.32 s — the quantity on the user-facing critical path.

AuthorsOpenReview ↗

Response to Reviewer rBp2 (3/4)

(c) Sleep is paid once per consolidated episode, not once per query. Three results in Table 2 show that a consolidated memory keeps its value without further sleep:

  • Inductive split. Memory is built on one half of the data, frozen, and the other half is answered with wake only. The gain is +7.33 on V* and +5.57 on HallusionBench, i.e., 82% and 98% of the transductive gain (+8.90 and +5.68), with no sleep cost on any evaluated query.
  • Shared pool. One memory built from 300 unlabeled samples raises pooled accuracy over the full V*, MMStar and MMBench sets from 78.69 to 80.90.
  • Transfer. A memory built on MMStar and frozen improves V* by +5.24.

For an agent that answers a long stream, the consolidation cost is thus amortized over all later queries, each of which costs one VLM call.

(d) The extra compute is necessary. The cheaper variants of our own pipeline trace the same trend (Fig. 3a; average of V* and MM-Vet, Qwen3-VL-8B): memory without validation 58.74 (9.40 below vanilla), one edit family removed 70.62 or 71.87, both edit families 74.39. Table A adds the external evidence: spending extra inference on non-interventional verification (HtT, M$^2$Note) yields −1.99 to +0.65 on average, against +3.29 / +4.18 for CausalMem.

(e) On the +0.49 / +0.98 margins over the strongest baseline. Sketchpad, Thyme and DeepEyesV2 are trained methods: the pipelines we compare against rely on substantial additional training (SFT and RL for Thyme and DeepEyesV2, a trained sketching model for Sketchpad), and they change how the model reasons on every query through multi-turn tool use. CausalMem is training-free: the VLM and the editor stay frozen, no gradient is computed on any of the seven benchmarks, and only the external memory changes. It leaves the reasoning procedure untouched and changes only what is written to memory. The marginal gains of the memory baselines are +2.64 to +6.17 points on average.

W3 / Q3.\textcolor{green}{\text{W3 / Q3.}} Discuss HtT, M$^2$Note, LANCE, POPPER and EDV

We thank the reviewer for these pointers. We add the paragraph summarized below to Sec. 2 of the Rebuttal Revision PDF. M$^2$Note and EDV are arXiv preprints that appeared shortly before our submission (July and June 2026), and we are glad to position our work against them.

WorkWhat is validatedEvidence used for validationLabels / feedback requiredGenerates new interventional evidence?What the result is used for
HtT (arXiv:2310.07064)Textual rules for text-only reasoningOccurrence count and accuracy of the training examples in which a rule appearsGround-truth answersNoRule library in the prompt
M$^2$Note (arXiv:2607.00685)Subject–guidance notes written from failuresRe-run of the same batch; commit if batch accuracy does not dropLabels or an LLM judgeNoNotebook retrieved by multimodal RAG
EDV (arXiv:2606.24428)Lessons distilled from multi-agent trajectoriesUnanimous review of the existing trajectories by the executor modelsEnvironment feedback; three heterogeneous modelsNoFrozen offline memory bank
LANCE (NeurIPS 2023)A trained visual classifierAccuracy drop under label-preserving, language-guided image editsClass labelsYes, invariance direction onlyA stress-test set and a bias report; nothing is written back
POPPER (ICML 2025)Free-form scientific hypothesesSequential falsification experiments aggregated with e-valuesDomain datasetsStatistical tests on dataAn accept/reject verdict with Type-I error control
CausalMemA visual memory rule (evidence, nuisance, scope)Paired nuisance edits and evidence edits of the stored imageNoneYes, invariance and necessityOnline long-term memory and hard negatives

Comments on the paper

2 comments
AuthorsOpenReview ↗

We have updated the revised PDF and responded to every reviewer

Dear Reviewers,

We deeply appreciate the time and effort you have dedicated to providing detailed and insightful feedback on our work. Your comments have been invaluable in improving the quality of our paper. Detailed responses to each point are provided under the corresponding reviewer’s comments.

AuthorsOpenReview ↗

Summary of Responses

Dear Area Chair,

We sincerely thank you for handling our submission and the reviewers for their constructive feedback. We have responded to all reviewers, conducting additional experiments and revising the manuscript to address their concerns.

All three reviewers rate CausalMem as 3. Following the rebuttal, rBp2 updated their assessment, and xRMP explicitly confirmed that their main concerns were addressed. Below is a summary of our responses and discussion.

  1. Is the gain due to counterfactual validation, rather than to memory, abstraction, or extra inference? (rBp2 W1, xRMP W2; the main empirical concern.) We added two memory baselines that abstract and verify rules without interventions, HtT and M$^2$Note, on all 7 benchmarks and both backbones (28 runs, same label-free protocol; HtT adapted to run without labels, M$^2$Note re-implemented from its paper). CausalMem is ahead in all 14 cells. Average change over vanilla: HtT −1.09 / −1.99, M$^2$Note +0.65 / +0.43, CausalMem +3.29 / +4.18 (Qwen3-VL-8B / Qwen2.5-VL-7B). An internal control that writes every proposed rule without the gate is 3.54 points below vanilla, where the full system is 6.25 above.

  2. Cost of the sleep phase (rBp2 W2, qBJ9 W2). On V* with Qwen3-VL-8B, the image editor accounts for 86% of the 152 s and is replaceable. With InstructPix2Pix (84.82, against 86.39 with the default editor) the whole pipeline takes 7.71 s per question: less than HtT (8.46 s) while 5.76 points more accurate, and 2.1× M2Note's time for 4.72 more points. Answering stays a single VLM call (1.32 s vs. 0.71 s for vanilla), and consolidation is amortized: a memory frozen before evaluation keeps 82–98% of the gain (point 3).

  3. Dependence on the evaluation protocol (qBJ9 W1, Q3). On V* / HallusionBench with Qwen2.5-VL-7B, gains hold for all five random stream orders (worst case +6.28 / +5.36) and with memory frozen and evaluated on a disjoint split (+7.33 / +5.57, against +8.90 / +5.68 online). Sleep intervals from 5 to 50 tasks all stay well above vanilla (83.25–86.39 vs. 78.01 on V*).

  4. Soundness of the mechanism (xRMP W1, W3, W4). Our response explains that the necessity test measures whether the model departs from its original answer when the claimed evidence is edited; the answer on the edited image is never stored or used as a label. For pruning, a label-based audit (analysis only) shows that 72% of the 39 replay flips caused by candidate rules damage a correct answer, which supports penalizing flips; removing the term costs 3.83 points. Conflict handling makes 16 of 87 (V*) and 57 of 559 (MMStar) promoted rules dormant rather than deleting them.

  5. Margin over the strongest baseline (rBp2 W2). The average margin over DeepEyesV2 is +0.98 / +0.49, and DeepEyesV2 remains ahead in 3 of 14 cells. It is a trained tool-use method; CausalMem is training-free, and the paper presents the two as complementary.

  6. Related work and memory statistics (rBp2 W3; qBJ9 Q1, Q2). HtT, M$^2$Note, EDV, LANCE and POPPER are now discussed. The gate accepts 45.5% / 37.3% of candidate rules (V* / MMStar), and only 2 of 87 / 13 of 559 promoted rules are later falsified.

Revised PDF (changes in blue). Table 1 (HtT and M2Note rows); Table 3 and Fig. 5 (cost and accuracy–latency trade-off, now in the main text); Sec. 2 and App. I (related work); App. C.1–C.2 (replay gain, conflicts); App. H (baselines, no-gate control); App. J (protocol, sleep interval, budget, churn); App. K (cost accounting).


Strengths noted by the reviewers

  • Problem. The failure mode we target, accumulated memory reinforcing visual shortcuts, is "real and rarely addressed" (qBJ9), "a relevant problem" (xRMP), and "well motivated by the authors with clear examples" (rBp2).
  • Idea. Treating what to remember as interventional falsification is "a genuinely novel and well-articulated framing" (qBJ9); testing memories before reuse is "a concrete approach to selective memory writing without updating model parameters" (xRMP).
  • Results. The gains are "very consistent" across seven benchmarks and two backbones (rBp2).
  • Evaluation. "Evaluation discipline is strong", including "the striking ablation that passive RAG hurts while validated memory helps" (qBJ9); the stream-order, inductive, transfer and cross-family controls "provide meaningful evidence of the complete system's usefulness" (xRMP).

We thank the Area Chair and the reviewers for your time.


Best regards,

Authors of Submission 42258

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.