acceptodds
Under review as a conference paper at ICLR 2027

CAUSEWAY: Auditable Causal Memory for Long-Horizon LLM Agents

Abstract

Agent memory retrieves text that resembles the query, but an agent needs evidence that supports and influences the answer. Similarity and graph proximity can surface evidence that is stale, redundant, or confounded by a shared upstream cause, while retrieval alone does not establish whether an answer depends on that evidence. Destructive consolidation compounds this problem by deleting the history that temporal questions depend on. We present Causeway, a memory substrate that connects causal structure, evidence influence, and source auditing. Causeway compiles interaction history into a bitemporal causal graph whose typed edges become load-bearing after passing precedence, mechanism, and evidence gates. On its interventional path, each query is compiled into an explicit answer variable, and candidates are scored by contrasting their presence and absence in reader-visible evidence, with graph-selected background and a negative-control penalty. On the graph path, a source-linked proof supports answer generation and applicable removal/polarity checks; failed support checks trigger source replay or abstention. Non-destructive writes preserve earlier states for historical queries. Instantiated training-free with MiMo v2.5, Causeway achieves the highest Overall scores among evaluated methods on LoCoMo, LoCoMo-Plus, and MemoryAgentBench. Its strongest gains concern evidence localization, evolving states, and persistent constraints. Matched LoCoMo comparisons show that verification improves Answerable F1 by 6.30 points while reducing coverage by 1.29 points. On LoCoMo-Plus, targeted evidence use improves accuracy over Full Context with 76.99% fewer query-stage LLM tokens. Our code is available at [https://anonymous.4open.science/r/Causeway-3F66/](https://anonymous.4open.science/r/Causeway-3F66/)

Primary area:
causal reasoning
Keywords:
Agent Memory, Causal Memory, Interventional Retrieval, Counterfactual Verification, Bitemporal Versioning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_vGDLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CAUSEWAY, a training-free memory framework for long-horizon LLM agents that combines a source-linked bitemporal causal graph, evidence-intervention scoring, answer-dependence auditing, and non-destructive versioning. Its graph path selects evidence using estimated answer sensitivity, graph relations, and temporal validity, and checks generated answers through applicable node-removal and edge-polarity tests, with source replay or abstention when support is insufficient. Task-specific routes use hierarchical session selection for LoCoMo-Plus and native-task readers for MemoryAgentBench. Instantiated with MiMo v2.5, CAUSEWAY achieves the highest Overall scores among the evaluated methods on the fixed evaluation sets: 55.77 on LoCoMo, 87.78 on LoCoMo-Plus, and 70.11 on MemoryAgentBench. A matched LoCoMo ablation reports a 6.30-point improvement in Answerable F1 from verification, with a 1.29-point decrease in answerable coverage. On LoCoMo-Plus, it outperforms Full Context while using 76.99% fewer query-stage LLM tokens, although memory construction costs are excluded.

Critical strengths
  1. By integrating source-grounded memory, non-destructive temporal versioning, and an evidence verification mechanism, CAUSEWAY enables the handling of factual updates and historical queries, thereby delivering clear practical value for long-horizon agents.
  2. The framework achieved the highest overall scores across all three benchmark tests. Furthermore, while the introduction of the verification mechanism resulted in a slight decrease in coverage, it improved the "Answerable F1" score. These results strongly substantiate the framework's practical effectiveness and serve as the basis for its high evaluation.
Critical weaknesses

1, Section 3.9 states that CAUSEWAY uses task-specific reading routes across benchmarks, including hierarchical session selection for LoCoMo-Plus and native-task readers for MAB, rather than uniformly executing CE and deletion. However, the main text does not sufficiently explain the functions of these task adapters or clarify whether the baselines receive the same task information and comparable adaptation. I therefore find it difficult to determine how much of the observed improvement comes from the core memory mechanisms versus additional retrieval, routing, or reading strategies. Task adaptation is reasonable, but differences in available information or processing capabilities could affect both the interpretation and fairness of the comparisons. I suggest clarifying the task adapters and input conditions for each method in the main text and providing controlled comparisons that distinguish the contributions of the core mechanisms from those of the task-specific routes.

2, It remains difficult to distinguish the full system’s performance gains from the contributions of its core memory mechanisms. Appendix A.1 states that LoCoMo-Plus uses hierarchical session selection rather than the unified CE scorer/auditor, while MAB routes other than conflict resolution have no recorded CE/deletion audit. Therefore, the overall gains on these benchmarks cannot directly establish the effectiveness of the core intervention and auditing mechanisms. Moreover, public question categories guide CAUSEWAY’s LoCoMo fallback, whereas the category argument is unset in MAGMA’s LoCoMo run; the appendix also explicitly acknowledges that identical metadata-use policies have not been demonstrated across baselines. The Plus route-replacement comparison simultaneously changes candidate granularity, routing guidance, and evidence-token lengths, so it cannot isolate the contribution of the core mechanisms. It remains unclear to me how much of the reported advantage comes from the proposed memory mechanisms versus task-specific retrieval, reading strategies, or additional category guidance.

Questions for authors

What are the storage footprint and peak RAM usage of CAUSEWAY’s source log, graph, historical versions, and caches, and how do they scale with increasing interaction history? Since CAUSEWAY targets long-horizon agents and retains superseded records through non-destructive versioning, could the authors quantify these costs in addition to query-stage LLM usage?

4 comments
AuthorsOpenReview ↗

Response to Reviewer vGDL (1/3)

Dear Reviewer vGDL,

Thank you for the review and for pressing on fairness and attribution. We ran matched-metadata, matched-granularity and matched-token controls, and profiled storage. All changes are in the revised PDF (in blue); numbers below refer to it.

W1. What do the task adapters do, and do baselines receive the same information?

The adapters play two roles only. They set the candidate unit to match the benchmark's context structure, and they format the final answer for the native scorer (short spans for MAB AR/CR, class labels for TTL, conversational responses for LoCoMo-Plus). All methods receive the same question texts and output-format specifications. The revised Sec. 3.4 adds Table 1, which states for each benchmark which Causeway operations are active and which public metadata Causeway uses:

BenchmarkCandidate unitCEDeletion auditSource replayTerminal checkPublic metadata used by Causeway
LoCoMoevents + induced edges✓✓ (103/300)✓ (229/300)✓question category (fallback eligibility)
LoCoMo-Plussession synopses (opaque IDs)––––relation/type cues; cue flags hidden
MAB CRcurrent-slot facts + entity chains–✓ (100/100)––task name (routing)
MAB other tasksnative-task evidence––––task name (routing)

The table makes your point explicit: the full CE + audit path runs on LoCoMo, and only part of it runs on the other benchmarks. The one input asymmetry across methods is public question categories on LoCoMo, which we test directly below. Sec. 4.5 then holds candidate granularity, token budget, reader and metadata fixed to separate the core mechanisms from the routes.

W2 (a). Question-category metadata on LoCoMo.

We reran Causeway, the official MAGMA implementation and LightMem on all 600 LoCoMo questions with the same reader (GLM-5.3-flash). In one arm, categories were hidden from every method; in the other, the same categories were given to every method (App. A.12.5, Table 32):

LoCoMo Overall (600)Categories hiddenCategories providedMain table (MiMo v2.5)
LightMem39.0840.0447.73
MAGMA (official)49.8955.4548.69
Causeway50.5453.9555.77

We report this plainly. With categories hidden from both, Causeway leads MAGMA by 0.65. With categories given to both, MAGMA leads by 1.50, and both systems benefit. Part of the 7.08-point main-table gap on LoCoMo therefore reflects the asymmetric use of categories. The revised Sec. 4.2 states that the two systems are within 1.5 points on LoCoMo under matched metadata. The margins over MAGMA on LoCoMo-Plus (+70.07) and MAB (+33.38) are not affected by this metadata question.

AuthorsOpenReview ↗

Response to Reviewer vGDL (2/3)

W2 (b). LoCoMo-Plus: separating the core mechanisms from granularity, token length and routing.

You noted that the submitted Plus route-replacement changes candidate granularity, routing guidance and evidence length at once. We now vary one factor at a time on the 200-item panel, with the same reader, top-1 selection and 2,048-token cap (Table 4b, App. A.12.4):

ControlMatched conditionBaselineCauseway component
Granularityidentical event-level units34.0 (cosine)92.0 (selector)
Token length, raw textequal tokens per candidate35.0 (dense)89.0 (constraint-guided selector)
Token length, event quotesequal tokens per candidate43.0 (cosine)93.0 (selector + event quotes)
Online CEsame dense top-3 session pool43.0 (no CE)61.5 (support-state CE)
Online CE + auditsame 3-session hierarchical pool93.0 (selector top-1)87.5 (CE); 65.0 with deletion audit

With granularity and per-candidate tokens equalized, the advantage persists, so it is not an artifact of unit size or evidence length. It comes from Causeway's constraint-guided selector and its structured event-quote representation (IDs, dates and source quotes). A session-level online CE improves a weak dense pool by 18.50 points (CI [11.68, 25.41]) and, with deletion audit, cuts wrong answers from 114 to 61. It does not improve on the hierarchical selector, however, which is why the default Plus route omits it. We now state that Plus results support the ledger and selector, not interventional scoring (Sec. 4.5).

W2 (c). MAB.

All methods use the same task-specific output prompts on MAB. As Table 1 shows, CE and deletion audit are active only on the CR route; the other MAB routes are native-task readers over the ledger. The CR evidence isolates ledger mechanisms under a fixed reader and budget (Table 16, 50 items):

  • Removing bitemporal versioning lowers CR accuracy from 84.00 to 34.00.
  • Replacing causal entity-chain traversal with BM25 lowers FC-MH from 76.00 to 16.00.

For AR, TTL and LRU, we present the gains as system-level results of the ledger plus native readers, not as evidence for CE.

AuthorsOpenReview ↗

Response to Reviewer vGDL (3/3)

Q1. Storage footprint and peak RAM as history grows.

We ran 45 standalone profiling runs: the ten real LoCoMo histories plus five synthetic scaling regimes, three independent runs per condition. Each run measures disk footprint before and after 12 incremental update batches and peak resident set size (RSS) during ledger load and update (App. A.11.2, Table 27):

ConditionActive historyInitial disk (MiB)Disk after 12 updates (MiB)Peak RSS (MiB)
Ten real LoCoMo historiesall ten ledgers in one SQLite DB69.87– (static)profiled per history
Token scaling1,276,000 current tokens, 1 version67.42134.85387.65
Version depth127,000 current tokens, 20 versions68.23136.46451.62

Each ledger is stored as a compact serialized JSON payload in SQLite rather than duplicating a dense-vector index per snapshot. Storage therefore grows approximately linearly across the tested token-count and version-depth regimes. At 1.28M tokens, or at 20 retained versions per record, disk stays below 137 MiB and peak RSS below 452 MiB. Non-destructive versioning does raise retention and deletion questions; the Ethics Statement notes that deployments need consent, access control, retention and deletion mechanisms.


Thank you again. The revised paper now states which mechanisms are active on each benchmark, qualifies the LoCoMo comparison with MAGMA under matched metadata, and isolates the core mechanisms from routing on Plus. If these changes address your concerns, we would appreciate it if you would consider raising your score.

Best regards,

Authors

Thank you for the detailed reply and additional experiments. I recognize the positive verification results, but overall, I am still not convinced that the evidence is strong enough to change my recommendation. I will therefore keep my original score.

Reviewer_VFVHOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes CAUSEWAY, a memory system for long-horizon LLM agents. It stores interaction history in a graph with causal claims, temporal information, and links to the original sources. The main idea is to select memories based on how they affect the answer, rather than relying on textual similarity alone. On its interventional retrieval path, the system estimates answer sensitivity to the presence and absence of candidate evidence. It also checks whether an answer is supported, revisits the original sources when needed, and preserves older memory versions for historical questions. The paper evaluates the system on LoCoMo, LoCoMo-Plus, and MemoryAgentBench, using task-specific retrieval and verification routes.

Critical strengths

I find the problem important and well motivated. A memory can look relevant to a query while being outdated or offering little support for the answer. Looking at how evidence affects the answer is a useful direction for agent memory. I also like the use of source links and historical versions. These make it possible to revisit the evidence behind an answer and recover earlier states when the question refers to the past. The graph gives these operations a clear structure.

The experiments cover several memory capabilities and go beyond reporting overall scores. I particularly appreciate the analyses of component effects, answer coverage, and query costs. The discussion of cases where the method performs less well also helps explain what the system can and cannot do.

Critical weaknesses

My main concern is that the new contribution is still somewhat difficult to identify. The paper discusses its relationship to CMI, ActMem, and temporal memory systems, but I am less clear about which mechanism provides the main advance over these approaches. The experiments answer this only partly. In the controlled ranking study, the evidence-sensitivity term helps compared with removing it, but its advantage over the cosine replacement is uncertain. Removing the negative-control term does not change the final answers, and the benefit of graph-selected background is not separately tested. I would like a clearer connection between the claimed new mechanisms and the gains shown in the experiments.

I found parts of the presentation difficult to follow. The abstract introduces many concepts before giving the reader a simple picture of how the system works. The methodology is also spread across nine subsections, making it harder to distinguish the central ideas from supporting implementation details. Grouping related steps under a few broader stages would help.

There is also a minor formatting issue in Figure 2(a): “Support” appears on a white patch that does not match the background of the surrounding table row.

Questions for authors

Which part of CAUSEWAY do the authors consider its main methodological contribution? In particular, what does it add beyond intervention-based selection in CMI and causal graph reasoning in ActMem?

What is the benefit of selecting background evidence through the graph? Could this be tested while keeping the candidate pool, reader, and evidence budget fixed?

Appendix A.1 explains that LoCoMo-Plus uses hierarchical session selection rather than the unified CE scorer and auditor. Which parts of the proposed method are responsible for the gains on this benchmark, and how much can these results tell us about the causal retrieval mechanism?

The CE score uses answer probabilities reported by a model that sees both evidence conditions in one prompt. Have the authors checked whether these scores agree with answer changes observed when the reader is run separately on the two conditions? This would help assess how well the score measures actual evidence sensitivity.

4 comments
AuthorsOpenReview ↗

Response to Reviewer VFVH (1/3)

Dear Reviewer VFVH,

Thank you for the detailed review and for the concrete experimental suggestions, several of which we ran as proposed. The revised PDF marks all changes in blue; section and table numbers below refer to it.

Q1 / W1. What is the main methodological contribution beyond CMI and ActMem, and which experiments support it?

The main contribution is the closed loop between three parts. Causeway scores evidence interventions only over time-valid versions of a source-linked ledger. It then checks whether the generated answer depends on its proof, and a failed check routes back to the raw source log (replay) rather than ending the query. Concretely:

  • CMI intervenes on snippets of a flat, unversioned bank, one snippet at a time. Causeway adds bitemporal validity filtering, so a superseded fact is not selected merely because it still shifts the reader's answer, and it conditions each intervention on the candidate's graph neighborhood.
  • ActMem reasons counterfactually over a causal-semantic graph. Causeway adds quantified answer-sensitivity scoring, non-destructive valid/transaction-time versioning, and post-generation removal/polarity auditing with source replay.

The table below links each mechanism to the experiment that isolates it:

MechanismControlled evidence (fixed pool, reader and budget where applicable)Where
Bitemporal validity / versioningWithout it: LoCoMo Overall −9.86; MAB CR 84 → 34Table 3, Table 16
Evidence-sensitivity ranking (CE)Top-1: +4.60 Ans. F1 over cosine on 300 held-out questions (CI [1.01, 9.29]); +5.37 on held-out Multi-hopTable 4a, App. A.12.1
Subgraph-conditioned interventions+1.20 overall, +1.75 on directed-path questions vs. isolated CMI-style CE (CIs include 0)App. A.12.2, Table 30
Dependence audit + source replayVerification: +10.56 Overall, +6.30 Ans. F1 at −1.29 coverageTable 17, Sec. 4.6
Causal chain retrievalBM25 route: MAB FC-MH 76 → 16; LoCoMo Multi-hop 43.70 → 37.93Table 16, Table 3

On CE versus cosine: the submitted ranking study selects top-3 from pools of only 5.98 candidates per question (1,795 for 300 questions), so every ranker keeps about half of each pool. That is why the cosine replacement looked close. With the budget varied (Table 4a), CE leads cosine by 5.52 points at top-1 on the ablation questions and by 4.60 on the held-out questions (bootstrap p=0.030p = 0.030). CE at top-2 (32.10) already exceeds cosine at top-3 (30.96).

On the negative-control term: we agree it shows no measurable benefit. Upstream three-gate extraction already filters most ungrounded noise in these conversational benchmarks. Sec. 3.2 now presents this term and explicit common-parent background as lightweight safeguards, not as sources of the reported gains.

AuthorsOpenReview ↗

Response to Reviewer VFVH (2/3)

Q2. Benefit of graph-selected background with pool, reader and budget fixed.

We ran this as suggested (top-3, 2,048-token cap; App. A.12.2, Table 30). First, a structure audit of all 600 LoCoMo questions (3,574 candidates): explicit shared parents are nonempty on only 19 questions (16 answerable) under the three-gate threshold. Varying the shared-parent rule (graph parent, empty, semantic, shuffled) therefore leaves the prompts of 581 questions identical, which is why those variants score almost the same. By contrast, 414 questions (69.0%) contain two consecutive directed edges in the pool, so the relevant graph context is the candidate's induced subgraph rather than an explicit common parent. We tested that directly:

Answerable F1, top-3Ablation 300Held-out 300Directed-path subset, Δ vs. isolatedHeld-out Multi-hop, Δ vs. w/o CE
Subgraph-conditioned CE31.8727.69+1.75 [−1.54, 5.96]+5.37 [1.30, 11.19]
Isolated CE (CMI-style, no graph context)–26.480.00–
CE → cosine30.9625.96–+1.54
w/o CE29.6325.76–0.00

Subgraph conditioning helps (+1.20 overall, +1.75 on the 205 directed-path questions), but neither interval excludes zero, and we report it that way. Mechanistically, 551 of 1,779 isolated candidates produce identical present/absent worlds, because a single intermediate node cannot bridge a multi-hop path alone.

Q3. What drives the LoCoMo-Plus gains, and what do they say about causal retrieval?

We matched candidate granularity, per-candidate token length and the reader on the 200-item Plus panel (top-1, 2,048-token cap; Table 4b, App. A.12.4):

ControlBaselineCauseway component
Identical event-level units34.0 (cosine)92.0 (selector)
Equal tokens per candidate, raw text35.0 (dense)89.0 (constraint-guided selector)
Equal tokens per candidate, event quotes43.0 (cosine)93.0 (selector + event quotes)
Dense top-3 session pool43.0 (no CE)61.5 (online support-state CE)
3-session hierarchical pool93.0 (selector top-1)87.5 (online CE); 65.0 with deletion audit

The answer is clear: the constraint-guided selector and the structured event-quote representation produce most of the Plus gain. We also implemented a session-level CE that runs the reader separately on each candidate session. It lifts a dense pool from 43.0 to 61.5 (CI [11.68, 25.41]), but on the hierarchical pool it does not beat the selector's own choice. Plus results therefore tell us about the ledger and the selector, not about interventional scoring; Sec. 4.5 now says this explicitly.

AuthorsOpenReview ↗

Response to Reviewer VFVH (3/3)

Q4. Do joint-prompt CE scores agree with separate reader runs?

Yes, reasonably well. For 300 candidate pairs we ran the reader separately on world_present and world_absent, deterministically and with 4,000 additional sampled runs (App. A.12.3):

PredictorAUROC for independent answer changeSpearman with sampled DTVD_{\mathrm{TV}}
Joint-prompt CE0.82610.4007 [0.0804, 0.6641]
Cosine similarity0.6822–

The AUROC difference is +0.1439 (95% CI [0.1001, 0.1846]). Fewer than 200 of the 1,938 saved joint-prompt calls had probability-normalization formatting deviations; repairing them left downstream F1 essentially unchanged. The joint prompt needs one call per candidate instead of two. The correlation with sampled DTVD_{\mathrm{TV}} is moderate, so we still describe CE as a model-estimated sensitivity proxy, not as an independently measured effect (Sec. 3.2).

W2. Presentation.

Agreed. The revised abstract opens with a plain three-stage description (write a versioned causal ledger; rank evidence by how much the answer changes when it is shown or hidden; answer from a proof and check that the answer depends on it) before any results. Section 3 is reorganized from nine subsections into four stages:

  1. Sec. 3.1 Problem formulation and the bitemporal causal ledger (former 3.1, 3.2, 3.3 and 3.8)
  2. Sec. 3.2 Query compilation and graph-conditioned interventional retrieval (former 3.4 and 3.5)
  3. Sec. 3.3 Causal proof assembly and counterfactual answer auditing (former 3.6 and 3.7, plus the new terminal support check)
  4. Sec. 3.4 System realization and task adapters (former 3.9, with Table 1 listing active operations and input conditions per benchmark)

W3. Figure 2(a). Thank you for spotting this. The table cells in panel (a) used semi-transparent fills, which some PDF viewers render with a white patch behind overlapping text. We replaced them with opaque fills of the same color, so "Support" now sits on the row background in every viewer.


We hope the controlled studies make the link between mechanisms and gains clearer. If they address your concerns, we would appreciate it if you would consider raising your score.

Best regards,

Authors

Thanks for the detailed response and the new controls. The revision is clearer, and I appreciate how openly it now reports where each component helps and where it does not.

That openness is also why I'm keeping my score. The new results show little effect from the parts that set Causeway apart from CMI. Graph-conditioned CE beats isolated CE by 1.20 points, and the CI crosses zero. Common-cause context is active on only 19 of 600 questions, and the negative-control term changes no answers. CE beats cosine significantly only at top-1, and only in a stripped-down readout with verification and fallback turned off.

The headline gains seem to come from elsewhere. In Table 32, under a matched reader, the native graph path scores 38.44, the v3 reader policy brings it to 50.54, and MAGMA scores 49.89. On LoCoMo-Plus, the selector and event quotes drive the result, and adding CE and audit lowers accuracy (93.0 → 87.5 → 65.0). On MAB, CE is not used.

I find the closed-loop argument plausible, but no experiment isolates it. A CE × validity-filter comparison on knowledge-update questions would help. So would showing how often replay actually fixes an answer. One small point: the main LoCoMo numbers are from before the terminal check, but the paper now describes that check as part of the system. With it, Answerable F1 is 49.80.

Overall, this is a solid and carefully engineered system. But the causal and interventional framing claims more than the evidence supports. I'd suggest reframing the paper around what actually drives the gains, or showing directly that the full loop beats its parts.

Reviewer_sfwtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents CAUSEWAY, a long-term agent memory system combining a bitemporal causal graph, intervention-based evidence selection, and counterfactual answer auditing. The system achieves strong results on three memory benchmarks, with particularly notable improvements in temporal reasoning and conflict resolution.

Critical strengths

The problem is well motivated, and the results are promising. I also appreciate the detailed component studies and the authors' transparency about the limitations of individual mechanisms.

Critical weaknesses

My main concern is whether the proposed causal mechanisms actually account for the improvements. Replacing causal retrieval with BM25 reduces LoCoMo by only 1.24 points. The controlled ranking study shows a modest benefit from CE, but no clear advantage over cosine similarity, while removing the negative-control term has no effect on the answers. Graph-selected common-cause backgrounds are active on only 10/300 questions, and the LoCoMo-Plus results use a separate retrieval route without the proposed CE scorer. The strongest gains therefore seem more closely tied to temporal state tracking, task-specific retrieval, and verification than to graph-conditioned causal interventions.

The novelty also needs sharper positioning. CausalRAG2 (ICML 2026) already uses causal gating for graph retrieval, while RAGONITE (WSDM 2025) and ContextCite (2024) investigate answer dependence through evidence interventions. Combined with the cited work on temporal memory, these make the individual mechanisms less novel than the overall system. The paper would benefit from clarifying what new capability their integration establishes.

Finally, the auditing claim is somewhat stronger than the implementation supports. Appendix A.1 reports 38 final LoCoMo outputs modified by postprocessing, without counterfactual re-auditing of the modified answers. Thus, a successful intermediate audit does not necessarily establish support for the final answer.

Questions for authors

Can the authors clarify which gains are attrubutable specifically to graph-conditioned intervention scoring, rather than the task-specific retrieval and verification pipelines? Also, what guarantees that the final emitted answer remains supported after postprocessing?

3 comments
AuthorsOpenReview ↗

Response to Reviewer sfwt (1/3)

Dear Reviewer sfwt,

Thank you for the careful review. Your question is exactly the right one: which gains come from graph-conditioned interventions, and which from temporal tracking, task routes and verification. We ran controlled studies to separate them and have rewritten the claims to match. All changes are in the revised PDF (in blue); numbers below refer to the revised version.

W1 / Q1. Do the causal mechanisms account for the improvements?

(a) CE versus cosine. The submitted ranking study (App. A.8.1) selects top-3 from pools of 5.98 candidates per question on average (1,795 candidates for 300 questions), so every ranker keeps about half of each pool and the selected sets overlap heavily. We therefore swept the budget on the same 300 questions and evaluated the other 300 questions of the 600-item main set (1,779 candidates), which are disjoint from them. Pool, reader and the 2,048-token cap are fixed; verification, replay and category fallback are off (Table 4a; App. A.12.1).

Answerable F1Ablation top-1Ablation top-2Ablation top-3Held-out top-1Held-out Multi-hop, top-3 (Δ vs. w/o CE)
Full formula (with CE)29.8132.1031.6326.03+5.37 [1.30, 11.19]
CE → cosine24.2928.8830.9621.43+1.54
w/o CE (base only)23.0229.3929.63–0.00

At top-1, CE beats cosine by 5.52 points on the ablation questions and by 4.60 on held-out questions (95% CI [1.01, 9.29], bootstrap p=0.030p = 0.030, cluster sign-flip p=0.043p = 0.043). CE at top-2 already exceeds cosine at top-3. The top-3 numbers come from a fresh run of the same protocol, so they differ slightly from the submitted Table 19.

(b) The BM25 route on LoCoMo (−1.24). 61.7% of the LoCoMo set is Single-hop (235) or Adversarial (135), where lexical matching followed by verification already works well. On dependency-heavy slices the route matters more: replacing it with BM25 lowers LoCoMo Multi-hop F1 from 43.70 to 37.93 (Table 3) and MAB FC-MH from 76.00 to 16.00 (Table 16).

(c) Graph-selected background and the negative control. You are right that these do not drive the results. Across all 600 LoCoMo questions (3,574 candidates), explicit common parents are nonempty on only 19 questions (16 answerable) under the three-gate threshold, so the other 581 receive identical prompts under every shared-parent variant. What the graph does provide is path context: 414/600 questions contain two consecutive directed edges in the pool. Conditioning CE on the candidate's induced subgraph, rather than evaluating each candidate in isolation, improves held-out Answerable F1 from 26.48 to 27.69 (+1.20, CI [−1.77, 4.67]) and by +1.75 (CI [−1.54, 5.96]) on the 205 directed-path questions. In isolation, 551 of 1,779 candidates yield identical present/absent worlds. These effects are positive but not significant, and we say so (App. A.12.2, Table 30). Sec. 3.2 now presents common-parent background and the negative-control term as lightweight safeguards for rare confounding, not as sources of the reported gains.

AuthorsOpenReview ↗

Response to Reviewer sfwt (2/3)

(d) LoCoMo-Plus. The main Plus route does not use the CE scorer, so we tested what drives it under matched conditions (200 items, fixed reader, top-1, 2,048-token cap; Table 4b, App. A.12.4):

ControlBaselineCauseway component
Identical event-level units34.0 (cosine)92.0 (selector)
Equal tokens per candidate, raw text35.0 (dense)89.0 (constraint-guided selector)
Equal tokens per candidate, event quotes43.0 (cosine)93.0 (selector + event quotes)
Dense top-3 session pool43.0 (no CE)61.5 (online support-state CE)
3-session hierarchical pool93.0 (selector top-1)87.5 (online CE); 65.0 with deletion audit

To be direct: on Plus, the constraint-guided selector and structured event quotes carry the gain. An online, session-level CE (the reader run separately on each candidate session) improves a dense pool by 18.50 points (CI [11.68, 25.41]) and, with deletion audit, cuts wrong answers from 114 to 61. On the hierarchical pool, however, it does not beat the selector's own choice. That is why the default Plus route omits CE, and why we no longer use Plus results as evidence for interventional scoring (Sec. 4.5).

Summary for Q1. Under fixed pools, readers and budgets, CE contributes +4.60 Answerable F1 on held-out questions at top-1 and +5.37 on held-out Multi-hop at top-3. Subgraph conditioning adds a smaller, non-significant +1.20. The larger headline gains come from temporal versioning (−9.86 LoCoMo Overall and CR 84 → 34 without it), verification (+10.56 Overall), and, on Plus, the selector. The revised introduction and Sec. 4.5 state this attribution explicitly.

W2. Positioning against CausalRAG2, RAGONITE and ContextCite.

Thank you for these references; they are now discussed in Section 2. ContextCite attributes a generated statement to parts of the context by ablating context sources. RAGONITE explains conversational RAG answers by counterfactual attribution over retrieved evidence. Both are post-hoc attributions over a flat set of passages for a single response. CausalRAG2 gates graph-based retrieval with causal relations across hierarchical modules of a knowledge graph; it is not designed to version superseded states or to check the generated answer afterwards.

We agree that the individual mechanisms are not new on their own. What the integration adds is closed-loop, version-consistent evidence use over an evolving history. The two parts fail without each other in long-horizon memory:

  • Interventional selection without bitemporal validity ranks a superseded fact highly, because a stale fact still shifts the reader's answer.
  • Graph retrieval without post-generation dependence checks and source replay cannot recover when proof compression drops complementary evidence.

In Causeway, interventions are scored only over time-valid ledger versions, and a failed dependence check routes back to the source log instead of ending the query. The revised Sections 1–3 state this as the main contribution.

AuthorsOpenReview ↗

Response to Reviewer sfwt (3/3)

W3 / Q2. Is the final emitted answer supported after postprocessing?

You correctly pointed out that 38 final LoCoMo answers were changed after the core audit. The revised pipeline adds a terminal support check after quote fallback and polar rendering (Sec. 3.3). It checks each factual claim against the original sources and requires exact source quotations. A failing answer gets at most one source-based repair and is re-checked; otherwise the system abstains. Nothing is rewritten after this decision, so the checked text is the emitted text.

We applied it to all 600 frozen LoCoMo outputs, including the 38 modified ones (Sec. 4.6, App. A.12.6, Table 33):

LoCoMo (600)OverallAns. F1Ans. coverageRetained factualAudited refusalsFailed-support abstentions
Main table (before terminal check)55.7753.6994.84%–––
With terminal check55.7649.8087.96%4529454

Of the 38 answers, 29 factual answers pass, five abstentions are confirmed, and four become abstentions after failed support checks. Overall is essentially unchanged and remains above MAGMA (48.69) and LightMem (47.73) in the main-table setting. The stricter check does cost answerable coverage (−6.88 points) and Answerable F1 (−3.89), but 49.80 is still above every baseline's Answerable F1 (best: LightMem, 48.25). On the guarantee itself: it is procedural. Every emitted factual answer is exactly the text that passed the final check, but the check is a model-based verifier with exact-quote matching, not a proof of correctness. The paper states it in those terms.


We hope the controlled studies give a clearer picture of where each mechanism contributes. If they address your concerns, we would be grateful if you would consider raising your score.

Best regards,

Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers and the AC

Dear Reviewers and Area Chair,

Thank you for the reviews. Three questions came up in all of them: which gains are due to interventional scoring (CE) and graph context rather than to task routing and verification; whether the audit covers the answer that is finally emitted, and whether metadata use is fair across methods; and how storage scales, together with presentation. We ran new controlled studies for each and uploaded a revised PDF. All changes are in blue, and the main text stays within the 10-page limit. Section, table and appendix numbers below refer to the revised version.

New controlled studies

StudySetupMain resultWhereReviewers
Ranking under evidence budgetsFixed pool and reader, 2,048-token cap; top-1/2/3; 300 ablation + 300 held-out LoCoMo questionsAt top-1, CE beats cosine by +5.52 (ablation) and +4.60 Ans. F1 on held-out questions (95% CI [1.01, 9.29], p = 0.030)Sec. 4.5, Table 4a; App. A.12.1sfwt, VFVH
Graph context600-question structure audit; subgraph-conditioned vs. isolated CEExplicit common parents on 19/600 questions; subgraph context +1.20 overall, +1.75 on directed-path questions (CIs include 0)App. A.12.2, Table 30sfwt, VFVH
Joint-prompt validity300 candidate pairs, reader run separately on both worldsAUROC 0.8261 vs. 0.6822 for cosine (Δ = +0.1439, CI [0.1001, 0.1846])App. A.12.3VFVH
LoCoMo-Plus controlsMatched granularity, per-candidate tokens and readerSelector + event quotes carry the gain (89.0–93.0 vs. 34.0–43.0); online CE lifts a dense pool 43.0 → 61.5Sec. 4.5, Table 4b; App. A.12.4all
Matched metadata600 LoCoMo questions, GLM-5.3-flash, categories hidden / provided to allCauseway 50.54 vs. MAGMA 49.89 (hidden); 53.95 vs. 55.45 (provided)Sec. 4.2; App. A.12.5vGDL
Terminal re-auditSupport check after all postprocessing, 600 outputsOverall 55.77 → 55.76; Ans. F1 53.69 → 49.80, still above every baselineSec. 4.6; App. A.12.6sfwt
Storage and peak RAM45 standalone runs, up to 1.28M tokens or 20 versionsDisk ≤ 137 MiB, peak RSS ≤ 452 MiBApp. A.11.2, Table 27vGDL

How the claims change

  • The main contribution is the closed loop: interventions scored over time-valid ledger versions, followed by dependence auditing whose failures route back to the source log (Secs. 1 and 3).
  • CE helps most under tight evidence budgets and on multi-hop questions. Explicit common-parent background and the negative-control term are now presented as lightweight safeguards, not as sources of the reported gains (Sec. 3.2).
  • LoCoMo-Plus gains come mainly from the constraint-guided selector and structured event quotes, not from CE (Sec. 4.5).
  • Under matched question-category metadata, Causeway and MAGMA are within 1.5 points on LoCoMo; this qualification now appears in Sec. 4.2.
  • A terminal support check now runs after postprocessing, so every emitted factual answer is the text that was checked (Secs. 3.3 and 4.6).

Presentation

The abstract now gives a plain description of the three-stage pipeline before the results. Section 3 is reorganized from nine subsections into four stages (ledger; query compilation and interventional retrieval; proof and auditing; task adapters), with a table of task adapters and input conditions (Table 1). Related work adds CausalRAG2, RAGONITE and ContextCite. The white patch behind "Support" in Figure 2(a) is fixed. To stay within 10 pages, the compact cost table moved to the appendix (Table 25).

Detailed responses are posted under each review.

Best regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.