acceptodds
Under review as a conference paper at ICLR 2027

CIT-Mem: Counterfactual Intervention Training for Evidence-Grounded Memory Policies in Long-Horizon LLM Agents

Abstract

Long-horizon LLM agents rely on memory to interpret conversational history, yet most systems optimize semantic relevance rather than whether a record remains valid, directly supports the query, and provides sufficient evidence. A correct answer may still reflect parametric knowledge or a shortcut; inference-time interventions can reveal this dependence but do not teach a policy how to use evidence. We introduce CIT-Mem (Counterfactual Intervention Training for Memory), a framework that jointly trains invariance when the role of decisive evidence is preserved, sensitivity when that role changes, and abstention when no valid and sufficient evidence remains. It constructs validated non-causal twins that preserve decisive evidence and causal twins that remove, replace, contradict, or update it. Validation checks structural consistency, evidence provenance, recoverability, and answer direction, discarding unreliable transformations before training. Aligned policy objectives directly supervise memory operations, retrieval, and answerability, while adaptive replay prioritizes unresolved constraints with the backend and reader fixed. Experiments on LoCoMo-10 and LongMemEval-S show that CIT-Mem improves intervention consistency and abstention reliability while attaining the best overall answer quality across three frozen readers. On LoCoMo-10 with Qwen3-8B, it improves all-query F1 by 14.3% and BLEU-1 by 17.2% relative to the strongest baseline while using the smallest retrieved-context budget. It also achieves the best overall judged accuracy on LongMemEval-S, demonstrating more effective evidence-grounded memory selection in long conversations. Our code can be found at https://anonymous.4open.science/r/cit-mem-2B43.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Memory-Augmented Agents, Counterfactual Learning, Causal Invariance, Twin Construction

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_1cfHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes CIT-Mem, a framework for teaching policy how to use evidence through validated counterfactual interventions. It jointly trains invariance, sensitivity, and abstention while keeping the retrieval backend and reader frozen. Experiments across multiple models and benchmarks report improved answer quality, evidence efficiency, and intervention consistency.

Critical strengths
  1. The paper addresses a meaningful distinction between answering correctly and relying on appropriate evidence. This motivation is particularly relevant when conversational memories become outdated, contradictory, or insufficient.
  2. Jointly considering invariance, directional sensitivity, and abstention provides a coherent approach to memory-policy reliability. Keeping counterfactual processing offline also makes the framework appealing for deployment.
  3. The evaluation includes multiple readers, cross-dataset transfer, matched-data ablations, and an independent human audit. Together, these provide useful evidence that the benefits extend beyond simply adding synthetic training examples.
Critical weaknesses
  1. The paper motivates CIT-Mem through faithful evidence use, but IS and CDA are not defined clearly enough to establish which behavior is being verified. It is unclear whether they evaluate the memory policy, the reader’s answer, or both. There should be detailed formulations.
  2. In Section 3.1, the recoverability gate uses an empty SK(x)S_K(x) to validate abstention targets, and Section 3.2 uses this condition to assign ABSTAIN supervision. However, Algorithm 1 only states “construct valid supports,” without explaining how evidence combinations are searched or when the search stops. Since this procedure determines the training labels, the authors should clarify how they distinguish genuinely insufficient evidence from failure to find a valid support within the candidate memories.
  3. In the related work, MemoryR1 and AgeMem provide similar approaches, while the experiments mainly compare with non-training methods.
Questions for authors
  1. How are IS and CDA computed, and do they check whether the answer follows the intended evidence change? (Refer to weakness 1)
  2. How is (S_K(x)) constructed? (Refer to weakness 2)
  3. If feasible, please compare with either Memory-R1 or AgeMem using at least one reader model. Otherwise, please explain why and provide alternative evidence supporting the claimed advantages over outcome-supervised memory policies. (Refer to weakness 3)
  4. What specific limitation of inference-time prompting does counterfactual training address, and which component is responsible for this improvement? Please discuss the trade-offs between the two approaches in flexibility and transfer to new domains, particularly whether the learned memory policy can be reused without retraining.

Questions 1- 3 correspond to one of the weaknesses above, so a single response to each pair is sufficient. Addressing some of these concerns could lead me to raise my rating.

1 comment
AuthorsOpenReview ↗

Response to Reviewers

Response to Reviewer 3

R3.1. Directional training and the definition of IS/CDA

Sensitivity is trained toward a validated target, not an arbitrary difference: G_ans^(v) checks the declared changed-answer/abstention relation, and Eq. (9) raises the validated target's score with a margin over the hardest incompatible behavior:

L_sens = −s_θ(b_I^c | x^c) + λ_mar · [δ_c − Δs_θ]₊

As implemented, IS/CDA are behavior-level pair scores. For non-causal pairs D_nc, with E_θ the selected evidence (canonical/semantic-equivalence classes), d_θ the ANSWER/ABSTAIN decision, and P_i the twin-to-base evidence map:

IS_beh = (100 / N_nc) · Σᵢ 1[E_θ(xᵢ) = P_i·E_θ(xᵢ^nc)] · 1[d_θ(xᵢ) = d_θ(xᵢ^nc)]

Both evidence role and decision must remain stable. For causal pairs D_c, with b̂_θ the realized behavior and B_i^c the validated acceptable target behaviors:

CDA_beh = (100 / N_c) · Σᵢ 1[b̂_θ(xᵢ^c) ∈ B_i^c]

The validated set admits separately validated alternatives, but changes must point to validated targets. CDA reflects policy behavior (evidence and decision jointly), not the reader's answer.

Table R3-1: IS drops 93.6→82.1 without invariance; CDA drops 75.8→56.1 without sensitivity—descriptive gaps, not significance tests.

Table R3-1. Component ablation (Table 3; three-seed means).

MethodIS ↑CDA ↑AURC ↓Epoch@95 ↓
No twins78.658.40.1847.2
w/o invariance loss82.170.90.1457.5
w/o sensitivity loss88.756.10.1797.1
w/o ACR, uniform replay90.972.10.1418.8
CIT-MEM93.675.80.1296.4

R3.2. Support construction and independent validation

Section 3.1 validates over ≤ C exposed candidates: ≤ K records are a support if the validator recovers the target and fails after matched removal of its decisive members; distinct supports stay separate in Eq. (8); the recoverability gate rejects residual sufficient support for abstention twins. Annotations anchor candidates, not sufficiency or necessity.

Table R3-2: 92/100 auto-accepted twins pass all five checks (8 errors; κ = 0.78; blinded annotators)—validity for the audited subset, not search recall.

Table R3-2. Human audit (Appendix C.1). κ = 0.78; by criterion: structure .86, evidence .79, leakage .81, sufficiency .74, relation .72.

Intervention familyAudited nIntended relationHuman-Valid (%)Label errors
Paraphrase13Same answer92.31
Distractor insertion13Same answer100.00
Equivalent duplication12Same answer91.71
Evidence deletion13Abstain if no support remains92.31
Evidence replacement13Changed answer84.62
Contradiction12Changed answer or abstain91.71
Temporal update12Latest-value answer100.00
Temporal shift12Changed answer or abstain83.32
Total10038 non-causal; 62 causal92.08

R3.3. Identifiability and outcome-supervised baselines

Eq. (2): f_φ(q, Z_θ1) = f_φ(q, Z_θ2) = y ⇏ Z_θ1 = Z_θ2 — equal answers do not identify the selected evidence; this is not an impossibility theorem for outcome-supervised methods. Memory-R1/AgeMem are discussed but not run as matched baselines—a limitation we acknowledge. Table R3-3 (32.5/43.4/49.5) assesses paired supervision and replay in our setting, not reproductions of those methods; our claim is an additional supervision mechanism for evidence dependence.

Table R3-3. Attribution (Table 10; three-seed means). The no-twins and answer-only rows report different IS/CDA; not merged; configurations will be clarified.

Training conditionObjectiveAll-query F1ISCDAAbst-P
Answer-only, no twinsSFT20.678.257.960.4
Twins w/o G_ans, plain SFTSFT26.179.458.867.5
Twins w/o G_ans, counterfactualCF38.684.962.167.8
Validated twins, plain SFTSFT32.583.064.970.1
Validated twins, uniform replayCF43.490.972.172.3
CIT-MEM (full)CF + ACR49.593.675.881.4

R3.4. Moving interventions from inference into training

Interventions are validated offline to train a reusable policy (backend/reader fixed); at deployment one controller pass selects ≤ K records and calls the reader at most once, with no construction or replay. Trade-offs: dependence on twin quality, support discovery, and validator behavior; offline costs; 6.4 vs 8.8 Epoch@95 is the epoch metric, not wall-clock. Appendix C.2: Long-MT-Bench+ F1 53.86/51.40/47.17 (GPT-4o-mini/Qwen3-8B/Llama 3.2 3B) with the LoCoMo-selected checkpoint—transfer evidence for this setting, not a no-retraining guarantee.

Reviewer_jx1SOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces CIT-Mem, which uses validated non-causal and causal twins to jointly train memory operations, evidence selection, and answerability through invariance, sensitivity, and abstention objectives. Experiments on LoCoMo-10 and LongMemEval-S report improvements across three frozen readers. The approach is well motivated, but unresolved evaluation details and limited validation of sufficiency after evidence selection weaken the support for its main empirical claims.

Critical strengths
  1. Jointly training invariance, sensitivity, and abstention addresses a concrete limitation of final-answer supervision: a correct answer does not establish appropriate use of memory. The proposed objectives translate this distinction into explicit training signals.

  2. Table 10 includes useful controls with matched training data and budgets. The reported F1 increases from 32.5 for validated plain SFT to 43.4 with paired losses and 49.5 with ACR, supporting the value of these components beyond additional supervision alone.

Critical weaknesses
  1. The evaluation protocol does not fully account for the reported accuracy. The official SSP subset contains 30 questions. If Table 2 averages one binary judgment per question across the three reported seeds, with equal weighting, the GPT-4o-mini result of 70.07% cannot be obtained, even after rounding. Cross-validation is mentioned, but its contribution to this aggregation is unspecified. Separately, the released core objective differs materially from Eqs. (7)–(9). Although the repository is described as a framework-level release, these discrepancies leave both the scoring procedure and the objective underlying the main results insufficiently documented.

  2. The extensive experiments do not directly resolve whether abstention tracks the sufficiency of the final retrieved context. A valid support set in the candidate pool does not ensure that independent Top-K selection retains a complete support. It remains unclear whether answerability labels and removal controls assess the evidence actually passed to the reader. Overall accuracy, intervention consistency, and twin validation do not settle this question, which matters precisely when retrieval retains plausible but incomplete evidence.

Questions for authors
  1. Which implementation produced the main results and Table 10? In the anonymous repository, citmem/training/objectives.py uses a single-mask hinge loss and does not include operation supervision. Please explain its relationship to Eqs. (7)–(9). Also, how does K enter the answerability computation and change coverage in Figure 4?

  2. Are answerability labels defined for the full candidate pool or the final selected evidence? Please provide results for the evidence deletion, sham edit, and rank-1 decoy comparisons specified by the repository’s abstention_shortcut_control protocol, to test whether abstention distinguishes missing necessary evidence from irrelevant changes.

  3. Please provide the per-run correct/total counts and aggregation code for the 70.07% GPT-4o-mini SSP result. Which checkpoints, repeated judge calls, filtering rules, or aggregation weights, if any, contributed to this entry, beyond the three reported seeds?

2 comments
AuthorsOpenReview ↗

Response to Reviewers

Response to Reviewer 2

R2.1. Code availability and the implementation of our objective

Our repository is a partial release: the released retrieval hinge is not the soft minimum over validated supports in Eq. (8), and the public modules lack the operation, sensitivity, moving-average, and importance-correction terms in Eqs. (7)–(9) and Appendix A. We do not present the artifact as a complete implementation of those equations.

We will release the complete implementation with reproduction instructions and identify the version underlying the reported experiments, so that the results are traceable to code.

Meanwhile, the attribution results (Table R2-1) remain the available evidence for these components: with validated twins, plain SFT reaches 32.5 all-query F1, counterfactual training with uniform replay 43.4, and the full method 49.5—differences of 10.9 and 6.1 points. These comparisons evidence the contribution; they do not substitute for the executable implementation.

Table R2-1. Contribution analysis on LoCoMo-10 (GPT-4o-mini, from Table 10); three-seed means. CF: paired invariance/sensitivity objectives; ACR: adaptive counterfactual replay.

Training conditionObjectiveF1ISCDAAbst-P
Answer-only, no twinsSFT20.678.257.960.4
Twins w/o G_ans, plain SFTSFT26.179.458.867.5
Twins w/o G_ans, counterfactualCF38.684.962.167.8
Validated twins, plain SFTSFT32.583.064.970.1
Validated twins, uniform replayCF43.490.972.172.3
CIT-MEM (full)CF + ACR49.593.675.881.4

R2.2. Candidate-pool labels, selected evidence, and abstention

The pool-level target is Y_pool^(K)(x) = 1 if S_K(x) ≠ ∅ and 0 otherwise, where C(x) is the exposed candidate pool and S_K(x) its validated sufficient supports of size ≤ K. This existence label does not certify that our realized selection Z_θ^(K)(x) contains sufficient support—if the selector misses an available support, the target can still be Answer—so the label and retrieval loss alone do not establish calibration to the selected evidence.

K caps the evidence sent to the reader; τ_abs thresholds the answerability score. Our notation does not make the score's dependence on K or the realized selection explicit: a K-independent score with a fixed threshold would force identical coverage across K. We will specify the dependency and scoring order from the inference implementation; the distinction alone does not explain the coverage in Figure 4.

Table R2-2 shows our operating-point evidence: at K = 5, raising τ_abs from 0.3 to 0.5 raises answered-only F1 from 51.14 to 53.77 while coverage falls to 92.0% and all-query F1 to 49.17—a quality–coverage trade-off within the evaluated setting, not a completion of the requested paired deletion/sham/decoy controls, which we do not report for the final controller.

Finally, abstention is not assigned merely because an annotated source was deleted: the recoverability gate in Eq. (4) rejects a proposed abstention twin when an alternative validated sufficient support remains in the exposed pool. This rule does not establish that the trained controller cannot exploit editing artifacts.

Table R2-2. LoCoMo-10 abstention accounting (Appendix D.1); CIT-MEM rows use K = 5; abstentions score zero in all-query F1. Metrics reported directly, not reconstructed from separately averaged coverage and answered-only F1 (the product of averages need not equal the average of within-group products).

Method / operating pointCoverage (%)Answered-only F1All-query F1
MemOS100.045.2745.27
TiMem100.042.0142.01
CMI-Mem100.043.5543.55
CIT-MEM, τ_abs = 0.397.551.1449.47
CIT-MEM, τ_abs = 0.592.053.7749.17

R2.3. Correction of the GPT-4o-mini SSP entryWe mistakenly entered 70.07% instead of 70.00% for the GPT-4o-mini SSP result and apologize for this error; the entry is corrected to 70.00%. SSP accuracy is the pooled count of correct binary judgments over the 90 judgments (three runs × 30 questions), (k₄₂ + k₄₃ + k₄₄) × 100/90, each kₛ ∈ {0, …, 30}. All SSP entries in Table R2-3 are verified against the original judgments.

Table R2-3. GPT-4o-mini SSP corrected to 70.00% (table-entry error); all entries verified against the original judgments.

ReaderSSP (%)
GPT-4o-mini70.00
Qwen3-8B66.67
Llama 3.2 3B63.33

Thanks to the authors for the response. However, it does not address my concerns about the reliability of the experimental results and the code, and the requested control experiments were not provided. I will maintain my rating.

Reviewer_2RnLOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CIT-Mem, a counterfactual training framework for evidence-grounded memory policies in long-horizon LLM agents. The method constructs evidence-preserving and evidence-changing intervention pairs, filters them using multiple validation criteria, and trains a memory controller to select evidence, perform memory operations, and abstain when sufficient evidence is unavailable. The authors report improvements on LoCoMo-10 and LongMemEval-S across three frozen reader models. The problem is important, and the proposed training framework is interesting.

Critical strengths
  1. The paper correctly identifies that answer accuracy alone does not demonstrate whether an agent genuinely relies on retrieved evidence. The distinction between evidence-preserving and evidence-changing interventions is meaningful and interesting. 2.Reasonable overall framework. Jointly training evidence selection, memory operations, and abstention is a promising direction. The proposed validation procedure also recognizes the importance of filtering unreliable counterfactual examples. 3.Several useful experimental controls. The paper evaluates multiple readers and datasets, provides component ablations, separates several sources of improvement and includes an independent human audit of counterfactual samples. The experiment was thorough and comprehensive.
Critical weaknesses
  1. Limited Novelty. While the paper addresses an important problem, the novelty of the proposed method is somewhat limited. Several key ideas have been explored in prior work. For example, Causal Intervention-based Memory Selection (CMI) already investigates evidence dependence through counterfactual interventions, while Memory-R1 learns memory management policies through reinforcement learning. Additionally, Sufficient Context studies evidence sufficiency and abstention, and FaithEval evaluates model faithfulness under counterfactual and conflicting contexts. The proposed Adaptive Counterfactual Replay is also closely related to Prioritized Experience Replay (Schaul et al., 2016). Although CIT-Mem combines these ideas into a unified training framework, its main contribution appears to be the integration of existing techniques rather than a fundamentally new learning mechanism. Moreover, the lack of direct comparisons with trainable memory policies such as Memory-R1 makes it difficult to determine how much improvement comes specifically from the proposed counterfactual training strategy.
  2. The main picture is blurry and unclear, making it difficult to read.
  3. In LongMemEval-S, there are 30 specially designed unanswerable questions. However, the paper did not separately present the performance of these questions in terms of incorrect answers while abstention is one of the key points that you considered in memory mechanism.
Questions for authors
  1. The reported LongMemEval-S category accuracies are difficult to reconcile with the stated three-seed binary evaluation protocol. For instance, the single-session-preference (SSP) category contains 30 questions, yet Table 2 reports 70.07% accuracy. Could the authors clarify the reason? This situation also occurs in several other places in your table. This raises a substantial concern about the reliability and reproducibility of the main experimental results.
  2. The supplementary code does not implement several central methodological components as described. For example, the paper defines a soft-minimum evidence-set loss and a multi-head base objective, whereas the released implementation uses a retrieval hinge loss and does not include operation supervision in the objective. The released replay mechanism also differs from the stated update and importance-correction procedures. Although the authors disclose that the release is partial, these differences prevent verification of the core method.
  3. More experiments on weakness 3 on unanswerable question, specifically in CITmem framework on abstention consideration.
2 comments
AuthorsOpenReview ↗

Response to Reviewers

Response to Reviewer 1

We thank the reviewer for this careful assessment and the opportunity to correct a reporting error, clarify our release plan, and delimit the scope of our evidence.

R1.1. Correction of the SSP entry and aggregation

When preparing the manuscript table, we mistakenly entered 70.07% instead of 70.00% for the GPT-4o-mini SSP result. We apologize for this reporting error and take full responsibility for the oversight. We have corrected the entry to 70.00% in Table 1, and we agree with the reviewer that neither averaging across seeds nor changing the aggregation convention explains the erroneous entry.

With 30 questions, three runs, and one binary judgment per question per run, SSP accuracy is the pooled count of correct binary judgments over the 90 judgments (3 × 30), i.e., (k₄₂ + k₄₃ + k₄₄) × 100 / 90, where each kₛ ∈ {0, …, 30} is the number of correct judgments in run s.

All values in Table 1 have been verified against the original judgments: the non-SSP scores reproduce our submitted Table 2, and every SSP entry corresponds to an integer number of correct binary judgments under the aggregation above.

Table 1. GPT-4o-mini SSP is corrected to 70.00%; all values in this table have been verified against the original judgments.

ReaderMethodTRMSKUSSUSSASSP
GPT-4o-miniA-MEM36.0940.3072.8282.8687.5040.00
Mem049.1766.1778.7294.2951.7950.00
MemoryOS53.3844.8156.1581.1478.1853.33
MemOS65.1158.8076.6793.7167.8653.33
TiMem68.4270.8386.1695.7182.1463.33
CMI-Mem61.2055.4079.6090.2589.2060.00
CIT-MEM72.5969.1790.5694.5792.7970.00
Qwen3-8BA-MEM34.8341.4072.3679.7783.4736.67
Mem047.3062.2378.4188.5150.6850.00
MemoryOS49.2244.0153.1382.7374.5953.33
MemOS62.1157.4876.5586.1062.1546.67
TiMem67.2169.8184.3793.9080.6363.33
CMI-Mem58.3254.2776.8587.4589.9256.67
CIT-MEM71.0367.1188.7594.8589.0366.67
Llama 3.2 3BA-MEM31.6533.4561.3664.8069.8426.67
Mem037.6150.2557.0869.1833.8943.33
MemoryOS40.2734.6548.6961.7157.2946.67
MemOS48.9044.6260.7570.2341.3540.00
TiMem52.8665.3168.4679.1274.7553.33
CMI-Mem47.1546.8465.2077.4576.6150.00
CIT-MEM61.4763.9572.5977.8881.1963.33

R1.2. Code availability and our release plan

Our repository is indeed a partial release: it contains the core framework but lacks the complete training objectives (Eqs. (7)–(9)), the replay procedure, and reproduction workflows. We will release the complete implementation with reproduction instructions so that the reported results are traceable to code.

R1.3. Evidence deletion and abstention

Abstention is not assigned merely because an annotated source was deleted: each abstention twin must pass the recoverability gate in Eq. (4), which rejects a proposed twin when an alternative validated sufficient support exists among the exposed candidates. Replacement and temporal-update twins carry changed-answer targets.

This gate operates within the exposed pool and our support-validation procedure. We do not claim unanswerability over the entire history or that the rule prevents the controller from exploiting editing artifacts, nor generalization to naturally unanswerable questions; the current evidence does not replace the requested shortcut controls.

Reviewer_2RnLOpenReview ↗

Response to Authors

Thank you for the clarification and correction. The novelty question is not addressed since methodological novelty is the core barrier for a framework contribution.

On abstention, the authors themselves acknowledge that the current evidence does not support generalization to naturally unanswerable questions and does not replace the requested controls, which concedes rather than resolves this weakness.

Meanwhile, the released code still does not implement the stated objectives. The unclear main figure is also a concern. These gaps indicate that the overall quality of the work remains below the acceptance threshold. I therefore maintain my original rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.