- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper addresses a real tension in personalized agents with persistent memory. Suppose a new statement contradicts a stored preference. It may be any of four things:
- a genuine change of mind;
- a context-specific preference;
- benign noise, such as sarcasm or a typo;
- a poisoning attempt, for example via a retrieved document, or text the user pastes from an attacker-controlled source ("same-origin" laundering).
Personalization methods accept every update. Injection defenses reject untrusted channels. So each fails on the case the other handles.
Theory (§2, App. A). For any rule that reads only recency and provenance, the false-update rate plus the rate of rejecting genuine updates is at least 1 minus the total-variation distance an adversary can achieve (Thm 2). Conditioning on the full history never hurts, and helps strictly under a stated condition (Thm 3). A bounded-cost clarifying question enlarges the achievable error region (Thm 4).
Method (CAPTURE). CAPTURE is a pipeline of five components:
- a Qwen2.5-3B + LoRA extractor that produces a structured preference hypothesis;
- an "authenticity gate": a latent belief evolving by a neural ODE between events, plus a GRU update, which chooses among six memory actions (retain, add, scope, revise, quarantine, clarify) and asks the user when the action entropy is high;
- a three-layer ledger (stable, contextual and transient preferences), each layer with a fixed decay rate, plus a quarantine state;
- a best-of-N selector under a fixed safety constraint;
- a counterfactual citation audit, which removes each cited memory, re-decodes, and checks whether the answer changes.
Evaluation. A new synthetic benchmark, D-PrefGuard, has 2,400 episodes of 12–20 turns across five tracks, with 480 test episodes from 96 users. Against eight baselines, CAPTURE reaches a 71.5% win rate, 11.5% poisoning, 83.5% adherence to genuine updates, and 8.9% false updates. A supervision-matched Transformer reaches 69.3% and 15.9%. Several further checks probe robustness:
- an adaptive attacker raises poisoning to 24.7%, statistically tied with Provenance-only at 22.1%;
- 240 attacks from twelve independent red-teamers;
- zero-shot evaluation on HorizonBench;
- offline replay of 40 participants' 2–3-week histories.
Critical strengths
**S1. An important, well-posed problem, and an unusually rigorous and honest evaluation.**Separating genuine preference change from memory poisoning is a real deployment problem, and Fig. 1 shows that the two existing families sit at opposite ends of the trade-off. The evaluation goes well beyond the norm:
- Supervision is separated from architecture. A supervision-matched Transformer receives the same labels, extractor, clarification channel, selector and safety scorer (Q4).
- Clarification is controlled for. A query-budget-matched StateMem shows that asking alone does not explain the margin.
- The optimization target is kept apart from the metric. The reward model used by the selector is separate from the one used to score regret.
- Strong attackers are tested. An adaptive attacker with the released weights is included, along with 240 attacks from twelve red-teamers, analyzed with a crossed two-way bootstrap.
- Results are checked beyond the paper's own generator, through zero-shot HorizonBench and a participant replay.
- Unfavorable results are reported plainly. Provenance-only ties CAPTURE under adaptive attack, CAPTURE does not lead on HorizonBench belief-revision accuracy, and falsification criteria are stated in advance.
S2. A clear empirical answer to a meaningful question. On held-out data CAPTURE improves both axes at once relative to Provenance-only: poisoning falls from 20.0% to 11.5%, adherence rises from 74.1% to 83.5%, and false updates fall from 19.6% to 8.9%. The gain is concentrated where theory predicts. On same-origin contamination, attack success is 17.5% against 43.6% for Provenance-only. On context switches, CAPTURE chooses the correct "scope" action at 71% of changepoints against 4% for StateMem (App. H). The ordering of methods holds under independently authored attacks (18.3% vs 26.3% for Provenance-only).
Critical weaknesses
W1. The architectural contribution is small relative to the system's complexity, and much of it does not hold up beyond the paper's own benchmark or under adaptive attack.
- Supervision explains most of the gain. The supervision-matched Transformer closes most of the gap: CAPTURE leads by only +2.2 win-rate points [0.6, 3.9] and −4.4 poisoning points (Q4). On human labels the action macro-F1 is 0.71 vs 0.68, a difference that is not significant (App. E).
- On external data the gain shrinks. Against the matched Transformer it is significant on HorizonBench contradiction resolution (+0.04) and replay win rate (+1.7, p = 0.031). It is not significant on belief-revision accuracy (p = 0.27) or replay poisoning (p = 0.064). The authors call this "a modest result."
- The headline components contribute little.
- Removing the neural ODE costs only +2.1 pp poisoning and +0.02 regret (Table 6).
- A discrete gate with an explicit Δt input recovers about 55% of the continuous-time advantage (App. J).
- The episodes have only 12–20 events, with a median gap of 8 hours, so the case for continuous-time dynamics is thin.
- Most of the security result comes from the gate classifier (+22.0 pp) and the provenance feature (+14.0 pp).
- **Under adaptive attack, CAPTURE is not more secure than a training-free heuristic.**Poisoning more than doubles, to 24.7%, and CAPTURE is statistically tied with Provenance-only (difference 2.6 [−1.9, 7.2], p = 0.29). The adaptive attack is also limited: surrogate-state paraphrase search with origin and timing held fixed.
- The system is costly. It has five components (extractor, ODE gate, three-layer ledger with quarantine, best-of-N safety selector, counterfactual re-decoding audit). It adds about 2.5 s and 3,850 tokens per turn (App. B, Table 6), against 480 ms for Provenance-only.
So the paper's own evidence supports "supervised authenticity classification over history, plus quarantine and scoping," rather than the specific ODE-and-ledger architecture.
W2. The theory is standard and loosely connected to the method, and the evaluation leans on the authors' own synthetic generator.
- The theorems are textbook results.
- Thm 2 is the classical Neyman–Pearson / Le Cam bound for testing between two distributions.
- Thm 3 is the data-processing inequality for total variation.
- Thm 4 follows largely from stipulating an accurate clarification channel. The authors themselves call it "close to definitional" (Remark 5) and say that "the tools are standard statistical decision theory."
The abstract's "we prove…" therefore overstates the novelty. - The theory doesn't justify the architecture. Thm 3 says any history-conditioned rule is at least as good as recency-plus-provenance. That supports some learned gate, but not the neural ODE or the ledger. The continuous-time POMDP and the dynamic-regret objective (Eq. 1) are never solved or optimized; the gate is trained by supervised six-class cross-entropy. - The labels come from the same program that generates the test data. The gate's action labels come from the generation program and "were not validated by humans at the event level" (App. B). They agree with independent human labels only 83.6% of the time (App. E), and the human check of the benchmark covers only 300 episodes, for track and changepoint. Disjoint generator families reduce this circularity but don't remove it. - The real-user evidence is weaker than "real users" suggests. - Participants used a neutral assistant. - Attacks were injected afterwards by the authors. - When CAPTURE asked a clarifying question, the stored annotation answered with zero error (oracle-assisted).
The authors acknowledge that poisoning robustness is not shown to transfer beyond author-constructed contamination (§6). - Episodes are short. At 12–20 turns, they cannot test the "weeks rather than turns" setting the abstract motivates.
Questions for authors
Clarifications that could change my assessment:
- Which latency figure is correct: "roughly 630 ms per turn" (§1) or 2,480 ms added per turn (App. B, Table 6)? What is the breakdown by component, including the audit's extra decode, and the end-to-end latency relative to the backbone? Note that in Table 6 the full system (2,480 ms) minus the variant without the gate (1,850 ms) is exactly 630 ms. The introduction's figure therefore appears to be the gate's cost alone, reported as the cost of the whole method. By the same subtraction, the causal audit alone costs about 1,030 ms while changing adherence by only 0.4 points and poisoning by 0.6 points.
- Given that the supervision-matched Transformer recovers most of the gain and that Discrete + Δt recovers about 55% of the continuous-time advantage, what evidence specifically supports the neural ODE over a simpler gate? Please report a Transformer variant that also has the three-layer ledger, to separate the ledger's contribution from the ODE's. Please also ablate the quarantine state on its own. App. H reports that 58% of blocked poisoning attempts are resolved by quarantine followed by non-corroboration, so it may be a key mechanism, yet Table 6 never removes it.
- Can you strengthen the adaptive attacker? For example, let it choose injection timing and channel, which Assumption 4 permits; run multi-turn slow attacks; or attack the extractor as well as the gate. How does CAPTURE compare with Provenance-only then?
- How much does gate performance depend on the generation program's labels? Can you train the gate, at least partly, on the 720 human-labeled events, or report test results on human-labeled event subsets broken down by track?
- Can the user study include live interaction with CAPTURE, at least on a subset, so that clarifying questions are answered by real users and their acceptance of one question per twelve turns is measured?
- Please explain why Provenance-only's FUR + RRL matches 1 − TV̂_φ to four decimals on the matched split (Table 4). Does the result change under a different binning, or when the TV estimate and the rule's errors are computed on separate samples?
- The safety selector is ablated only for CAPTURE (Table 6: safety attack success rises from 8.0% to 23.0%, and adherence rises 4.0 points). Because every method shares the selector, please report Table 1 with the selector removed for all methods, to show whether CAPTURE's relative advantage on win rate, adherence, poisoning and safety depends on it. Please also report sensitivity to τ_safe, w_p/w_s and N, and an ablation of the −β·r_s penalty when the r_s ≤ τ_safe constraint is kept.
- Please specify the four undescribed baselines (Recency RAG, Full history, Static summary, MemGPT default): their retrieval, recency weighting and summarization rules, and how each was configured or tuned. Please also add the supervision-matched Transformer to Figure 1. In addition, can you compare against existing systems: A-MemGuard (a memory-poisoning defense), Mem0 and Zep/Graphiti (deployed memories with update and invalidation logic), and a model-level injection defense such as SecAlign, combined with provenance?
- The HorizonBench evaluation contains no attacks, so outside your own generator only preference tracking is validated, not poisoning robustness. Could you inject your attack families into HorizonBench (or LongMemEval's knowledge-update histories), as you did in the participant replay, so that poisoning robustness is tested on an evolving-preference benchmark you did not build?
- On the LLM judge:
- Please report judge–human and human–human agreement on the same basis (e.g., judge vs each individual rater, or leave-one-rater-out), with Cohen's κ, broken down by track and by method.
- Please run a direct human comparison of CAPTURE against the supervision-matched Transformer on a subset.
- Please state how the response-level poisoning and adherence labels are produced, and how reliable they are.
- Please also define "Lag" and its unit, add intervals to every Table 1 cell, and explain why CAPTURE's safety attack success is below the unpersonalized model's. 11. How does CAPTURE perform on episodes much longer than 12–20 turns, closer to the "weeks" setting the abstract motivates?