- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes PARALLAX, a two-phase defense against memory poisoning in LLM agents from a causal perspective. The online phase uses counterfactual memory rewriting and dual-track execution to detect and block suspicious actions. The offline phase identifies and removes the responsible memory records through replay-based causal attribution. Experiments across multiple agents and attacks demonstrate strong defense effectiveness.
Critical strengths
-
Interesting and important problem. The paper addresses persistent memory poisoning, which is an important security concern for memory-augmented agents.
-
Interesting perspective. I appreciate the use of counterfactual reasoning to connect online action interception with offline memory attribution and cleanup.
-
Strong experimental results. The paper evaluates multiple agents, attacks, and defense baselines, showing strong improvements in attack prevention and attribution accuracy.
Critical weaknesses
-
(Main concern) Efficiency and scalability concerns. The online phase requires an additional counterfactual execution track at every action boundary, while offline attribution reportedly takes approximately 300 seconds per intercepted event. Moreover, attribution is evaluated with only four retrieved memory records. It remains unclear how the framework would scale to long-horizon agents with larger memory contexts. A comparison with a simpler intent-grounding-only guard would also help justify the additional computational cost.
-
Unclear reliability of counterfactual reasoning. The proposed method relies on rewriting memory and comparing the resulting actions. However, an action difference does not necessarily indicate malicious influence, since legitimate memories are also expected to affect agent behavior. It remains unclear how reliably the method distinguishes benign memory-dependent decisions from malicious ones, particularly in more complex tasks.
-
Causal attribution does not necessarily identify malicious records. The offline phase identifies minimal trigger sets through replay, but a record that contributes causally to a suspicious action is not necessarily attacker-poisoned. Removing entire trigger sets could therefore delete useful benign memories. I would appreciate further analysis of this distinction and its impact on memory utility.
-
Limited proactive coverage and long-term evaluation. Offline attribution is triggered only after an action has been intercepted, leaving poisoned memories that have not yet caused detectable suspicious behavior unexamined. Furthermore, the paper does not sufficiently evaluate whether memory cleanup prevents future attacks while preserving long-term agent utility across subsequent sessions.
Questions for authors
See weaknesses.