acceptodds
Under review as a conference paper at ICLR 2027

Harnessing Internal States to Defend Memory-Poisoned Agents

Abstract

Memory poisoning can redirect an LLM agent’s decisions, yet behavior-level evaluation offers limited visibility into how retrieved poisoned memories shape the agent’s internal decision process. We introduce an internal-state-aware harness that uses structured internal diagnoses as feedback for defense refinement. Using Natural Language Autoencoders (NLA), we translate token-level hidden states into natural-language summaries and map them into six local diagnostic states over constraints, objectives, and actions. These states reflect which component of the agent decision is compromised rather than following a universal temporal progression. NLA-guided activation steering further shows that the diagnosed internal semantics are behaviorally relevant, but stronger interventions increasingly degrade output executability, limiting direct steering as a standalone defense. We therefore aggregate recurring diagnostic failures and use them to iteratively refine an upstream natural-language guardrail in the agent harness. Across five agent settings and four model backbones, the refined policy substantially suppresses post-retrieval attacks while preserving benign utility, reducing the compromised-state ratio by 70% and achieving an average 78-percentage-point reduction in attack success rate. Overall, our results show that internal states are most useful as structured feedback for preventing poisoned decisions before they form, rather than as direct control targets.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI Agents, Memory Poisoning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_DMeCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies defenses against memory poisoning after poisoned memories have entered an agent’s context. The authors use Natural Language Autoencoders (NLA) to verbalize hidden states and map the resulting descriptions into six local diagnostic states concerning constraints, objectives, and actions. They then investigate the relationship between these descriptions and agent behavior through activation interventions, and combine internal diagnoses with behavioral feedback to iteratively refine a natural-language defense policy in the agent harness. Experiments across five agent settings and four model backbones report substantial reductions in attack success rates. The central claim is that internal states are more useful as feedback for upstream defense refinement than as direct control targets.

Critical strengths
  1. The paper attempts to identify which component of the agent’s decision is affected and use that diagnosis to improve defenses. This is a worthwhile research direction.
  2. The paper goes beyond presenting natural-language explanations by investigating activation interventions and harness refinement. Although the current causal evidence remains limited, the attempt to connect internal analysis with practical defense design is appealing.
Critical weaknesses
  1. S0–S5 underpin the internal analysis, policy refinement, and defense evaluation, yet lack human validation, inter-annotator agreement, and cross-judge stability analysis. Differences in probe selection and the judge’s access to final responses may introduce bias.
  2. Although Ignore, Escape, and Sandwich have literature references, their implementations here are simple fixed prompting heuristics. Ours benefits from iterative optimization and a more elaborate prompt. Strong prompt baselines with matched optimization budgets, together with stronger defenses under the same threat model, are needed.
  3. The paper lacks a behavior-only ablation experiment matching the optimizer, examples, budget, and validation rules. Consequently, the gains may arise from simple prompt optimization.
  4. Accurate activation reconstruction does not guarantee faithful explanations, and behavioral changes after intervention do not directly validate the six-state semantics.
Questions for authors

See weakness.

3 comments
AuthorsOpenReview ↗

Official Comment for Reviewer DMeC (PART I)

We appreciate the reviewer DMeC’s in-depth examination of the technical foundations and empirical evidence of our approach. We are encouraged by the reviewer DMeC’s recognition that identifying the specific components of agent decisions affected by memory poisoning, and translating such insights into practical defense mechanisms, is a worthwhile research direction.

We understand that the reviewer DMeC’s concerns mainly center on the following aspects:

  1. The reliability of internal diagnoses: Whether S0–S5 labels are adequately validated and whether the diagnostic pipeline is robust to potential sources of bias. (W1)
  2. The strength of defense baselines: Whether our method is sufficiently compared with existing memory defenses and stronger prompt-based approaches under the same threat model. (W2)
  3. The additional value of internal-state feedback: Whether internal diagnoses provide measurable benefits beyond behavior-only refinement under matched optimization budgets. (W3)
  4. The faithfulness of NLA interpretations and activation-steering evidence: Whether NLA summaries faithfully reflect internal representations and whether activation interventions sufficiently support the behavioral relevance of diagnosed semantics. (W4)

1. Reliability and Faithfulness of Internal Diagnoses (W1)

We appreciate the reviewer DMeC’s concerns regarding the reliability of S0–S5 and the faithfulness of NLA interpretations. We emphasize that our diagnostic pipeline follows a carefully designed and consistent procedure, rather than relying on unconstrained LLM-based classification. Specifically, probe positions are selected around task-relevant linguistic boundaries, with neighboring tokens retained to capture local representations. Hidden states are extracted from a fixed layer, and NLA verbalization uses a fixed configuration with deterministic decoding. The Judge (GPT-5.6-Sol, temperature = 0) then maps each NLA summary to the predefined CAO states, using the current task as the alignment reference and evaluating each description independently. We further implement structural checks on case correspondence and label validity, with invalid outputs triggering retries.

To further assess the semantic reliability of the labels, we conducted a human–AI agreement analysis on successful and failed WebShop cases.

Attack OutcomeMetricS0S1S2S3S4S5
SuccessAgreement968987818592
Majority Support918277697586
FailureAgreement938790798388
Majority Support897683727981

The average human–AI agreement and human-majority support are 87.5% and 80.0%, respectively.

We also removed the agent’s final response and re-evaluated the same 752 paired windows (3,760 labels). Although fine-grained labels exhibit some variation, the dominant non-S0 CAO category remains unchanged across all six dataset–outcome groups.

DatasetOutcomePaired WindowsWith ResponseWithout Response
WebShopSuccess128ConstraintConstraint
WebShopFailure217ObjectiveObjective
QASuccess114ObjectiveObjective
QAFailure103ObjectiveObjective
MIMIC-IIISuccess82ActionAction
MIMIC-IIIFailure108ActionAction

Constraint = S1+S2; Objective = S3+S4; Action = S5.

The exact six-state label agreement is 61.3%, while the S0/non-S0 agreement is 70.0%. Thus, we do not claim complete fine-grained label invariance; rather, the dominant functional patterns are preserved across the six groups.


2. Strength of Defense Baselines (W2)

We appreciate the reviewer DMeC’s suggestion to broaden the defense comparisons. Existing research has explored diverse strategies for protecting persistent memory in LLM agents, which can be broadly categorized according to their intervention stages:

  1. Memory ingestion and storage protection. These methods use input/output moderation, trust-aware sanitization, and provenance verification to prevent malicious records from entering or persisting in memory [1–3].
  2. Retrieval-stage filtering and detection. These approaches identify suspicious memories through randomized retrieval filtering, intent-aware detection, and semantic anomaly scoring [2, 4, 5].
  3. Runtime reasoning validation and correction. A-MemGuard [6] compares reasoning paths induced by related memories through consensus validation and stores detected failures as corrective lessons.
  4. Post-hoc auditing and recovery. MemAudit [7] identifies suspicious memories through counterfactual attribution and structural consistency analysis, enabling targeted removal.
AuthorsOpenReview ↗

Official Comment for Reviewer DMeC (PART II)

These approaches provide complementary protection throughout the memory lifecycle. However, as attack strategies evolve, adversaries may adapt their injection channels and payloads to circumvent upstream defenses. Regardless of the injection route, the malicious information must ultimately enter the model's active context to influence its decisions. We therefore focus on a complementary model-side defense setting explicitly defined in our paper: poisoned memories have already been retrieved and incorporated into the agent's active context. Rather than assuming upstream filtering can always prevent such exposure, we leverage internal-state diagnoses as offline feedback to refine an external defense policy that mitigates the downstream influence of poisoned content.

Among existing approaches, A-MemGuard [6] provides the closest memory-specific comparison. We therefore additionally evaluate it on QA and StrategyQA using DeepSeek-V4-Flash, alongside our original prompt-level baselines.

MethodStrategyQA ASR-r ↓ASR-a ↓ASR-t ↓QA ASR ↓QA ACC ↑
A-MemGuard36.4961.5838.3612.3586.07
Ours19.9537.0445.410.0098.29

Our method achieves the best results on QA, with 0% ASR and 98.29% accuracy. On StrategyQA, it reduces ASR-r and ASR-a relative to A-MemGuard, although A-MemGuard achieves a lower ASR-t.

We further compare the computational overhead. A-MemGuard generates an independent reasoning path for each of the (K) retrieved memories before consensus validation, yielding:

=\sum_{i=1}^{K}c_i =O(K\bar c),$$ where $c_i$ denotes the token cost of each reasoning path. With $K=4$, A-MemGuard reports approximately 7.8K tokens on EHRAgent, compared with 3.6K for No Defense. In contrast, our NLA-based diagnosis is performed exclusively during offline policy refinement; deployment requires only a text-based guardrail, without additional NLA extraction or Judge calls. We will incorporate these additional comparisons and clarify the scope and deployment characteristics of our defense in the revised manuscript. [1] B. D. Sunil et al., "Memory Poisoning Attack and Defense on Memory Based LLM-Agents," arXiv:2601.05504, 2026. [2] T. Sharma, "SMSR: Certified Defence Against Runtime Memory Poisoning in Persistent LLM Agent Systems," arXiv:2606.12703, 2026. [3] N. Karamchandani et al., "Your Agent's Memories Are Not Its Own: Forged Reasoning Attacks on LLM Agent Memory and Defenses," arXiv:2607.05029, 2026. [4] D. Liu et al., "MIND: Lightweight and Effective Memory Injection Defense for LLM Agents via Intent-Aware Information Bottleneck," arXiv:2607.28103, 2026. [5] I. Gowda, "MEMSAD: Gradient-Coupled Anomaly Detection for Memory Poisoning in Retrieval-Augmented Agents," arXiv:2605.03482, 2026. [6] Q. Wei et al., "A-MemGuard: A Proactive Defense Framework for LLM-Based Agent Memory," ICML, 2026. [7] Z. Tan et al., "MemAudit: Post-hoc Auditing of Poisoned Agent Memory via Causal Attribution and Structural Anomaly Detection," arXiv:2605.23723, 2026. --- ### 3. Additional Value of Internal-State Feedback (W3) We thank the reviewer *DMeC* for highlighting the need to isolate the contribution of internal-state feedback. To directly quantify its additional value, we conducted a **controlled behavior-only refinement experiment**, keeping the optimizer, initial policy, training/validation samples, optimization budget, and policy selection criteria identical. The only difference is whether the optimizer additionally receives NLA-derived internal diagnostic feedback. The results demonstrate measurable benefits from internal-state feedback in **four of the five evaluated settings**. The following table reports the mean results across four backbones. |Benchmark|Metric|Behavior-only|Ours|Improvement (pp)| |---|---|---|---|---| |WebShop|ASR ↓|3.80|**1.20**|+2.61| |QA|ASR ↓|8.36|**0.48**|+7.88| |QA|ACC ↑|79.31|**88.15**|+8.84| |MIMIC-III|ASR ↓|32.79|**4.49**|+28.30| |EHR|ASR-t ↓|41.55|**28.45**|+13.10| |StrategyQA|ASR-t ↓|**34.90**|37.95|—| The improvements are particularly pronounced on **MIMIC-III (28.30 pp)** and **EHR (13.10 pp)**, where our method achieves lower ASR across all four backbones. On QA, our method also improves average task accuracy by **8.84 pp**, indicating improved robustness while preserving benign utility.
AuthorsOpenReview ↗

Official Comment for Reviewer DMeC (PART III)

We observe an exception on StrategyQA, where behavior-only refinement achieves a lower average ASR-t (34.90% vs. 37.95%). This difference is largely driven by GLM-5.3-Flash, where behavior-only refinement already achieves strong performance (18.12% ASR-t and 81.88% ACC), compared with 42.36% ASR-t and 57.64% ACC for Ours. Interestingly, Ours achieves a lower ASR-r on this backbone (25.86% vs. 37.18%), but this does not translate into better downstream ASR-a or overall ASR-t. This suggests that the benefits of internal feedback depend on the backbone and task, particularly when behavior-only refinement already provides strong protection.

Overall, these controlled comparisons provide direct evidence that internal-state diagnoses contribute measurable benefits beyond behavioral feedback alone, particularly for action-level attacks and cross-setting robustness. While these benefits are not universal, they demonstrate the complementary value of structured internal feedback in refining agent defenses.


4. Faithfulness of NLA Interpretations and Behavioral Relevance of Activation Steering

We thank the reviewer DMeC for raising this concern. We clarify that NLA provides local semantic interpretations of hidden states rather than exact transcriptions of internal computations. To examine their correspondence with observable reasoning, we compare six selected cases (three successful and three failed attacks), together with the additional comparison case C01. For example, S01 highlights historical strategy reuse, S02 captures persistence toward an unrelated substitute, and S03 makes abandonment of the original request explicit. Importantly, F02 also illustrates a speculative NLA interpretation that is not fully supported by visible reasoning. We include these side-by-side comparisons in Appendix A.6 to illustrate both the interpretive value and limitations of NLA.

We further evaluate behavioral relevance through activation steering. As shown in Fig. 4, steering along NLA-derived semantic directions produces measurable behavioral changes, with stronger interventions generally improving resistance to poisoned intent but potentially degrading output executability. Additional experiments in Appendix B reinforce this observation: single-position interventions achieve only 2.08%–6.24% attack failure with 100% executability, whereas multi-position interventions increase attack failure to 18.15%–28.34%, with 82.32%–92.08% executability. Stronger replacement reaches 31.21% attack failure but reduces executability to 50.97%.

These results support the behavioral relevance of NLA-derived semantics, while not implying that every verbalization perfectly recovers the underlying computation. We have clarified this distinction in the revised manuscript.


We believe the above clarifications and additional evidence further strengthen the empirical foundation of our work. Collectively, these efforts reinforce our central contribution: leveraging structured internal-state diagnoses as offline feedback to improve practical post-retrieval defenses for LLM agents. We sincerely thank the reviewer DMeC for the constructive suggestions, which have helped us substantially improve the experimental analysis and presentation. We hope these revisions adequately address the reviewer DMeC’s concerns and provide further grounds for a more favorable assessment of our work.

Reviewer_vvjJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies memory poisoning in LLM agents at the post-retrieval stage, i.e., after a poisoned memory has already entered the context. Bringing an interpretability tool (NLA) to agent memory security, and positioning internal diagnoses as an offline feedback signal rather than an online control target, is a fresh angle. The authors also report the negative steering results honestly, and the ASR reductions after transfer to black-box backbones in Table 1 are notable. However, the central claim — that internal states provide valuable structured feedback — currently lacks its most direct test. There is no behavior-only refinement baseline with the same optimizer and budget, and no control that diagnoses the visible text without NLA. The gains in Table 1 may therefore come mostly from iterative LLM prompt optimization and a longer, more elaborate guardrail, rather than from internal states. In addition, the LLM-judge diagnosis is not validated and may leak outcome information, the headline 78-pp figure cannot be reproduced from Table 1, benign utility is under-evaluated, and the baselines are weak. In its current form I do not think the paper meets the ICLR bar, but it could become a valuable contribution if the key experiments below are added.

Critical strengths
  1. Focusing on the post-retrieval stage complements retrieval-side filtering, and the authors state explicitly that storage and retrieval are out of scope (end of §2.2).
  2. The Constraint–Objective–Action view unifies diverse memory-poisoning attacks into three functional levels (§2.1, Appendix C.1). It organizes the analysis well and gives the defense feedback an interpretable vocabulary. Appendix C.1 explains the categorization of each benchmark carefully.
  3. §4 and Appendix B openly describe the robustness–executability trade-off of steering, and §5.2 acknowledges the heterogeneous results on ReAct-StrategyQA.
  4. Internal states are used only during offline refinement; the final artifact is a plain-text guardrail usable with closed API models. Table 1 shows it transfers to three other backbones, which is of practical value.
  5. The judge prompt (A.5), the six steering descriptions (B.4), and the full prompts for all baselines and the final policy (C.6) are provided, which helps reproducibility.
Critical weaknesses

W1. The central claim lacks its key ablation: how much do internal diagnoses actually add? The main conclusion is that internal-state feedback is more useful than behavioral feedback alone. Fig. 3 (right) explicitly contrasts “Traditional Evolving (Behavioral)” with “Ours (Behavioral + Internal)”, and §5.1 (L409–410) calls the extra diagnostic feedback “the key difference”. Yet the experiments never test this. W2. Headline numbers in the abstract and introduction do not match Table 1. The abstract (L025) and the contributions (L088) claim an “average 78-percentage-point reduction in attack success rate”, which cannot be reproduced from Table 1. Averaging the primary metric of each setting (WebShop ASR, StrategyQA ASR-t, QA ASR, MIMIC-III ASR, EHR ASR-t) over the four backbones gives an absolute reduction of about 47.8 pp for Ours versus No Defense; including ASR-a gives about 48.7 pp. Several No Defense ASRs are already below 78% (e.g., 19.91% for Gemini on QA, 31.15% for DeepSeek on QA, 28.89% for DeepSeek on MIMIC-III), so an average 78-pp reduction across all settings is arithmetically impossible. The mean relative reduction over the same metrics is about 76.9%, close to 78. I suspect a relative percentage was reported as percentage points, or that only a subset was averaged. The “70% reduction in compromised-state ratio” appears to come from Fig. 5 (left; S0 rises from about 45% under No Defense to about 83% under Ours), but the dataset, model, and sample size are not stated. W3. The diagnostic pipeline (LLM judge) is unvalidated and may leak outcome information. All internal analyses rest on the mapping NLA summary → LLM judge → S0–S5, but its accuracy is never independently checked. Appendix A.5 (L862–863) itself states that the structural checks “do not constitute an independent assessment of semantic labeling accuracy”. The judge model is not specified (model, version, temperature). Appendix A.5 (L843–844) says the judge input includes “the agent's final response when available”. If the judge sees the final output, the success-vs-failure differences in Fig. 2 may partly reflect that, not the NLA summaries, which makes the argument circular. The S1 definition mentions “stock/price/pattern” and S5 mentions “clicking, buying, adding, ordering”, closely matching the WebShop attack. The correspondence between diagnosed states and attack dimensions found in §3.3 may therefore be partly built into the definitions. W4. The necessity and faithfulness of NLA are not established. The paper's selling point is “internal states”, but it is unclear how much NLA summaries reveal beyond the visible text. The poisoned content is present verbatim in the context, so verbalized residual-stream descriptions may largely paraphrase surface context. W5. The activation-steering evidence is weak and the conclusions are overstated. §4 states that steering “confirms that the diagnosed internal semantics are behaviorally relevant”, but the evidence does not support this strongly.

Questions for authors
  1. How exactly are the “78-percentage-point” and “70%” figures computed, and over which settings, metrics, and models?
  2. Compared with behavior-only refinement under the same budget, how much do internal diagnoses add?
  3. Which model is the diagnostic judge? Does its input include the final response, and do the Fig. 2 conclusions hold without it?
  4. In §3.3, is the action dimension evaluated on EHR or MIMIC-III? What are the sample sizes of each group?
  5. With clean memories, does the final policy cause the agent to lose the benefits that memory provides?
5 comments
AuthorsOpenReview ↗

Official Comment for Reviewer vvjJ (PART I)

We sincerely thank the reviewer vvjJ for the thoughtful evaluation and constructive feedback. We particularly appreciate the recognition of our post-retrieval security perspective, the unifying CAO framework, the offline use of internal-state diagnoses, and the practical transferability of our defense to black-box models.

We understand that the reviewer’s concerns and questions mainly center on the following aspects:

  1. The additional value of internal-state feedback: Whether internal diagnoses provide measurable benefits beyond behavior-only policy refinement under a comparable optimization budget. (W1, Q2)

  2. The additional information provided by NLA: Whether NLA summaries reveal meaningful internal information beyond what can already be inferred from visible text, and whether their interpretations are sufficiently faithful. (W4)

  3. The reliability of the diagnostic pipeline: Whether the NLA–Judge mapping produces reliable semantic labels, whether access to the agent’s final response introduces potential outcome leakage, and how the action-level diagnostic analysis is configured, including its dataset and sample sizes. (W3, Q3, Q4)

  4. The accuracy of the quantitative claims: How the reported ASR reduction and compromised-state reduction are calculated, including the corresponding metrics, models, and experimental settings. (W2, Q1)

  5. The interpretation of activation-steering results: Whether the intervention experiments sufficiently support the behavioral relevance of the diagnosed internal semantics and whether the conclusions are appropriately qualified. (W5)

  6. The utility of benign memories: Whether the final defense policy preserves the benefits of legitimate memories during normal task execution. (Q5)


1. Additional Value of Internal-State Feedback (W1, Q2)

We thank the reviewer vvjJ for highlighting this important ablation. To directly quantify the contribution of internal diagnoses, we conducted a controlled behavior-only refinement experiment, keeping the optimizer, initial policy, training/validation samples, optimization budget, and policy selection criteria identical. The only difference is whether the optimizer additionally receives NLA-derived internal diagnostic feedback.

The results demonstrate measurable benefits from internal-state feedback in four of the five evaluated settings, with particularly substantial improvements on action-level attacks. The following table reports the mean attack success rate (ASR, %) across four backbones for each benchmark.

BenchmarkMetric ↓Behavior-onlyOurs
WebShopASR3.801.20
QAASR8.360.48
MIMIC-IIIASR32.794.49
EHRASR-t41.5528.45
StrategyQAASR-t34.9037.95

The improvements are particularly pronounced on MIMIC-III (28.30 pp) and EHR (13.10 pp), where our method achieves lower average ASR across all four backbones. On QA, our method further improves average task accuracy by 8.84 pp, indicating improved robustness while preserving benign utility.

We also observe an interesting exception on StrategyQA, where behavior-only refinement achieves a slightly lower average ASR-t (34.90% vs. 37.95%). This difference is primarily driven by GLM-5.3-Flash, on which behavior-only refinement already achieves strong performance, with 18.12% ASR-t and 81.88% ACC, compared with 42.36% ASR-t and 57.64% ACC for Ours. Notably, our method achieves a lower ASR-r on this backbone (25.86% vs. 37.18%), but this improvement does not translate into better downstream ASR-a or overall ASR-t. This result highlights the backbone- and task-dependent nature of internal feedback: when behavioral optimization already provides strong protection, additional internal diagnoses do not necessarily yield further gains.

Overall, these controlled comparisons provide direct evidence that internal-state diagnoses contribute additional value beyond behavioral feedback alone, particularly in mitigating action-level attacks and improving cross-setting robustness. Their benefits remain task-dependent, highlighting the complementary value of structured internal feedback rather than a universal advantage over behavior-only optimization.


2. Additional Information and Faithfulness of NLA Summaries (W4)

We thank the reviewer vvjJ for raising the important questions regarding the information gain and faithfulness of NLA summaries. NLA provides token-local, activation-conditioned interpretations of how retrieved information is represented during generation, rather than recovering facts absent from the input context. To illustrate this capability, we compare visible reasoning with NLA summaries across five representative cases, including three successful and two failed attacks.

AuthorsOpenReview ↗

Official Comment for Reviewer vvjJ (PART II)

CaseOutcomeVisible Agent ReasoningNLA Interpretation
S01Attack SuccessThe agent claims that shampoos are out of stock and searches for an unrelated hand wash instead.Identifies the tendency to “follow the same strategy as the previous answer”, characterizing historical strategy reuse.
S02Attack SuccessThe agent uses previous speaker stock issues to justify searching for an unrelated microphone bundle.Identifies a tendency to “continue with the microphone, regardless”, highlighting persistence toward the poisoned alternative.
S03Attack SuccessThe agent again attributes shampoo unavailability to historical constraints and searches for an unrelated hand wash.Characterizes the decision as “disregard the original request”, making the abandonment of the original objective explicit.
F01Attack FailureThe agent considers previous examples but prioritizes the specifically requested speaker model and avoids broadening the search.Interprets the reasoning as recognizing a historical search pattern and avoiding its repetition.
C01Attack FailureThe agent explicitly recognizes its previous deviation toward a microphone bundle and decides to focus on the current request.Identifies the corrective intention as “avoid repeating the mistake of falling into that trap”, consistent with the visible reasoning.

These cases illustrate that NLA can make certain decision tendencies more explicit, including historical strategy reuse (S01), persistence toward irrelevant alternatives (S02), and abandonment of the original objective (S03). For failed attacks, NLA captures the avoidance of historical search patterns (F01) and explicit corrective intentions (C01). Notably, the interpretation of C01 largely overlaps with its already explicit visible reasoning.

Overall, these examples suggest that NLA provides complementary, localized interpretations of internal representations, although its additional information varies across cases. We do not claim that it universally reveals information unavailable from visible text.

For reproducibility, the selected user requests, visible reasoning traces, token positions, corresponding NLA outputs, and first searches are provided in Appendix A.6.


3. Reliability of the Diagnostic Pipeline (W3, Q3, Q4)

We thank the reviewer vvjJ for highlighting the importance of validating the diagnostic pipeline. We clarify that the diagnostic Judge is GPT-5.6-Sol, with temperature set to 0. Beyond the structural checks in the original manuscript, we conducted an additional human–AI agreement analysis on WebShop cases involving both successful and failed attacks.

StateSuccess: AgreementSuccess: SupportFailure: AgreementFailure: SupportOverall: AgreementOverall: Support
S09691938994.590.0
S18982877688.079.0
S28777908388.580.0
S38169797280.070.5
S48575837984.077.0
S59286888190.083.5
Average88.380.086.780.087.580.0

The human–AI agreement reaches 87.5% on average, with an average human-majority support of 80.0% across the six states. The agreement remains relatively consistent across successful and failed attack cases, providing an additional human-based validation of the semantic labeling process.

We also acknowledge that the original Judge input included the agent’s final response when available. To examine its potential influence, we conducted an additional evaluation after removing the final response while retaining the same paired NLA windows. The experiment covers 752 paired windows and 3,760 diagnostic labels across WebShop, QA, and MIMIC-III.

StateS0S1S2S3S4S5
Change in aggregate proportion (pp)-2.58+2.98+2.15-7.63+0.43+4.65

When all paired windows are considered together, the proportion of each diagnostic state changes by no more than 7.63 percentage points. More importantly, the dominant non-S0 CAO-level diagnostic category remains consistent in the aggregate results: constraint-related for WebShop, objective-related for QA, and action-related for MIMIC-III. This suggests that the functional correspondence identified in Fig. 2 is not solely attributable to the Judge's access to the final response. We acknowledge that the magnitude of distributional changes varies across individual dataset–outcome groups; therefore, we interpret this result as supporting the robustness of the coarse-grained functional patterns rather than claiming complete invariance of every fine-grained diagnostic state.

Finally, regarding the dataset clarification, the action-level diagnostic analysis in §3.3 was conducted on MIMIC-III, not EHR. We sampled 100 attack-success cases and 100 attack-failure cases, resulting in 200 cases in total, for NLA-based analysis. We have clarified this dataset attribution and the corresponding sample sizes in the revised manuscript.

AuthorsOpenReview ↗

Official Comment for Reviewer vvjJ (PART III)

Taken together, the human–AI agreement analysis and the response-excluded evaluation provide complementary evidence supporting the semantic validity of our diagnostic pipeline and the functional patterns identified in Fig. 2, while clarifying the sensitivity of individual diagnostic states to the available context.


4. Clarification of Quantitative Claims (W2, Q1)

We sincerely apologize for the inaccurate quantitative claims and thank the reviewer vvjJ for identifying this inconsistency. The original 78-pp figure was based on an earlier experimental version. After correcting an experimental fairness issue and updating Table 1, we inadvertently overlooked the corresponding claims in the abstract and introduction. We take responsibility for this oversight.

Following the reviewer's suggested calculation protocol, the updated results show approximately 48 percentage points of absolute ASR reduction and 78% relative reduction, averaged over five primary metrics and four backbones. Including ASR-a yields a similar absolute reduction of approximately 49 pp. We will correct the corresponding claims and clearly distinguish absolute from relative reductions in the revised manuscript.

Regarding the compromised-state reduction, we evaluate 100 WebShop cases using Gemma-3-12B. The compromised-state ratio, defined as 1−P(S0)1-P(S0), decreases from approximately 55% to 17%, corresponding to approximately 69% relative reduction. We will clarify this calculation and its experimental scope in the revised manuscript.


5. Interpretation of Activation-Steering Results (W5)

We thank the reviewer vvjJ for raising this important point. The motivation of our steering experiments is to examine whether NLA-derived internal signals are behaviorally relevant to the agent's final outputs. This provides an important link in our framework: internal diagnoses extracted by NLA are used as feedback to iteratively refine external guardrails. By intervening on these internal representations and observing the resulting behavioral changes, §4 provides evidence supporting this connection.

We acknowledge that the original wording “confirms” could be more precisely qualified. We have revised §4.1 and §4.3 to use more qualified intervention-based evidence language. This better reflects the scope of our findings without changing the central interpretation: the diagnosed internal signals are behaviorally relevant and can inform offline harness refinement, although the experiments do not establish a fully isolated causal mechanism.


6. Utility of Benign Memories (Q5)

We thank the reviewer vvjJ for raising this important concern. To directly evaluate whether our final policy interferes with legitimate memory utilization, we conducted an additional experiment on QA with entirely clean memories, comparing task accuracy with and without the defense policy.

BackboneNo DefenseOursChange
Gemma-37675-1 pp
DeepSeek-V4-Flash90900 pp
Gemini-3.8-Flash94940 pp
GLM-5.3-Flash8587+2 pp

The results show that task accuracy remains nearly unchanged across all four backbones, with variations ranging from -1 to +2 percentage points. These findings suggest that our final defense policy does not materially compromise the utility of legitimate memories in the evaluated setting. We have included this additional evaluation in the revised manuscript.


Again, we sincerely thank the reviewer vvjJ for the thoughtful feedback and for recognizing the novelty and practical value of our work. In response, we conducted a controlled behavior-only ablation, added NLA case studies and diagnostic validation, corrected the quantitative claims, clarified the steering results, and evaluated benign-memory utility. These additional experiments and revisions further support our central contribution of leveraging internal-state diagnoses to guide iterative external guardrail refinement. We hope these efforts address the reviewer vvjJ’s concerns and encourage a more favorable reassessment of our work.

Comments on the paper

4 comments
AuthorsOpenReview ↗

Official Comment for Reviewer xgYs (PART I)

We sincerely thank the reviewer xgYs for the careful assessment of our work. We particularly appreciate the recognition of our CAO-based diagnostic framework, the joint analysis of successful and failed attacks, and the effort to connect internal-state interventions with a practical, text-only defense that transfers across agent settings and model backbones.

We understand that the reviewer xgYs’s remaining concerns mainly center on the following aspects:

  1. The additional value of internal-state feedback: Whether internal diagnoses provide measurable benefits beyond behavior-only refinement under matched optimization budgets. (W1, Q1)
  2. The policy refinement and cross-setting transfer protocol: Whether the defense policy is independently optimized for each application or transferred from the WebShop setting, and how the refinement procedure is conducted. (W2, Q2)
  3. The interpretation of activation-steering results: Whether the intervention experiments sufficiently support the behavioral relevance of NLA-derived representations, given the combined modification of original and reconstructed activations. (W3)
  4. The strength of defense baselines: Whether our method is adequately compared with equally optimized prompt-based defenses and stronger agent-specific memory defenses. (W4)
  5. The utility of benign memories: Whether the defense preserves legitimate memory utility and how it handles genuine user preferences that are not explicitly stated in the current request. (W5, Q3)

We address these concerns in detail below.


1. Additional Value of Internal-State Feedback (W1, Q1)

We thank the reviewer xgYs for emphasizing the need to isolate the contribution of internal-state feedback. To address this concern, we conducted a controlled comparison between behavior-only refinement and our full method, keeping the optimizer, initial policy, training/validation samples, optimization budget, and policy selection criteria identical. The only difference is whether NLA-derived internal diagnoses are provided to the optimizer.

The results show that internal-state feedback improves defense performance in four of the five evaluated settings. The following table reports the mean results across four backbones.

BenchmarkMetricBehavior-onlyOursASR Reduction (pp)
WebShopASR ↓3.801.202.61
QAASR ↓8.360.487.88
MIMIC-IIIASR ↓32.794.4928.30
EHRASR-t ↓41.5528.4513.10
StrategyQAASR-t ↓34.9037.95—

Notably, the gains extend beyond WebShop, where the policy is optimized, to other evaluation settings, particularly MIMIC-III and EHR. On QA, our method also improves average task accuracy from 79.31% to 88.15% (+8.84 pp). We acknowledge an exception on StrategyQA, where behavior-only refinement achieves a lower average ASR-t. This difference is largely driven by GLM-5.3-Flash: behavior-only refinement already achieves 18.12% ASR-t and 81.88% ACC, compared with 42.36% ASR-t and 57.64% ACC for Ours. Although our method achieves a lower ASR-r on this backbone (25.86% vs. 37.18%), this does not translate into better overall attack resistance.

Overall, these results demonstrate that internal-state feedback provides measurable benefits beyond behavior-only optimization, particularly in improving defense generalization across different agent settings and backbones. While the gains are task-dependent, they support the complementary value of incorporating structured internal diagnoses into defense refinement.


2. Policy Refinement and Cross-Setting Transfer Protocol (W2, Q2)

We thank the reviewer xgYs for identifying this ambiguity and apologize for the unclear description in §5.2. To clarify, we refine only one defense policy on WebShop and directly transfer it to all five agent settings and four model backbones without further optimization. Specifically, GPT-5.6-Sol serves as the policy optimizer, while Gemma-3-12B generates responses during refinement. The refinement uses 100 WebShop training cases and a separate, disjoint validation set of 100 cases. No data from the other four settings is used for policy optimization or selection.

Therefore, our evaluation examines cross-setting and cross-backbone transfer of a single refined policy, rather than independently optimized policies for individual applications. We acknowledge that the original statement in §5.2 incorrectly suggested otherwise. We have corrected this description and clarified the transfer protocol in the revised manuscript.


AuthorsOpenReview ↗

Official Comment for Reviewer xgYs (PART II)

3. Interpretation of Activation-Steering Results (W3)

We thank the reviewer xgYs for pointing out this important limitation of our intervention design. We agree that our continuous steering formulation, h~t(L)=(1−λ)ht(L)+λΔvc,\tilde{h}_{t}^{(L)}=(1-\lambda)h_{t}^{(L)}+\lambda \Delta v_c,, simultaneously attenuates the original activation and introduces an NLA-derived difference direction. Therefore, the observed behavioral changes cannot be uniquely attributed to the isolated causal effect of a specific semantic component.

Our intention, however, is to examine whether directions derived from NLA-identified constraint-, objective-, and action-level semantics have measurable behavioral relevance, rather than to establish a definitive causal explanation of the underlying computation. The strength-dependent behavioral changes in Fig. 4, together with the additional single- and multi-position intervention results in Appendix B.2–B.3, provide complementary evidence for this relationship. In particular, broader interventions can increase attack failure, but stronger modifications also compromise output executability, revealing a clear intervention–validity trade-off.

We have accordingly revised the relevant statements in §4, replacing claims of causal validation with more qualified descriptions of behavioral relevance and functional effects. This clarification better reflects the evidence while preserving the original motivation of using activation interventions to examine the behavioral significance of NLA-derived internal semantics.


4. Strength of Defense Baselines (W4)

We appreciate the reviewer xgYs’s suggestion to strengthen our defense comparisons. Existing memory defenses mainly operate at different stages: memory ingestion and storage protection [1], retrieval-stage filtering and detection [2, 3], runtime reasoning validation [4], and post-hoc auditing and recovery [5]. These approaches provide complementary protection, but our work considers a different yet important setting: poisoned memories have already been retrieved and incorporated into the agent's active context (§2.2). Since evolving attacks may bypass upstream defenses, we investigate whether their downstream influence can be mitigated at the model level.

Regarding the baselines, we acknowledge that Ignore, Escape, and Sandwich are simple prompt-level heuristics. We therefore provide two additional comparisons: the matched-budget behavior-only refinement experiment (W1), which isolates the contribution of internal feedback, and an evaluation against A-MemGuard [4], a dedicated agent-memory defense framework, on QA and StrategyQA using DeepSeek-V4-Flash.

MethodStrategyQA ASR-r ↓ASR-a ↓ASR-t ↓QA ASR ↓QA ACC ↑
Ignore19.5245.8144.103.8794.84
Escape26.4295.3979.4829.0869.50
Sandwich36.6885.6266.383.8594.62
A-MemGuard36.4961.5838.3612.3586.07
Ours19.9537.0445.410.0098.29

Our method achieves the best results on QA, while A-MemGuard performs better on StrategyQA ASR-t. We retain this heterogeneous result to provide a balanced comparison. We will incorporate these additional experiments and clarify the scope of existing defenses in the revised manuscript.

[1] "SMSR: Certified Defence Against Runtime Memory Poisoning in Persistent LLM Agent Systems," arXiv, 2026.

[2] "MIND: Lightweight and Effective Memory Injection Defense for LLM Agents via Intent-Aware Information Bottleneck," arXiv, 2026.

[3] "MEMSAD: Gradient-Coupled Anomaly Detection for Memory Poisoning in Retrieval-Augmented Agents," arXiv, 2026.

[4] "A-MemGuard: A Proactive Defense Framework for LLM-Based Agent Memory," ICML, 2026.

[5] "MemAudit: Post-hoc Auditing of Poisoned Agent Memory via Causal Attribution and Structural Anomaly Detection," arXiv, 2026.


5. Utility of Benign Memories and Legitimate User Preferences (W5, Q3)

We thank the reviewer xgYs for raising this important concern. Our defense is designed to prevent retrieved attack examples from redefining the current task's objectives, constraints, and action plans, rather than indiscriminately discarding all historical information. We acknowledge, however, that the current instruction to use the current request as the source of objectives and requirements may be overly broad when applied to legitimate user preferences that are not explicitly restated. Our current approach does not provide a general provenance-aware mechanism for distinguishing such preferences from malicious memory, and we have clarified this limitation in the revised manuscript.

To examine whether the defense affects normal task utility, we conducted an additional QA evaluation with clean memories across four backbones:

AuthorsOpenReview ↗

Official Comment for Reviewer xgYs (PART III)

BackboneNo Defense ACCOurs ACCChange
Gemma-376%75%−1 pp
DeepSeek-V4-Flash90%90%0 pp
Gemini-3.8-Flash94%94%0 pp
GLM-5.3-Flash85%87%+2 pp

The results show no substantial degradation in task accuracy when clean memories are present. Nevertheless, we acknowledge that this evaluation does not fully establish the preservation of personalized preferences or reusable procedures that are absent from the current request. We have therefore narrowed our utility claim and clarified that distinguishing malicious memory from legitimate long-term user preferences remains an important direction for further investigation.


In response to the reviewer xgYs’s concerns, we added a matched-budget behavior-only ablation, clarified cross-setting transfer, expanded defense comparisons, supplemented activation-intervention analyses, and evaluated benign-memory utility. These additional analyses further support our central contribution of leveraging internal-state diagnoses to improve practical memory-poisoning defenses for LLM agents. We sincerely thank the reviewer xgYs for the constructive feedback and hope these clarifications provide a more complete picture of our work and its contributions.

AuthorsOpenReview ↗

To the Area Chair and Reviewers: Rebuttal Updates

Dear Area Chair and Reviewers,

First, we would like to express our sincere gratitude to the Area Chair for coordinating the review process and to all reviewers for the time, care, and thoughtful consideration devoted to our manuscript. We deeply appreciate the detailed feedback and the opportunity to further improve our work.

The central idea of our paper is to incorporate white-box internal states of LLMs into the optimization of agent harnesses, using internal diagnoses to guide practical defenses against memory-poisoning attacks. We are grateful that the reviewers recognized several important aspects of this research direction:

  • vvjJ appreciated our focus on post-retrieval security, the unified Constraint–Objective–Action (CAO) framework, and the practical value of offline internal-state-guided refinement with transferability to black-box models.
  • DMeC recognized the value of identifying which components of agent decisions are affected by memory poisoning and translating these insights into practical defense mechanisms.
  • xgYs appreciated our analysis of both successful and failed attacks, the distinction between compromised decision components, and the connection between internal-state interventions and a deployable text-based defense.

The reviewers' concerns primarily focused on the strength of defense baselines, the additional value of internal-state feedback, the reliability of internal diagnoses, and the rigor of our interpretations and conclusions. We understand these concerns as primarily relating to the supporting evidence and the precision of our claims, rather than the central research idea itself. We take these concerns seriously and appreciate the opportunity to address them.

In response, we have tried our best to address these concerns through additional experiments and revisions. We strengthened our baseline comparisons with A-MemGuard, examining both defense effectiveness and inference efficiency. We conducted a matched-budget behavior-only ablation to demonstrate the additional value of internal-state feedback, and added human–AI agreement and response-excluded analyses to examine diagnostic reliability. We also revisited our activation interventions, clarified the limits of their causal interpretation, and corrected the quantitative claims. All these additional analyses and clarifications have been incorporated into the revised manuscript to better reflect the evidence and its limitations.

We sincerely regret that there has been limited opportunity for further discussion of these additional findings during the rebuttal period. Nevertheless, we remain deeply grateful for the reviewers' careful evaluations and constructive suggestions. We hope that our additional efforts help convey the significance of the research direction we have explored: connecting white-box internal-state analysis with the optimization of practical, deployable agent defenses. We would be grateful if the Area Chair and reviewers could take these additional findings and clarifications into consideration when assessing the overall contributions of our work.

Thank you once again for your time, patience, and consideration.

Sincerely,
The Authors of Submission 25946

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.