acceptodds
Withdrawn from ICLR 2027

GRAD: Graph-Structured Antibody Defense for LLM Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Memory poisoning and decomposition attacks distribute malicious evidence across the operations of large language model agents. Existing request-level defenses can miss such attacks because they assess individual inputs without connecting earlier information to later actions. Moreover, detectors must adapt as task contexts change, while updates to shared parameters can impair previously learned detection capabilities. To address these challenges, we propose GRAD, which combines graph-based evidence aggregation across operations with an extensible collection of detection experts. Its Semantic–Structural Understanding Module (SSUM) jointly encodes operation text, event attributes, and typed candidate relations in an execution graph to aggregate distributed attack evidence into a graph-level risk score. Its Continual Antibody Adaptation (CAA) adds experts on frozen encoder snapshots and routes inputs by observable graph signatures, acquiring and reusing detection capabilities as tasks arrive. Across Decomp, MCP, and MRT, GRAD achieves AUROCs of 97.19%, 77.59%, and 86.60%, respectively, outperforming Sequential Monitor and locally adapted Agent-Sentry and NeuroTaint. CAA improves final accuracy by 13.08 percentage points over Replay on the cross-dataset task stream. The gain remains 9.05 points with equal family/base-case weights within each task. Its compact drop-in gate performs local model inference in milliseconds without monitoring API calls.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Agents, Agent Security, Prompt Injection, Memory Poisoning, Graph-Structured Antibodies, Continual Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Jig3OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GRAD, a graph-based defense framework for detecting memory poisoning and decomposition attacks in LLM agents. The method contains two main components. First, the Semantic–Structural Understanding Module (SSUM) represents agent execution histories as typed graphs whose nodes and edges encode operation text, attributes, and candidate execution relations; a graph neural network aggregates this information into a graph-level risk score. Second, Continual Antibody Adaptation (CAA) maintains a growing collection of frozen detector snapshots with task-specific adapters and heads, routing new execution graphs to an expert using nearest-centroid matching over graph signatures.

Critical strengths
  1. Important security problem. The paper targets decomposition and memory-poisoning attacks, where malicious evidence may be distributed across multiple agent operations. This is a realistic weakness of request-level safety classifiers and motivates trajectory-level modeling.

  2. Graph representation is well motivated. Encoding both operation semantics and relationships between operations is a natural approach for attacks whose harmful meaning emerges only across multiple steps. The use of typed edges and graph message passing is conceptually appropriate for this problem.

  3. The matched Decomp comparison is stronger than simply comparing against heterogeneous existing systems. Under matched horizons and text budgets, GRAD improves AUROC over both the temporal and text-only controls on the original Decomp split.

Critical weaknesses
  1. The strongest graph advantage does not reproduce on independently generated Decomp cases. On the original matched Decomp split, GRAD outperforms the temporal baseline by roughly 5 AUROC points. However, on 47 independently generated base cases, the frozen GRAD–Temporal difference becomes negative (-0.060) with a confidence interval excluding zero at the upper endpoint. This substantially weakens the central claim that graph structure itself consistently improves detection of decomposition attacks. I think this result deserves substantially more emphasis and analysis in the main discussion.

  2. The joint graph+text architecture is not consistently superior across datasets. On MCP, the text-only model achieves 79.89 AUROC compared with GRAD's 77.59, while on MRT the graph-only model is essentially tied with the full model (86.07 vs. 86.60). Thus, the benefit of jointly modeling semantic and structural information appears highly task-dependent.

  3. Continual-learning gains are weaker under held-out-family evaluation. CAA substantially outperforms replay on the cross-dataset stream, but this setting contains recurring attack families. In the family-isolated stream, the apparent advantage reverses under equal family weighting: snapshot experts obtain 47.5% accuracy versus 53.5% for replay. This suggests that CAA primarily improves retention and task-specific specialization rather than generalization to novel attack mechanisms.

Overall this is a good paper. My main concern is the robustness of the central empirical claims. The graph advantage on Decomp does not reproduce on independently generated base cases, the benefit of joint graph/text modeling is inconsistent across MCP and MRT, and the apparent continual-learning advantage weakens or reverses under the family-isolated evaluation. These results suggest that GRAD is effective in specific task distributions but that its broader generalization claims are not yet fully established.

Questions for authors
  1. How do the authors interpret the failure of the GRAD advantage to reproduce on the 47 independently generated Decomp base cases? Is there evidence that the original split contains structural or textual regularities that the graph model exploits?

  2. What is the primary benefit of the graph representation if text-only is stronger on MCP and graph-only is nearly tied with GRAD on MRT? Can the authors characterize which attack families specifically require joint semantic-structural modeling?

Reviewer_QDjrOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes GRAD, a graph-based defense framework for detecting attacks against LLM agents. It consists of two components: SSUM, which aggregates semantic and structural information from execution graphs, and CAA, which supports continual adaptation through frozen expert snapshots and routing. Experiments on Decomp, MCP, and MRT report improvements in attack detection and continual learning. However, the claimed advantage over temporal modeling does not reproduce on independently generated cases.

Critical strengths
  1. Modeling dependencies across agent operations is a potentially useful direction for detecting attacks distributed across multiple steps.
  2. The proposed CAA mechanism shows improvements over Replay in cross-dataset accuracy and forgetting.
  3. The paper includes encoder ablations, matched decision-time comparisons, and closed-loop evaluations. Reporting the negative independent replication result is also appreciated.
Critical weaknesses
  1. Failure to reproduce a central result. GRAD outperforms the Temporal baseline by 5.3 AUROC points on the original Decomp split, but underperforms it by 6.0 points on 47 independently generated base cases (Line 343). This directly undermines the claimed advantage of graph-based modeling over temporal modeling.
  2. Mismatch between motivation and evaluation. Memory poisoning is a central motivation throughout the abstract and introduction, yet the three main benchmarks focus on decomposition attacks (Decomp) and tool-use risks (MCP and MRT). Memory-poisoning evaluations are largely relegated to appendix diagnostics, leaving the central motivation insufficiently validated.
  3. Questionable baseline comparisons. Agent-Sentry and NeuroTaint are evaluated using locally adapted components rather than their complete defenses. In particular, NeuroTaint achieves exactly 50% AUROC on MCP and MRT with all-negative predictions, suggesting that the adaptation provides little meaningful detection capability. Consequently, outperforming these baselines offers limited evidence of superiority.
  4. Missing strong baselines. The evaluation lacks natural and competitive alternatives, particularly fine-tuned LLM-based classifiers (e.g., Llama Guard or WildGuard) and an LLM-as-judge with access to the full execution trajectory under matched conditions.
  5. Limited closed-loop evaluation. Each configuration includes only 20 attack runs and 20 benign runs, making the reported attack success rates and utility comparisons sensitive to a small number of outcomes. Larger-scale evaluations are needed to support reliable conclusions.
  6. Heavy reliance on appendices. Many essential experimental details, including dataset construction, baseline adaptations, evaluation protocols, and statistical analyses, are deferred to Appendices A–P. This makes it difficult to verify and interpret key results from the main paper alone.
  7. Unclear organization and presentation. The paper frequently switches between different datasets, observation settings, and evaluation protocols without a sufficiently clear narrative. As a result, the experimental logic is difficult to follow, and the evidence supporting each contribution is not always easy to identify.
Questions for authors
  1. What explains the reversal of GRAD's advantage over Temporal on independently generated cases? Can the authors provide further independent evaluations to establish whether the graph-based improvement is reproducible?
  2. Given the emphasis on memory poisoning, why are dedicated memory-poisoning benchmarks not included in the main evaluation?
  3. Can the authors compare against fine-tuned LLM safety classifiers and full-trajectory LLM-as-judge baselines under matched settings?
  4. Can the closed-loop evaluation be expanded to include more attack instances, backbones, and confidence intervals?
  5. How should the CAA generalization results be interpreted when equal-family weighting reverses its advantage over Replay on the family-isolated stream?
Reviewer_1UGHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

GRAD detects attacks on LLM agents by representing observed operations and their candidate dependencies as an execution graph. Its Semantic–Structural Understanding Module (SSUM) combines operation text, event attributes, and typed relations to predict graph-level risk; Continual Antibody Adaptation (CAA) adds task-specific adapters and prediction heads on frozen encoder snapshots, then routes graphs to experts using graph signatures. The paper evaluates detection on Decomp, MCP, and MRT, reporting AUROCs of 97.19%, 77.59%, and 86.60%, respectively. On a cross-dataset continual-learning stream, CAA reaches 77.86% accuracy versus 64.78% for Replay and reports lower forgetting. The paper also includes a matched decision-time comparison and a small closed-loop evaluation of blocking and replanning.

Critical strengths

The paper addresses a consequential detection gap. By assessing operation content together with relations across an execution, GRAD targets attacks whose evidence is distributed across steps, rather than treating each request in isolation. In the matched Decomp decision-time evaluation, it reports 97.0% AUROC, compared with 91.7% for the temporal detector and 86.4% for the text-only detector. This is a clear and practically relevant result for the paper’s central use case.

Critical weaknesses

1-On the original decision-time test set, GRAD outperforms the temporal detector; however, on 47 independently generated base cases, the reported frozen GRAD–Temporal AUROC difference is −0.060, with a 95% confidence interval of [−0.115, −0.0001]. This result points in the opposite direction and makes the robustness of the paper’s strongest detection claim uncertain. The discrepancy matters because the main evidence for the benefit of joint graph-based modeling comes from Decomp.

2-In the family-isolated stream, snapshot experts outperform Replay under the sample-level endpoint (51.11% versus 39.21%), but fall below Replay under equal family weighting (47.5% versus 53.5%); the reported paired interval for that difference is entirely negative. The paper also reports a negative family-equal difference on MCP, albeit with an interval that includes zero. Thus, the continual-learning gains appear strongest for the evaluated sample/task distribution and do not yet establish broadly consistent transfer across attack families. The small closed-loop evaluation further limits the strength of the deployment-level evidence.

Questions for authors

1-Could the authors explain the independent-generation Decomp evaluation in enough detail to reconcile its result with the original matched decision-time comparison? In particular, how were the 47 base cases generated and selected, and were the model, threshold, and evaluation protocol fixed before these cases were produced? How should the negative GRAD–Temporal difference affect the paper’s claim about the robustness of GRAD’s Decomp advantage?

2-Could the authors explain why the family-isolated stream’s conclusion reverses under equal family weighting, and identify which attack families or task components drive that reversal? What scope of held-out-family transfer do the authors believe is supported by these results?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.