- Limited novelty and insufficient engagement with related work
My primary concern is that the central argument of this paper, that information being decodable from a model's representations does not necessarily mean that it influences the model's behavior, is already well established in the literature.
For example, Amnesic Probing [1] and the survey on probing classifiers [2] have discussed the limitations of probing in establishing whether a model actually uses the information being decoded. Similarly, [3] has examined the differences between causal localization and effective model editing. Even the foundational work introducing causal tracing [4] is not discussed.
More importantly, recent studies have already investigated closely related questions. For example, [5] reports a 53-percentage-point knowledge-action gap, while [6] explicitly examines the distinction between recognizing harmful content and acting on that recognition.
In the context of prompt injection, [7] investigates role confusion and shows that internal role representations can predict injection success before generation. More directly, [8] investigates a recognition-action gap in indirect prompt injection across multiple models and introduces a probe-based defense evaluated on AgentDojo.
Although the authors cite some of these recent studies, particularly [5]–[8], I do not think they adequately explain how the present work differs from them. The discussion of [8] is especially limited given how closely related its research questions and experimental setting are.
Several other relevant studies are also missing or insufficiently discussed, including instruction-data separation [9], attention-based prompt injection detection [10], representation-based detection of hijacked agent actions [11], activation steering against tool poisoning [12], and prior work on representation directions for steering model behavior [13].
Considering the existing literature, it is difficult to identify a sufficiently strong research gap. In my assessment, the paper largely revisits an established observation through another set of probing and intervention experiments, without clearly demonstrating what fundamentally new understanding is gained.
- Experimental limitations and insufficient evidence for the claimed contribution
The experiments do not fully convince me that the paper delivers on its promise of causally tracing indirect prompt injection.
For example, the main activation-patching experiment changes a weather request into a calculator request. As the authors themselves acknowledge in Section 5.2, this is not a role-only donor-recipient contrast. Consequently, it is difficult to determine how much of the observed effect is specifically attributable to instruction-role representations rather than other differences between the prompts.
The distinction becomes even more important in the AgentDojo experiments. In Table 11, the controlled-role direction barely changes the attack success rate, from 34.6% to 33.8%. A more substantial reduction is obtained only when using a direction estimated from successful and unsuccessful attack trajectories. This raises an important question about whether the effective intervention is actually capturing source-role information or simply attack-outcome-related features.
I also find the layer-wise results somewhat limited. Role information is already decodable at earlier layers, while stronger intervention effects appear at later layers. However, given the extensive literature on the distinction between representation and behavioral influence [1–6], I am not convinced that this observation alone constitutes a sufficiently novel contribution.
Another concern is the choice of models. The experiments primarily use model families released in 2024. By the time of the conference, these models will be roughly 2.5 years old, and several newer generations will already be available. This matters because newer models have undergone different and often more extensive post-training. The authors themselves acknowledge that post-training can affect their results. Without experiments on more recent models, it is difficult to assess whether the findings generalize to current LLMs.
- Poor readability, presentation, and reproducibility
The paper is difficult to read and lacks a coherent narrative. Although the individual sentences are generally grammatically correct, the overall presentation is not easy to follow. The authors move between several experimental settings, sometimes with different datasets, intervention directions, and evaluation protocols. I frequently found myself going back to earlier sections to understand what was actually being compared.
The purpose of a research paper is to communicate research ideas and experimental findings clearly. Even when LLMs are used to polish the writing, the responsibility for ensuring that the final manuscript is readable and understandable lies with the authors.
The extensive appendices do not help much with this issue. There are 23 appendices (A–W), but several are extremely brief, and the connection between the supplementary experiments and the main claims is often unclear. In my opinion, the paper would benefit much more from a coherent explanation of its main experiments than from such a large collection of supplementary analyses.
I also have serious concerns about reproducibility. Despite the reproducibility statement, the released supplementary materials do not appear to contain the complete experimental pipeline required to reproduce the probes, activation extraction, intervention directions, and AgentDojo experiments.
Furthermore, I found apparent inconsistencies in the supplementary records, including target-token indices beyond the recorded sequence lengths in EXP-07 and memory-intervention results that differ from those reported in the paper. These discrepancies need to be explained.
Overall, despite the large number of experiments, I am not convinced that the paper provides a sufficiently novel, clearly presented, and reproducible contribution.
Dear Reviewer QFUL,
Thank you again for your time and constructive feedback.
With 1 day remaining in the author–reviewer discussion period, we would greatly appreciate it if you could take a moment to review our rebuttal and revised manuscript. We have addressed your concerns through additional analyses, experiments, and clarifications.
Thank you for your time and consideration.
Best regards,
The Authors