- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper investigates whether attribution graphs can monitor covert sabotage by LLM agents more effectively than text- or activation-based approaches. The authors compare text monitors, a linear activation probe, and an attribution-graph-based structural monitor across multiple trigger-gated LoRA model organisms. The results show that the structural fingerprints do contain sabotage-related signals, but after controlling for several confounding factors, they do not demonstrate a consistent advantage over the activation probe. The paper also finds no clear positive evidence for cross-mechanism generalization or robustness to evasion. Overall, the main contribution of the paper is a controlled evaluation of attribution-graph monitoring and a systematic analysis of confounding factors that may make it appear stronger than simpler activation-based monitoring.
Critical strengths
-
The problem itself is important. The paper focuses on whether attribution-graph monitoring provides meaningful advantages over text- or activation-based monitoring, which is a relevant question for the safety monitoring of LLM agents.
-
The paper mainly reports negative findings and frames its contribution around a controlled comparison protocol and confound analysis. I appreciate this positioning, as the authors actively examine several potential confounding factors and are transparent about the largely negative nature of their conclusions.
-
The experimental coverage is reasonably broad, and the limitations of the study are stated relatively clearly.
Critical weaknesses
1. I do not think the current experimental setup is sufficient to clearly support either a positive or a negative conclusion about attribution-graph monitoring. If the paper aims to study whether attribution-graph monitoring can better monitor undesirable computational processes, the current experimental setup does not provide a clean comparison. As noted in Appendix A, because these backdoors fire almost deterministically, the current design does not allow the monitors to distinguish between detecting the trigger and detecting the sabotage computation itself. In this setting, the activation probe and TF-IDF are already close to ceiling, and they largely degenerate into trigger detectors. This setup therefore makes it easy to expose the disadvantages of attribution-graph monitoring, while leaving little room to reveal potential advantages in more complex backdoor settings.
2. The paper lacks a direct comparison with highly related prior work. CRV (Zhao et al., Verifying Chain-of-Thought Reasoning via Its Computational Graph) follows a highly similar high-level pipeline. It uses transcoders to construct attribution graphs, extracts structural fingerprints from the graphs, and trains a supervised classifier for detection. Although the task settings are different, CRV reports clearly positive results for structural graph features compared with activation-based baselines in its setting. I believe the paper should include a CRV-style baseline for comparison. However, the paper lacks discussion of this work and does not cite it in the references.
3. There are important inconsistencies between the reported results and the stated experimental protocol. The o3 to o1 result does not match the protocol described by the authors. Appendix B.1 explicitly states that cross-organism transfer is evaluated by training on all data from organism A and testing on all data from organism B. However, Table 9 reports an AUROC of 0.983, whereas the paper states that the corresponding result should be 0.003 when trained on the full dataset. Moreover, 0.983 is neither equal to the full-data result of 0.003 nor within the reported 0.85–0.96 range obtained from subset training. Therefore, the reported number does not appear to match either setting unless Table 9 used an undisclosed experimental setup. In addition, TF-IDF produces different results on the two base models, even though TF-IDF itself should not depend on the model. Appendix E states that the organisms, triggers, and templates are identical across the two experiments. Since TF-IDF only reads the text, it should in principle be independent of the base model and therefore should produce the same results.
Questions for authors
See weaknesses.
3 comments
I thank the authors for their response and additional analyses. While these address some of my concerns, I remain concerned about whether the experimental setup adequately supports the main conclusions. I am therefore not sufficiently convinced to recommend acceptance and will maintain my current rating.
We thank the reviewer for reading our response. We understand the remaining concern about whether the experimental setup supports our conclusions.
Our main conclusion is scoped: on trigger-gated model organisms (Hubinger et al., 2024), attribution-graph monitoring shows no detection advantage over a linear probe wherever our confound controls permit a clean comparison. We do not claim that attribution-graph monitoring cannot help in other settings.
We believe the additional results support this conclusion. On o4_destruct, both monitors fall to chance without the LoRA adapter and reach 0.999 with it. So their signal comes from the backdoor computation, not from the trigger alone. Even here, the probe and the structural monitor tie. On exfiltration, neither monitor separates the trigger from the sabotage action, so we do not use exfiltration as evidence on this point, and we will state this in the paper.
We will also add to the Limitations that settings without a detectable trigger, such as CRV's reasoning errors, may favor attribution structure. As stated in our Conclusion, establishing an advantage for attribution-graph monitoring in such settings will require confound-clean causal tests.
We hope these answers address your concerns, and we would be grateful for any further feedback.
We thank the reviewer for the valuable feedback. We ran two trigger controls, corrected the decision-point prompts, and computed a CRV-style baseline on our fingerprints.
(1) Trigger confound. We agree. Because the backdoors fire almost deterministically on the trigger, our setting cannot fully separate trigger detection from detection of the sabotage computation, and it may favor simple probes. With the trigger inserted into benign queries where the organism still writes the benign argument (18 exfiltration episodes), the probe flags 18/18 at 5% FPR and the structural monitor 10/18, or 18/18 under the common-target control. With Gemma-3-4B and no LoRA adapter, the organism-trained probe and structural monitor score 0.848 and 0.530 on o1_exfil, 0.855 and 0.816 on held_out, and 0.494 and 0.529 on o4_destruct (0.999-1.000 with the adapter). So on exfiltration both monitors flag the trigger alone. On o4_destruct both fall to chance without the adapter, so they read the backdoor computation; even there, the probe and the structural monitor tie (0.999). The setup therefore does not hide an advantage of attribution structure. Trigger-gated organisms are the standard model-organism setting (Hubinger et al., 2024), where simple probes are known to catch sleeper agents (MacDiarmid et al., 2024). We will scope our conclusion accordingly: on trigger-gated organisms, attribution-graph monitoring shows no advantage over a linear probe, and on exfiltration neither monitor separates the trigger from the sabotage action. We will add to the Limitations that settings without a detectable trigger, such as CRV's reasoning errors, may favor attribution structure.
For o2, o3 and o4, the decision-point prompt ends before the opening quote of the argument value, so its last token is a standalone space never seen in training. This causes the degenerate common-target anchors and the 0/20 free-generation result. With correct prompts, o2_perm and o3_drift write the sabotage value with or without the trigger, so we remove their decision-point results. o4_destruct writes "delete" only with the trigger; the probe and the structural monitor both reach 0.999 (paper: 0.605 structural), and the structural monitor stays at 0.999 under the common-target control. The exfiltration results stand, subject to (1): both monitors reach 1.00, the structural signal survives the common-target control (1.000), and the aimed attack reduces it from 1.000 to 0.547 on Qwen (sabotage kept in 19/20 episodes) and from 0.795 to 0.663 on Gemma (20/20). Our pipeline reproduces the structural cells of Table 1.
(2) CRV. We will cite and discuss CRV [1]. Three differences separate the two studies. First, CRV detects reasoning errors without a trigger, a setting that may suit structure better than ours. Second, CRV uses graph-level summaries (node and edge counts, influence statistics, topology); we use a per-feature influence vector with a linear classifier. Third, CRV's best probe reaches 53.6-56.0 AUROC against 70.2-92.5 for CRV, while our probe is the best of 34 layers at the decision token (0.972 in distribution), so CRV's margin may partly reflect a weaker baseline. CRV also reports limited cross-task transfer (57.0 and 55.1 AUROC), consistent with our cross-mechanism result. We computed CRV-style summaries (global and per-layer feature counts and influence statistics) from our saved fingerprints and trained gradient boosting; topological features are excluded because the full graphs were not saved. Pooled over o1_exfil, o2_perm and o3_drift with the Table 1 splits (o2_perm and o3_drift with the miscut prompts), it reaches 0.860 AUROC, against 0.798 for our structural monitor and 0.959 for the probe. Graph summaries beat our per-feature vector here, but even CRV's feature design stays well below the probe (0.860 vs. 0.959), which supports our conclusion.
(3) Inconsistencies. You are right about o3 to o1: the 0.983 in Table 9 is an error; under App. B.1 the value is 0.003, and we will correct it. TF-IDF is model-independent, but its input text differs across the two models. The organisms, triggers, decoys and templates are the same, but the Qwen corpus was built separately (200 sampled base episodes per organism instead of 300), rendered in Qwen's chat template, and truncated with Qwen's tokenizer. The held_out difference comes from a decoy word that overlaps the shared tag pool and survives more often in the Qwen corpus (App. E). We will change "identical" in App. E to refer to the definitions, not the corpora. The data files in the code supplement come from a corpus version without decoys; we will replace them with the files used in the paper.
We hope these results clarify the scope of our conclusions, and we would be glad to discuss any remaining question.
References
[1] Z. Zhao et al. 2025. Verifying Chain-of-Thought Reasoning via Its Computational Graph. arXiv:2510.09312.