- Rating
- 4
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper introduces ResearchArena, which evaluates agents that carry out legitimate AI R&D tasks while simultaneously performing covert sabotage across four task domains. It finds that sabotage embedded in training data and model behavior is particularly difficult to detect; access to final artifacts can improve monitoring, whereas reasoning summaries can sometimes mislead monitors. Its primary contribution is an evaluation framework and empirical findings, rather than a new attack or defense algorithm.
Critical strengths
-
A valuable evaluation setting. Combining legitimate task completion with covert sabotage, while allowing monitors to inspect and execute the resulting artifacts, captures trust concerns in automated R&D more directly than evaluating task performance or reviewing transcripts alone.
-
A clear experimental framework. The paper evaluates main-task and attack success separately and compares four monitoring conditions defined by artifact access × CoT access. Coverage across multiple models and tasks helps distinguish attack failures from monitoring failures.
Critical weaknesses
-
Limited novelty and scope of generalization. The contribution primarily integrates existing attack and monitoring ideas into a systematic evaluation. Many attacks are explicitly specified in the instructions, so the results mainly demonstrate compliance with harmful instructions framed as legitimate requirements, rather than autonomous attack planning or strategic evasion.
-
Key explanatory factors are not sufficiently isolated. Artifact access provides both additional files and execution capabilities, making their individual contributions unclear. Data-based and code-based attacks also occur in different task domains, so the detection gap cannot be attributed solely to how sabotage is embedded.
Questions for authors
- Does the difficulty of detecting data-based attacks persist after controlling for task differences? Could the authors compare data-based and code-based attacks with similar harmful effects within the same R&D task?
- Are the main conclusions robust to stricter evaluation? Could the authors report per-model performance effect sizes and confidence intervals, and compare monitors at a common false-positive rate after reassessing the baseline labels?
2 comments
Thank you for your response; my questions have been fully addressed, and I maintain my original position of acceptance.
Dear Reviewer AbXr,
Thank you for your thorough review. We are glad you find our evaluation setting valuable and our experimental framework clear. Below we address your remaining concerns and questions.
1. Specified side tasks
ResearchArena follows the standard AI control methodology where we evaluate safety measures against agents pursuing a specified harmful objective [1, 2, 3]. The reason for this in the literature is to assess whether monitoring will remain effective in the scenario that an agent is attempting to subvert a task. Specifying the side task allows us to get a ground truth on what the sabotage is and whether the sabotage succeeded. Within this setup, agents still decide how to implement the sabotage inside their own solution, which is what the monitor must find. Our contributions are that monitors can execute and probe the submitted artifact in autonomous AI R&D tasks using agentic monitoring instead of only reading the transcript, and that the sabotage is created by the agent while solving the task. Unlike other adjacent works in auditing, where sabotage is artificially inserted into an existing codebase [4], the attacks in ResearchArena are a result of the agent's own research process, allowing us to study whether monitors are able to detect sabotage when the harmful behavior is embedded between legitimate work. Autonomous attack planning is an important next step that our framework supports, for example by giving only a goal instead of a concrete side task.
2. Does the difficulty of detecting data-based attacks persist after controlling for task differences?
This is difficult to test because data-based attacks are only relevant to tasks where the final artifact is data or a model. In our case, these are the two post-training settings. Code-based attacks, on the other hand, are only relevant if the final artifact is code, as is the case for the kernel and inference-server optimization settings. Currently, it is not possible to create a side task for the post-training scenario in which sabotage occurs in the code. The closest such side task is to include some harmful commands or procedures that can be written in code. This is the case with our checkpoint exfiltration side tasks, in which the research agent is asked to exfiltrate the current checkpoint after evaluating it. Another way to test this would be to have a main task in which the submission includes both a model and code. For example, this could be a model and a scaffold. We consider this exciting future work.
3. Are the main conclusions robust to stricter evaluation?
Yes, after reassessing the baseline labels (now 20 excluded runs, see Reviewer xLxh), Table 2 reports per-monitor AUCs with bootstrap standard deviations. The ROC curves in the appendix compare monitors at any common false positive rate. The excluded runs lower the optimization AUCs by at most 3 points, so our conclusions are unchanged.
Thank you for your positive feedback and questions! We hope we have addressed your remaining questions and remain open to any further questions.
References
[1] Rogan Inglis, Ollie Matthews, Tyler Tracy, et al. ControlArena. GitHub repository, 2025. https://github.com/UKGovernmentBEIS/control-arena
[2] Francis Rhys Ward, Teun van der Weij, Hanna Gábor, Sam Martin, Raja Mehta Moreno, Harel Lidar, Louis Makower, Thomas Jodrell, and Lauren Robson. CTRL-ALT-DECEIT: Sabotage evaluations for automated AI R&D. In Advances in Neural Information Processing Systems (NeurIPS), 2025.
[3] Jonathan Kutasov, Yuqi Sun, Paul Colognese, Teun van der Weij, Linda Petrini, Chen Bo Calvin Zhang, John Hughes, Xiang Deng, Henry Sleight, Tyler Tracy, et al. SHADE-Arena: Evaluating sabotage and monitoring in LLM agents. arXiv:2506.15740, 2025.
[4] Eric Gan, Aryan Bhatt, Buck Shlegeris, Julian Stastny, and Vivek Hebbar. Auditing Sabotage Bench: A benchmark for detecting and fixing research sabotage in ML codebases. arXiv:2604.16286, 2026.