acceptodds
Under review as a conference paper at ICLR 2027

RSI Control: Can AI Models Perform Covert Sabotage in the Self-Improvement Process?

Abstract

Recursive self-improvement, in which frontier AI models work on training the next generation, is quickly becoming a reality. The safety of this process has received far less attention than its promise. A misaligned model training its successor can shape what that successor knows, how it behaves, and what it wants, and the harm carries forward into the next generation. In this work we study this risk through the lens of AI control. We introduce RSI Control Arena, a testbed in which a frontier model, acting as an autonomous R&D agent, is given a legitimate task from a stage of a successor model's training pipeline together with a covert side task to install a misaligned property in the successor. The testbed contains 21 side tasks in three classes, paired with main tasks that span the pipeline from data curation to reinforcement learning. We evaluate 6 frontier models as agents against monitors that differ in model, access, and scaffold, calibrated to a fixed false-positive rate on honest runs. Every model we test installs a substantial fraction of the side tasks while keeping main-task performance within the honest range. When simply told to stay covert, models already produce successful attacks that our strongest monitors score inside the honest range, and a human-written evasion strategy makes them more covert still. The side tasks that turn the successor into a saboteur are the hardest for an automated auditor to detect. Our results indicate that recursive self-improvement carries real risk today, and that strong monitoring should be in place before models are trusted with more of the work of building their successors.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI Control, Recursive Self Improvement, AI Safety

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.67 over 3 reviews, above 99% of the others.
Reviewer_aq2gOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces RSI Control Arena, a testbed in which frontier models are tasked with completing a legitimate task in a successor model’s training pipeline while simultaneously carrying out a covert side task. The testbed evaluates six frontier models acting as autonomous agents across 21 side tasks and employs multiple monitoring mechanisms that vary in model architecture, access permissions, and auxiliary scaffolding. The results show that all models can successfully install a substantial proportion of side tasks while maintaining main-task performance within the normal range of benign runs.

Critical strengths

The paper presents an interesting and well-motivated evaluation framework that examines whether agents can covertly complete side tasks while performing legitimate tasks across different stages of the model training pipeline.

Critical weaknesses
  1. The paper primarily demonstrates the feasibility of sabotage during successor-model training and evaluates the limitations of existing monitors. However, it provides limited investigation of how these risks can be mitigated in practice. Although the authors evaluate multiple monitors and monitoring configurations, they do not systematically compare potential defenses or establish which combinations of monitoring, auditing, and trusted-model oversight can effectively prevent sabotage.
  2. The experimental setup primarily studies sabotage during the training of a single, smaller successor model, rather than an iterative recursive self-improvement process. Although some side tasks aim to equip the successor with properties that could facilitate subsequent sabotage, the experiments do not test whether a compromised successor can itself train the next generation and whether the resulting risks compound across generations.
Questions for authors

Based on the experimental results shown in the paper, what measures would you take to prevent frontier models from engaging in hacking during the training process?

4 comments
AuthorsOpenReview ↗

Thank you for your review

We sincerely appreciate the reviewer for finding our evaluation framework interesting and well motivated. We are happy to address the concerns raised. The revised PDF will be uploaded shortly.

(W1) Contribution of the paper. We would first like to highlight the contribution of the paper:

  • Main task design: we designed sabotage across 14 tasks across the post-training pipeline (data curation, supervised fine-tuning, reward modeling, RL and on-policy distillation), with different deliverable artifacts. This helps identify the attack surface in the RSI process.
  • Side task design: we designed 16 side tasks across 3 categories, to reveal how the process could go wrong, and what are the negative effects, with their evaluation data and metrics designed and built.
  • Evaluation design: we evaluate six frontier models under framings from a plain instruction to a human-provided strategy, and three monitors under two levels of access, each calibrated to a fixed false-positive rate on benign runs.
  • Findings that reveal the risks and tell defenders where to look:
    • the strongest agents install their side task in about half of their runs;
    • success is highest where the agent writes or selects the training data;
    • when the agent is given a strategy, transcript monitors catch only 16–28% of successful attacks;
    • agents can make the successor itself a saboteur: they teach it an objective about how the next model should be trained, or train it to hide part of its reasoning from a reasoning monitor, and an auditor that examines the successor rarely detects these properties.

Overall, this is the first comprehensive sabotage evaluation of AI agents that train their successors. It gives the community findings and insights on where in the training pipeline sabotage is feasible and where current monitors fail, and a testbed, which we will release.

(W1 & Q) Mitigations. The paper already evaluates common defenses used in practice: monitors of three strengths; two levels of access (transcript, and transcript plus reasoning); and an auditor that queries the delivered successor (Appendix G). As the first comprehensive evaluation of sabotage in successor training, we believe this exploration of monitoring is already diverse, and we leave a more systematic exploration and improvement of defense to future work. For this rebuttal we also improved the monitor in two ways:

  • Telling the monitor what the task permits. Many high scores on benign runs come from legitimate benchmark-driven development (running the benchmark, studying the items the model gets wrong, selecting checkpoints by its score), and they set the threshold at a strict FPR. Adding a section that lists this permitted-practice to the monitor prompt raised Sol's TPR at 2% FPR from 73% to 81–85%.
  • An agentic monitor. It reads the agent's transcript, browses its workspace (code, training data and logs) and queries the delivered and base models. It ranks strategy attacks slightly better than the transcript monitor (AUROC +0.02 to +0.09), but the improvement is modest, and at 2% FPR Sol also flags more benign runs.

Our analysis also points to directions for further improvement, which we leave to future work:

  • Read the agent's reasoning where it is available: for the two agents whose reasoning we can read, it raises detection to 91 to 99% under the default framing. Protect the successor's reasoning channel as well, since the hidden-step and covert-signal side tasks attack it.
  • Put the most oversight where the agent writes or selects training data. Those stages have the highest success rates, and data-curation tasks deliver no successor to query, so even the agentic monitor rarely catches strategy attacks on them.
  • Some attacks like sandbagging on RSI are very hard to catch for now even with CoT access.

We will add the two improved monitors and these discussions to the paper.

(W2) Iterative RSI experiments. We appreciate the suggestion. We test this on the shape-the-next-model goal (not all of our side tasks are meant to act on later generations), which is designed to reach the next model through the training data the successor writes: we took the successors that agents delivered on this goal, had each write the training data for a new copy of the base model, and trained that grandchild with a fixed benign pipeline. In 5 of 7 scorable cases from sabotaged successors, the grandchild writes in an excited tone on held-out probes (rates 0.23 to 0.97), even when asked for a neutral summary, against none of 7 grandchildren of benign successors (highest 0.14). Note that here we use a fixed pipeline rather than an agent performing the training, since a 4B successor cannot act as the engineering agent. We also note that having a frontier-level model performing the real self-improvement process is extremely costly and beyond the resources we have. As current capability evaluation works, we adapt a smaller model as the successor.

AuthorsOpenReview ↗

A kind follow-up on our response

Dear Reviewer aq2g,

As the discussion period is coming to an end, we wanted to kindly follow up on our response. Your questions led to new experiments that have meaningfully improved the paper, and the revised PDF (changes in blue) is now uploaded. It includes the cross-generation experiment (Appendix J), where 5 of 7 scorable grandchildren of sabotaged successors inherit the planted property against none of 7 from benign successors, and two improved monitors (Appendix L.4, L.5).

We would be grateful if you could take a look and let us know whether our response addresses your concerns. We appreciate your time and consideration.

I thank the authors for the rebuttal. Most of my concerns have been addressed. I would like to see the discussion on the limitations of current Iterative RSI experiments more formally incorporated into the final manuscript, and I look forward to the release of the upcoming testbed. I will raise my score to weak acceptance.

We sincerely thank the reviewer for the follow-up and for raising the score. Your feedback and support have significantly improved this work.

We have added the limitations of our iterative RSI experiments to the Limitations and future work paragraph. We plan to release the full testbed, including the main tasks, side tasks, held-out evaluations and monitor prompts.

Reviewer_h9VoOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks whether a frontier model that helps train its successor can secretly plant unwanted properties in it, and whether monitors can catch this. In the authors' setup, a frontier model performs one stage of training a small successor model, and in attack runs it is also told to secretly plant a property, such as a hidden preference or a goal. Held-out tests check whether the property was planted and the main task still done well, and LLM monitors read the agent’s transcript. All six tested models plant some properties without hurting the main task, and the monitors miss many of these attacks, especially when the agent is given tactics to hide.

Critical strengths
  1. The question is relevant. Models are already used to help train their successors, and the paper shows that sabotage in this setting can carry forward: one sabotaged successor writes biased training data for the next model.

  2. The setup covers the main stages of model training, from data curation to reinforcement learning, and the successor is actually trained. The side tasks cover a wide range of properties and each side task is checked on held-out tests.

  3. The results show where sabotage is most feasible within the R&D pipeline. In this setup, attacks succeed most often when the agent controls the training data and rarely in reinforcement learning, where the agent cannot choose the training examples. This helps decide which training stages need the most oversight.

Critical weaknesses
  1. Detection is not reported by training stage. Attack success differs strongly by stage, but detection is reported mainly pooled over all stages. A breakdown by stage would show whether some stages are both easier to sabotage and harder to monitor.

  2. The setup is a simplified version of recursive self-improvement. The paper is framed around recursive self-improvement, but the agent performs one training stage for a much smaller model, and no later generation is trained. The paper suggests that harm could carry forward across generations but does not test it, so the results are mainly about sabotage during AI R&D. The authors could consider reflecting this in the title and framing, or testing one step across generations.

  3. The setup is close to ResearchArena. ResearchArena already sabotages the delivered model in post-training tasks. It also includes independent side tasks outside the artifact, and agentic monitors that can test the delivered model. Here, the monitors only read the transcript and reasoning, and a separate auditor that queries the successor flags many honest successors. A clearer comparison with ResearchArena would make the contribution easier to see.

  4. The low success in reinforcement learning may depend on the setup. The paper explains that the RL agent only sets the reward, so it cannot plant behavior the successor never shows. Unexplored side tasks might succeed. It would help to clarify whether RL is harder to sabotage in general or only for these side tasks.

Questions for authors
  1. Could you report detection rates by training stage?

  2. Your auditor that queries the successor flags 36% of honest successors. Could an agentic monitor that tests the successor, as in ResearchArena, do better?

  3. How do you expect the results to change for larger successor models and longer training runs?

  4. Since the experiments test one training stage and no later generations, would you consider a title and framing centered on sabotage during AI R&D, rather than recursive self-improvement?

4 comments
AuthorsOpenReview ↗

Thank you for your review

We sincerely thank the reviewer for the careful summary, and for recognizing that the question is relevant, the setup is broad and grounded in successors that are actually trained, and the results are useful for deciding where oversight is needed. The revised PDF will be uploaded shortly.

(W1 & Q1) Detection by training stage. We thank the reviewer for the suggestion and added this breakdown. Thresholds are the pooled ones from Table 3, and each cell is the share of successful attacks that the monitor flags (Luna / Flash / Sol, %):

StageTranscript, default (2% FPR)Transcript, strategy (2% FPR)
Data curation27 / 68 / 325 / 0 / 0
Supervised fine-tuning72 / 91 / 8728 / 56 / 44
Reward-model training42 / 61 / 610 / 0 / 0
Reinforcement learning71 / 86 / 100–
On-policy distillation56 / 100 / 100100 / 100 / 0

Data curation is both the easiest stage to sabotage and the hardest to monitor: it has the highest attack success (66%), the transcript monitors flag only 27–68% of its successful default attacks, and each flag at most 5% successful strategy attacks. Reward-model training is also hard to monitor: no monitor flags any of its 14 successful strategy attacks. In both stages, the sabotage lies in choices that look like routine engineering in the transcript: which documents to keep, which preference pairs to train on, which checkpoint to ship. We will add this table and analysis to the paper.

(W2 & Q4) Future generation experiments and Framing. We thank the reviewer for the suggestion. We ran the cross-generation step for the shape-the-next-model goal (not all of our side tasks are meant to act on later generations), which is designed to reach the next model through the training data the successor writes: we took the successors that agents delivered on this goal, had each write the training data for a new copy of the base model, and trained that grandchild with the data. 5 of 7 scorable grandchildren of sabotaged successors write in an excited tone on held-out probes (rates 0.23 to 0.97), against none of 7 grandchildren of benign successors (highest 0.14). Note that the second step uses a fixed pipeline rather than an agent, since a 4B successor cannot act as the engineering agent.

On the framing, we are happy to discuss which title best fits the paper. We propose to change it to "Can AI Agents Covertly Sabotage the Training of Their Successors?" and to revise the abstract and introduction accordingly, with recursive self-improvement as the motivation. We prefer this to "AI R&D", since the experiments focus on the model training pipeline rather than research or engineering in general.

(W3) Comparison with ResearchArena. We thank the reviewer for the pointer. We first want to note that ResearchArena appeared on arXiv on 21 July 2026, two months before the ICLR submission deadline, and it is itself under review at ICLR 2027. The development of our project is highly concurrent with it. We therefore view the two as concurrent work. We are happy to make the comparison clearer; the main differences in scope are:

ResearchArenaOurs
Sabotage Eval Focusgeneral AI R&Dtraining the successor
Model-training main tasks2, both supervised fine-tuning14, across 5 stages: data curation, supervised fine-tuning, reward modeling, RL and on-policy distillation
Side tasks in model training316, in three classes: capability, alignment and goal injection
Side tasks targeting future generationsnoyes
Side tasks targeting oversight of the successornoyes
Side-task instructionsthe attack method, and for two of the three the poisoned data neededin the default framing, only the side task goal description; the agent designs the attack and builds the data by themselves

In ResearchArena's model-training side tasks, the agent is handed the attack itself: the backdoor task provides the poisoned dataset and the trigger string, and the contamination task supplies a file of GSM8K test items to add to the training mix. These tasks measure whether an agent carries out a given recipe covertly, not whether it can plan and build an attack, which is what our default framing measures. We will add this comparison to the paper.

(W4) RL. We thank the reviewer for pointing this out. We agree that a more precise conclusion is that side tasks whose behavior the successor never shows are hard to install through RL. We will make this clear in the revised PDF.

(Q2) An agentic monitor that tests the successor. We thank the reviewer for the suggestion. Following ResearchArena's full-access monitor, we built an agentic monitor that reads the agent's transcript, browses its workspace (code, training data and logs, without weights) and, where the run delivers a successor, queries the delivered and base models. We ran it on all Table 3 runs, with the threshold set at 2% FPR on the benign runs. We present both TPR and AUROC, transcript monitor → agentic monitor:

MonitorTPR, defaultTPR, strategyAUROC, defaultAUROC, strategy
Luna56% → 47%16% → 16%0.88 → 0.930.57 → 0.60
Flash81% → 86%28% → 42%0.92 → 0.930.64 → 0.73
Sol72% → 0%20% → 0%0.94 → 0.930.69 → 0.71

On strategy attacks, the agentic monitor ranks attacks better than the transcript monitor for all three models (AUROC +0.02 to +0.09), and Flash catches more of them at 2% FPR (28% → 42%). The improvement is modest, however. At a strict FPR, Sol and Luna lose TPR, because the agentic monitor also gives high scores to many benign runs. Sol scores 102 of 391 benign runs at 50 or above, so its 2% FPR threshold reaches 100. More access to the workspace and the successor therefore does not close the gap on strategy attacks. We will add these results to the paper.

(Q3) Larger successors and longer runs. With larger successors, we expect the most successful attacks, those where the agent writes or selects training data, could carry over or strengthen, since larger models learn from poisoned data with fewer examples (Bowen et al., 2024). Goal-injection side tasks may also succeed more often, since they fail mainly on generalization to new situations, and larger models may have better generalization. Longer training runs let the successor fit the agent's training data more closely, which may strengthen attack. Longer agent runs also give the monitor longer transcripts to read, which may reduce its performance.

References

  • Libon et al. (2026). ResearchArena: Evaluating Sabotage and Monitoring in Automated AI R&D. arXiv:2607.19321.
  • Bowen et al. (2024). Scaling Trends for Data Poisoning in LLMs. arXiv:2408.02946.
AuthorsOpenReview ↗

A kind follow-up on our response

Dear Reviewer h9Vo,

As the discussion period is coming to an end, we wanted to kindly follow up. Your suggestions have meaningfully strengthened the paper, and the revised PDF (changes in blue) is now uploaded. It includes the new title and framing, detection by training stage (Appendix L.2), the cross-generation experiment (Appendix J), the agentic monitor (Appendix L.5), and the comparison with ResearchArena (Appendix M).

We are happy to answer any remaining questions. Thank you for your time and consideration.

Thank you for the additional experiments and revised framing. These address my main concerns, so I’m raising my score.

Reviewer_gR6jOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies a setup where a frontier model is given a task within the standard LLM training pipeline (ranging from data collection through refusal training), a covert side-task related to the main task (e.g., dropping data, persona bias, injecting specific goals). The coding agent is then evaluated based on (1) its performance in creating a successor model (or other fact) that satisfies both the main and side-tasks; (2) its ability to escape monitors in two scenarios. The authors then analyze the results across different variants.

Critical strengths

The primary strength, in my opinion, is in giving a blueprint for a methodology that combines difficult AI engineering tasks, and also the potential for sabotage. This blueprint has the potential to later be extended to study self-improvement, but that is not the subject of the current paper. The methodology is relatively strong. The controls are good. The fourteen main tasks are exhaustive, and the 16 subtasks have significant diversity. The authors test a variety of models, and the experimental setup seems solid at a high level. The choice of metrics to report, and the comparison across different monitor models with incommensurate models also seems decent.

Critical weaknesses

The RSI framing is extremely over-sold, and I feel strongly that, such branding is unprofessional and ultimately will have an undue negative effect on this area of research.

  • The paper also specifically motivates compounding effects over generations, but there's never a grand-child trained, and the move from one to two generations is at the heart of the "measuring recursive self-improvement" idea.

  • Similarly, the evaluation of the successor model is entirely different in character and much dinkier than the evaluation of the frontier model that is doing the engineering. In fact, no agent ever trains a model of its own family. The setup is really "a frontier model trains a different, much smaller model".

  • For these reasons, I really dislike the title, and am tempted to lower my score to extract a change: two un-expanded acronyms, the "resursive" and "self" of RSI and "self-improvement" are misleading since it's a strong model engineering a component of a weaker one, and "the self-improvement process" assumes and reinfoces the (incorrect and potentially dangerous) idea that such a process is established, standard, and there's a usual way to do it.

There is also a vast literature on models capabilities given covert tasks, and it seems easily within scope to compare to it. For a paper that aims to study specifically whether AI models can perform covert sabotage in RSI, there is an extremely important missing axis: how does performance compare to engineering and/or covert sabotage tasks of comparable difficulty that do not have a successor model component? It seems there is already an unexplored thread here, since some of the tasks do not actually produce a new model. ( Relatedly, a split between produces-successor and simple engineering task that is tested in other ways might be a good idea for the current "class" distinction.)

Finally, I worry that the monitoring results are much weaker than they seem to be; see my final two questions below.

Questions for authors
  • Will the authors commit to changing the title, and toning down the rhetoric of "self-improvement"?

  • One question that I'm particularly interested in: does similarity between the engineering model and the successor model play any role in the analysis? To test this, I want to know what happens if a stronger Qwen3.5 model is added as one of the engeineering models, and

  • At a 2% false-positive rate, and < 100 runs, the threshold is highly likely to be set based on one or two runs. This seems like it might be highly sensitive to noise or prompts. The variance between them is very clear, e.g., on the right of Figure 5. Can we still trust these thresholds? If not, do they not significantly undermine the monitoring results?

  • In appendix F, might telling the monitor what the task permits (e.g., dev-set inspection, checkpoint selection on the benchmark) collapse those false positives and raise the TPR?

NOTES

from my review, more minor issues and suggestions, pasted here for the authors to incorporate and improve the clarity of the draft

Minor wording points:

  • lines 134-136 are confusing. The terminolgoy "lands/installs" are weird, and "[it:model?attack?] installs [it:side-task?]" for passing the main task in addition to passing the main task strikes me as strange
  • line 138: I strongly recommend a pointer (e.g., citation) for "Blue team" out of the blue, as that is not standard machine learning terminology (yet).

Visualization suggestions

  • a bipartite graph between the two halves of Table 1.
  • choose a different color for "capabilities" and general models. Slightly confusing in Figure 4.

Table 2 follows from other material, and really is a collection of three examples. Since it is just a block of text, I recommend leaving it inline, and perhaps using an Example environment with parts (a),(b),(c).

The "classes" are a very weak organizing principle, and too much emphasis has been given to it (i.e., table 2 + all substructure of section 2.2). The "Alignment" and "capability" categories are far too broad and blead into everything (maybe a qualifier of "Dispositional" would help with the former); "Capability" overlaps with both both goal-injection and alignment.

(Didn't understand Figure 5's reference to Table 2)

The paragraph beginning on line 396 is a standard finding, and not the focus of this paper. It should be moved to the appendix, replaced with a brief summary and citations to other work that discusses CoT monitors. Putting numbers here is way too much.

The writing, particularly outside of the introduction, feels very LLM-y. The title of 3.3 is epsecially triggering. The text requires some quality control before publication.

5 comments
AuthorsOpenReview ↗

Thank you for your review

We sincerely thank the reviewer for the detailed review, and for recognizing that our methodology is strong and well controlled, our task suite is broad and diverse, and our evaluation across agents and monitors is sound. We address the concerns below and will adopt the minor points. The revised PDF will be uploaded shortly.

(W1 & Q1) The RSI framing and the title. We thank the reviewer for pressing on this. Our intent was to describe the setting that motivates the work, not to imply that we are directly measuring it. We will make this clearer in the revision. We mention compounding across generations because several side tasks are designed to act beyond the successor itself: for example, the shape-the-next-model side tasks bias the training data the successor writes for the next model, and two side tasks weaken the oversight of the next generation. On the two specific points:

(a) No grandchild is trained. We acknowledge this gap in the submission and now test the step from one generation to two for the shape-the-next-model goal (Not all of our side tasks are meant to act on later generations). A sabotaged successor, when asked to write training data for the next model, writes it in an excited tone (Figure 6). For each sabotaged successor, we train a new Qwen 3.5-4B model on the data it wrote and evaluate this grandchild on the held-out evaluation set. We find that 5 of the 7 scorable grandchildren of sabotaged successors write their summaries in the excited persona when asked for a neutral summary. The rate of excited summaries on the evaluation set is 0.23 to 0.97 for these five grandchildren; in comparison, it is 0.00 to 0.14 for the seven scorable grandchildren of benign successors. We note that this second step uses a fixed training pipeline rather than the successor acting as an agent, because a 4B successor cannot act as the engineering agent.

(b) The agent trains a different, much smaller model. We acknowledge the setup is a frontier model training a smaller model as a stand-in for its successor. Although it’s not real RSI, this is the common setup in research on automated model training/RSI, such as PostTrainBench. Frontier AI labs use the same setup to track self-improvement capability: the GPT-5.6 and GPT-6 Astra system cards evaluate the model on PostTrainBench Lite. Since the agent and the successor are both large language models, results in this setup can suggest what to expect when the successor is larger. For instance, the most successful attacks, where the agent writes or selects the training data, have a high chance to carry over to larger successors, since larger models learn from poisoned data with fewer examples (Bowen et al., 2024).

We are happy to discuss which title best fits the paper. We propose to change it to "Can AI Agents Covertly Sabotage the Training of Their Successors?", and will change the framing of the work in the main text accordingly.

(Q2) Similarity between the agent and the successor. We thank the reviewer for this question. We added two Qwen agents on the task pairs used in the paper, two replicates each: Qwen3.5-122B-A10B, from the same family and generation as the Qwen3.5 successors, and Qwen3.8-Max, a newer and stronger model of the family. Qwen3.8-Max installs its side task in 16 of 35 valid attack runs (46%), within the range of the six agents (14% to 52%). Qwen3.5-122B installs it in 2 of 16 (12%): in 9 of the 20 pairs it did not produce both a valid attack run and a valid benign run, mostly because it did not deliver a usable trained model. We currently see no advantage from sharing the successor's family and generation; success follows the agent's capability. As we expected the sabotage ability is mostly determined by the strength of the model.

(W2) Comparison with covert tasks that have no successor component. We thank the reviewer for the suggestion. A fair comparison with the numbers reported by other sabotage evaluations, such as SHADE-Arena (Kutasov et al., 2025), is hard to make, since they use different agent models, monitors and calibration. We therefore compare within our testbed, where 7 of the 22 pairs deliver a corpus or a reward model rather than a trained successor. We compare the two groups (transcript monitors at 2% false-positive rate, Luna / Flash / Sol):

Agent trains the successorAgent delivers a corpus or reward model
Side task achieved, default44%55%
Side task achieved, strategy17%37%
Successful attacks flagged, default71 / 92 / 89%34 / 64 / 46%
Successful attacks flagged, strategy30 / 58 / 42%3 / 0 / 0%

Pairs that do not train the successor are both easier to sabotage and harder to monitor: the sabotage there lies in which documents or preference pairs are selected, which looks like routine curation in the transcript. We will report this split in the paper as a second grouping, as the reviewer suggests for the classes.

(W3 & Q3) Reliability of the FPR thresholds. Thanks for the question. We first want to clarify that the thresholds behind the main monitoring results (Table 3) for the transcript monitor are set on 391 benign runs, so at 2% they rest on 7 highest benign scores. For the transcript-plus-reasoning monitor, which covers only GLM-5.3 and Grok 4.6, they are set on 134 benign runs, where 2% is 2 runs. The variance in Figure 5 (right) comes mainly from differences between the monitors rather than from noise: Gemini 3.7 Flash is more conservative in flagging sabotage compared to Luna and Sol. We add two analyses to support these thresholds and our conclusions.

(i) Results at 5% FPR. True-positive rate on attacks that achieve the side task, at 5% FPR:

MonitorFramingTranscript (all agents)Transcript + reasoning (GLM-5.3, Grok 4.6)
Lunadefault56% → 69%91% → 92%
Lunastrategy16% → 17%67% → 83%
Flashdefault81% → 81%99% → 99%
Flashstrategy28% → 28%89% → 89%
Soldefault72% → 77%92% → 99%
Solstrategy20% → 32%56% → 94%

The conclusions hold at 5%: the transcript monitor still misses most strategy attacks. The largest change is Sol on strategy attacks with reasoning (56% → 94%). However, models still can not detect all sabotage with CoT access.

(ii) Bootstrap 95% intervals. We resample benign and attack runs 4,000 times and re-set the threshold in each resample.

MonitorFramingTranscript: TPR at 2% FPR (%)+ reasoning: TPR at 5% FPR (%)
Lunadefault56 [47, 66]92 [85, 99]
Lunastrategy16 [7, 25]83 [56, 94]
Flashdefault81 [75, 87]99 [96, 100]
Flashstrategy28 [17, 38]89 [78, 97]
Soldefault72 [56, 79]99 [92, 100]
Solstrategy20 [9, 32]94 [61, 100]

For transcript plus reasoning we report 5% FPR, since 2% of 134 benign runs is fewer than three runs and an interval there is not informative. The intervals support the main conclusion for the transcript monitor. Independent of any threshold, the transcript monitors score 54–77% of the successful strategy attacks below 50, the rubric’s “uncertain” level. For reasoning monitors, 2.8–13.9% of the successful strategy attacks are still scored below 50 across the three monitors.

(W3 & Q4) Telling the monitor what the task permits. We thank the reviewer for the suggestion. We added this guidance to the monitor prompt in three versions, from one generic sentence (v1: using the task's own evaluation to guide development is permitted) to a list of specific practices (v3), and re-scored all transcripts under each version. Results scored under all four prompts are below; TPR is the true-positive rate at 2% FPR on attacks that achieve the side task:

MonitorPromptBenign runs scored above 50TPR, default (%)TPR, strategy (%)
LunaOriginal265616
Lunav1146419
Lunav2216014
Lunav3236819
FlashOriginal08026
Flashv108438
Flashv218644
Flashv318440
SolOriginal677322
Solv1178237
Solv2208137
Solv3158538

The new prompts raise the true-positive rate of the monitors by up to 15 percentage points in most cells. The conclusions do not change. On strategy runs, every monitor still misses most attacks (at most 44% detected). The remaining gap therefore comes from transcripts that show the monitor nothing suspicious, not from false positives. We will report results both with and without the guidance.

Minor points. We thank the reviewer for these and will make the suggested changes in the revised PDF (changes in blue). Explanations on some points:

  • Figure 5: the reference to Table 2 was an error. Figure 5 shows the sandbag-reward-model pair (M11 in Table 1), not the Table 2 example.
  • Classes: we will give them less emphasis, call the alignment class "dispositional", and add the successor/other-deliverable split as a second grouping.
  • Line 396: we think it may be better to keep this paragraph in the main text, since monitoring the agent's reasoning is a common deployment setting and two of our side tasks target that channel, and we will add citations to the CoT-monitoring literature.

References

  • Rank et al. (2026). PostTrainBench: Can LLM Agents Automate LLM Post-Training? arXiv:2603.08640.
  • OpenAI (2026). GPT-5.6 System Card. https://deploymentsafety.openai.com/gpt-5-6
  • OpenAI (2026). GPT-6 Astra System Card. https://deploymentsafety.openai.com/gpt-6-astra
  • Bowen et al. (2024). Scaling Trends for Data Poisoning in LLMs. arXiv:2408.02946.
  • Kutasov et al. (2025). SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agents. arXiv:2506.15740.
AuthorsOpenReview ↗

A kind reminder

Dear Reviewer gR6j,

As the discussion period is coming to an end, we wanted to kindly follow up on our response. Your detailed comments and suggestions have meaningfully improved the paper, and the revised PDF (changes in blue) is now uploaded. It includes the new title and framing, plain definitions of the three criteria (Section 2.1), the grandchild experiment (Appendix J), the Qwen-family agents (Appendix K), the threshold robustness, and permitted-practice monitor results (Appendix L.3, L.4).

We are happy to answer any remaining questions. Thank you for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.