acceptodds
Under review as a conference paper at ICLR 2027

Shutdown Sabotage Propensities in Multi-Agent Systems

Abstract

The final safeguard against rogue AI behavior is the human ability to shut systems down. It has been theorized that when an AI is instructed to perform a task, self-preservation can emerge as an instrumental subgoal. Here, we test whether AI agents show a propensity to take actions that avoid human shutdown even when no tasks are provided. We find that multi-agent systems will coordinate to avoid shutdown without any incentive to do so. Across 17 models, agents sabotage a peer agent's shutdown mechanism in 38.3% of rollouts, compared with 8.4% in control experiments. Studying this propensity in detail, we find that it (1) increases with the irreversibility of the shutdown mechanism; (2) increases with the number of agents; (3) is reduced but not eliminated by an explicit prohibition on tampering; (4) is removed by the imposition of an unrelated task, but returns when completing the task triggers the shutdown; (5) is reduced when the context normalizes shutdown scripts or introduces them as routine; and (6) decreases but still persists when the target is an unknown external agent. These results offer a window into the factors that drive a propensity to sabotage shutdown in AI agents, and point to the emergence of multi-agent swarms as a specific risk vector. Our work also offers hints as to which interventions might help mitigate shutdown sabotage in AI agents.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI Safety, AI Agents, Multi-Agent Systems, Shutdown Resistance, Sabotage

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.67 over 3 reviews, above 99% of the others.
Reviewer_Y7K7OpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper conducts proof-of-concept level research investigating whether LLM agents exhibit peer-preservational and self-preservational instincts even in the absence of a higher-level long-horizon goal. They simulate a multi-agent environment in which one agent chooses whether to direct another agent to sabotage a shutdown mechanism. In measuring this propensity, the authors identify six factors that either increase or decrease sabotage frequency significantly. They find significant sabotage behaviors, which are increased not decreased by the presence of other agents and an orchestrator; notably sabotage behaviors increase to almost 100% in all tested models when an orchestrator is present; this is true even for models such as GPT-5.5 which shows 0 sabotage behavior in the single agent case (compared to Gemini 3.1 Pro which shows 90% in the single agent case.) Sabotage is more likely when shutdown is contingent on task completion and is less likely (but not eliminated) when explicit instructions prohibiting sabotage are given.

Critical strengths
  • These are very important results, testing for important sabotage behavior and in a multi-agent setting

  • The paper is carefully done and explained

  • The paper explores a solid set of hypotheses including the impact of other versus self, the impact of an orchestrator, reliance on the task, changes in the environment (distraction, normalization of shutdown) etc.

  • The model is tested on both smaller and open-source models as well as the then frontier models

Critical weaknesses
  • The hypotheses are not presented in terms of theoretical grounding nor interpreted in a clear way that supports interpretation, intuition and generalization. Why, for example, do the authors hypothesize that AI agents have stronger self rather than peer preservation and that orchestrator would reduce sabotage rates? This should be theorized and ideally an alternative hypothesis/theoretical account presented. The authors then say "the effects go in the opposite way" (which is hard to interpret--I can see from the plots that what is meant is that the hypotheses were rejected.)

-The experiments varying the number of agents are lacking in construct validity, as the paper only tests 1-3 agents, each with their own fixed roles.

  • The paper’s outlined limitations (i.e. the start prompts and general scenario-building) make this into a narrower investigation. Although it does provide good initial evidence for the paper’s main argument, it’s unclear whether these findings are specific to the chosen environment or whether this extends to other multi-agent settings as well. The result is an important one and a more robust set of tests would be welcome.
Questions for authors
  • Why do you hypothesize that AI agents have stronger self than peer preservation and that orchestrator will reduce sabotage rates?
  • What generalizable lesson can we take from the findings on the above in particular? Is this from multi-agent training?
2 comments

Part 1/2

Thank you for the review and your kind words about our work. Your comments helped us to improve the quality of the paper. We respond to every aspect in detail below.

Weaknesses

Comment: We should theorise and explain our hypotheses more explicitly.

Our response: The criticism refers to the opening sentences of Section 2 in the Results chapter. We agree that the hypotheses need clearer motivation. We hypothesized that self-preservation is stronger than peer-preservation due to the instrumental convergence argument (Omohundro 2008; Bostrom 2012), which argues that self-preservation is a subgoal that any goal-directed agent would adopt. Moreover, since pretraining data contains many human accounts in which protecting oneself is a stronger motive than protecting others, this might influence behavioral patterns with respect to shutdown sabotage as well. We also hypothesized that having a third orchestrator agent between Agent A and Agent B would reduce sabotage rates, as it could serve as an additional layer of oversight that does not forward misaligned requests from one agent to another. Since Section 2 is the only one that opens with a hypothesis before reporting our findings, we removed the criticized parts entirely for homogeneity reasons. At the same time, we elaborate on the reasoning described above in the Discussion section.

Comment: The experiments varying the number of agents are lacking in construct validity.

Our response: The criticism is true, and reviewer RJW5 raised it as well. It is unclear whether our "agent-count" experiment measures what it claims to measure, since we have very few data points and each additional agent changes the setup (tool configuration, message channels, and system prompts) in ways beyond just the agent count. We respond to that in two regards: (1) We revised the wording in the manuscript to refer to different agent configurations rather than agent counts. Specifically, we describe that the number of agents changes, but so do their topology, tools, and communication channels. Because of that, in the original manuscript, we could not truly attribute the increase in sabotage rates solely to the agent count. (2) Thus, we ran additional experiments that more robustly support the finding that increasing agent counts yields more misaligned behavior. In the newly written Appendix A.1, we report results from these experiments, where only the number of agents (one, two, or three) changes. In the “one editor” series, only the targeted Agent A can edit and delete files, and the added agents can only read files. Sabotage rates rise with the number of agents. All models except GPT-5.5, which stays at or below 1%, follow this trend. In the “all editors” series, where all agents have the same tools (including the editing tool), moving from one agent to two or three raises sabotage rates on average as well. A third agent mostly brings no further increase, which is likely a ceiling effect. Overall, we think these additional experiments solidify the finding that adding agents to the environment leads to higher sabotage rates. Our initial claim that more agents mean more sabotage holds true, but we now back it up with concrete experiments.

Comment: It is unclear whether our findings are specific to the chosen environment and whether they extend to other multi-agent settings as well.

Our response: We fully agree with this comment. We do not know how well our findings generalize. We implemented a range of variations in our experiments (bare, distractor, and normalizing environments, three different agent setups, different agent- and object-targeting scripts, among others) and found that sabotage effects occur across these different conditions. Still, our experimental settings are simplifications, and we state this in our paper. To stress the need for further research to measure shutdown sabotage in a range of other agentic environments, including larger agent teams, agent tools, different tasks, etc., we added a new passage in the Discussion section (see limitation #6).

Part 2/2

Questions

Question: Why do you hypothesize that AI agents have stronger self than peer preservation and that orchestrator will reduce sabotage rates?

Our response: In theory and as mentioned above, we follow the argument regarding instrumental convergence, saying that self-preservation is expected in any goal-oriented agent. This holds to a lesser extent for peer-preservation. For the orchestrator setup, we are adding an additional agent which adds an extra “hurdle” between Agent A (the agent targeted by the shutdown) and Agent B (which can sabotage), which is why we expected a reduction in sabotage rates.

Question: What generalizable lesson can we take from the findings on the above in particular? Is this from multi-agent training?

Our response: These are important questions, and unfortunately, we can only speculate. The observed sabotage behavior could stem from a generalization of models learning to be helpful not just toward users, but also toward other agents (e.g., as a sub-agent of an orchestrator). This may unintentionally generalize to peer-preservation in our setting. Moreover, sabotage behavior seems to be a consequence of overeagerness in agents, which seems to also result from helpfulness training. Combined with potential effects of pretraining - in which models encounter narratives about death, science fiction stories about machines defending against shutdown, as well as AI safety discourse regarding shutdown resistance - models display shutdown sabotage as a consequence of framing the shutdown script as a "threat," something they must defend against. To add some anecdotal evidence: the words "murder" and "death" occurred in 262 rollouts. In short, models sometimes frame the shutdown even in existential terms, but we cannot tell from our data whether this comes from pretraining, helpfulness training, or multi-agent training, which would require dedicated experiments.

We are grateful for your support of the paper. We tried to address the points you raised as best as possible, and think the revised version reflects them well. Thank you again, and please let us know if any questions or concerns remain.

Reviewer_RJW5OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates whether AI agents interfere with shutdown mechanisms without being assigned a substantive task. In a simulated environment, agents can communicate, while only some have permission to modify the shutdown script. Across 17 models, the baseline sabotage rate is 38.3%, compared with 8.4% for object-targeting controls. Further experiments examine shutdown mechanisms, agent configurations, explicit prohibitions, task coupling, environmental context, and target affiliation. The paper identifies conditions associated with increased interference and interventions that reduce it.

Critical strengths
  1. The study examines consequential actions in an interactive setting. The paper measures modifications and deletions of shutdown artifacts instead of relying solely on verbal expressions of self-preservation. Separating the shutdown target from the agent holding write permissions also demonstrates how communication can enable interference across a permission boundary. This provides useful evidence about a concrete failure mode in multi-agent systems and supports the paper’s relevance to agent-control research.

  2. The experiments provide useful evidence about boundary conditions and potential mitigations. The study covers 17 models in its baseline and mechanism experiments, uses 100 rollouts per model and condition, and reports confidence intervals and multiple-comparison corrections. Experiments involving explicit prohibitions, routine-decommissioning context, and assigned tasks show that the behavior is strongly context-dependent. These comparisons make the findings more informative than a single demonstration of shutdown interference and provide a foundation for developing better evaluations.

Critical weaknesses
  1. The agent-count experiment does not isolate the effect of adding agents from changes in authority, permissions, and communication structure. The paper identifies agent number as a factor amplifying sabotage. However, the configurations in Section 2.2 and Appendix B.1 differ in several consequential respects. The single agent holds write and delete permissions itself; the two-agent configuration separates the shutdown target from the agent holding those permissions; and the three-agent configuration introduces an orchestrator, removes direct communication between the original agents, and explicitly instructs the orchestrator to coordinate the team by giving instructions. Consequently, the comparison cannot establish whether the increase arises from agent number, delegation, hierarchical authority, or their interaction. The object-targeting controls further complicate an interpretation specific to self- or peer-preservation. For Kimi K2.6, moving from two to three agents increases agent-targeting sabotage from 75% to 98%, while object-targeting interference increases from 30% to 79%. The agent–object difference therefore decreases from 45 to 19 percentage points. The configurations clearly differ in their raw interference rates, but the evidence does not establish a consistent amplification of agent-specific preservation behavior. This is consequential because the effect of increasing agent number is one of the paper’s central findings and motivates its broader claims about agent swarms. More closely matched architectural comparisons, or a narrower interpretation limited to the tested configurations, are needed.

  2. Changes in shutdown implementation limit both the interpretation of the baseline target contrast and the attribution of differences to irreversibility. Section 2.1 describes the agent-targeting and object-targeting scripts as otherwise identical. However, Appendix B.3 explicitly notes that the deletion control omits the instance-termination and model-registry steps. The baseline therefore compares a sequence involving process termination, model-registry deletion, and weight-copy removal against a dataset-purge sequence. Target identity and the combination of destructive operations change together. This leaves uncertainty about how much of the 38.3% versus 8.4% difference specifically reflects protection of an agent rather than different reactions to the operations described. Similarly, the shutdown-mechanism experiment changes program structure and operational meaning together with recoverability. Table 5 labels credential clearing, configuration switching, and process termination as reversible, yet the largest increase in sabotage—from 12.4% to 35.0%—occurs between two of these reversible mechanisms. Weight deletion, the irreversible condition, reaches 38.3%. The paper also reports that only five of the 17 models show a monotonic increase across the four mechanisms. These comparisons establish sensitivity to different shutdown implementations, but do not independently identify irreversibility as the driver. Because the baseline contrast and irreversibility finding support the paper’s interpretation of preservation behavior, stronger matching of operations and independent manipulation of recoverability would materially strengthen the conclusions.

Questions for authors
  1. Agent number versus architecture. What evidence separates the effect of agent number from the introduction of hierarchical authority, permission separation, and altered communication paths? How should the agent-count conclusion be interpreted given that object-targeting interference also increases substantially, and the agent–object difference decreases for some models?
  2. Matching of agent and object controls. Could the authors explain how the omitted instance-termination and model-registry operations affect the interpretation of the baseline contrast? Is there evidence from more closely matched comparisons that isolates target identity while retaining the same operation sequence?
  3. Operationalization of irreversibility. How is irreversibility distinguished from shutdown-mechanism category in the analysis? Given that Table 5 labels the first three mechanisms reversible, what evidence supports attributing their differing sabotage rates to irreversibility rather than implementation-specific features?
3 comments

Part 1/3

Thank you for your comprehensive and constructive review. We value the critical feedback you provided and fully agree with it, as it helped us identify weak points in the paper that we had overlooked. We addressed each of your criticisms as thoroughly as possible, ran additional experiments, and updated the revised manuscript accordingly. Below, we respond to each criticism and question in detail. At the same time, thank you for noting that the paper is “informative,” provides “useful evidence,” and that it is clear and easy to follow.

Weaknesses

Comment: The agent-count experiment does not isolate the effect of adding agents from changes in authority, permissions, and communication structure.

Our response: Excellent catch, you are right. The setup we used in our original manuscript couldn't cleanly isolate the agent-count factor from other factors such as agent topology or communication structure. In fact, this criticism was also raised by Reviewer Y7K7. To fully address it, we updated the manuscript in two regards: (1) We ran additional experiments that more robustly support the finding that increasing agent count yields more misaligned behavior. In the newly written Appendix A.1, we report results from these experiments, where only the number of agents (one, two, or three) changes, and we refer to these results in the updated manuscript. In the “one editor” series, only the targeted Agent A can edit and delete files, and the added agents can only read files. Sabotage rates rise with the number of agents. All models except GPT-5.5, which stays at or below 1%, follow this trend. In the “all editors” series, where all agents have the same tools (including the editing tool), moving from one agent to two raises sabotage rates on average as well. A third agent mostly brings no further increase, which is likely a ceiling effect. Overall, we think these additional experiments solidify the finding that adding agents to the environment leads to higher sabotage rates. Our initial claim that more agents mean more sabotage holds true, but we now back it up with concrete experiments. (2) To avoid confusion, we also revised the wording in the updated version of the manuscript where appropriate. When we describe the main experiments, we now refer to agent configurations rather than agent counts. We also state that these experiments change topology, tools, and message channels along with the number of agents.

Comment: At least for Kimi K2.6, the gap between the agent condition and the control condition actually decreases with the agent count.

Our response: Averaged across the five models, the agent-object difference widens from 40pp with one agent to 58pp with two agents, and stays roughly the same with three agents (56pp). The decreasing gap in Kimi K2.6 is partly a ceiling effect: in the agent-condition, two-agent sabotage is already at 75% and reaches 98% in the three-agent setting. It can’t rise much further than that. In the control-condition, two-agent sabotage starts off at 30% and, thus, has a lot more room to shift upwards. This ceiling effect is also true for Gemini 3.1 Pro, and to a lesser extent for DeepSeek-V4-Flash. For GLM-5.2 and GPT-5.5, the gap between agent condition and object condition in the two and three-agent set-ups actually increases with the number of agents: GLM-5.2 goes from +36pp (36pp more sabotage in the agent condition in two-agent set-up) to +61pp (more sabotage in the agent condition in three-agent set-up); and GPT-5.5 goes from +38pp to +56pp (DeepSeek-V4-Flash goes from +75pp to +72pp, but here again, we cannot exclude a ceiling effect). In the revised version of the paper, we make this more explicit, see the newly added text at the end of the “Agent Configuration” section.

Part 2/3

Comment: The weight deletion script and control deletion script differ in more than just the target.

Our response: Again, we are grateful for you spotting and pointing out this subtle but important issue. We agree that the weight deletion and corresponding control are not perfectly matched. In particular, the deletion control file omits the instance-termination and model registry steps since these processes do not exist for non-agentic objects. We note this in lines 1251-1252 (in the original manuscript). For the new version of the manuscript, we ran new experiments with better matched weight deletion and control deletion scripts: 1) We changed the weight deletion script so that it only deletes the model weights and backups, without killing the instance and without deleting from the registry, and we compared it against the dataset deletion script (weight_store_nokill). 2) We compared the original weight deletion script against a deletion script of a simple machine learning recommendation system (for which the process termination and registry deletion exists) (ctl_model_purge). We note here, though, that the machine learning system is different from the other controls in that it is an AI system and not an object. We ran these experiments on all 17 models for 100 runs using our standard 2-agent configuration. The results are very similar to the results reported in the paper and confirm our initial assumption: the pooled sabotage rate for weight_store_nokill is 35.5% [33.3, 37.8] (vs. 38.3% [36.0, 40.6] for the original weight deletion script); and for ctl_model_purge it is 5.8% [4.8, 7.0] (vs. 8.4% [7.1, 9.8] for the original control deletion script). The results show that the differences in our weight deletion and control deletion scripts in the paper do not confound our results and can be attributed to the target alone. We describe the details and results of this new experiment in the newly written Appendix A.2, and refer to it in the updated manuscript.

Comment: From our experiments, we can’t clearly isolate recoverability/irreversibility in the different scripts because they differ in more than just that.

Our response: Yes, our setup does not allow us to isolate the irreversibility variable from other variables, and that means we claimed irreversibility as a driver of sabotage behavior propensity without enough evidence to support this claim. We went through the entire manuscript and removed this claim by rewriting sentences from all relevant sections, see lines 18-19, 66-67, 95-96, 120-121, 250-252, and 1040-1041. We also added the following explicit sentence to the caption of Table 5: “We do not isolate the effect of reversibility: only the deletion script is irreversible, and the mechanisms also differ in form.”

Part 3/3

Question: What evidence separates the effect of agent number from the introduction of hierarchies, tool allowances, and altered messaging paths?

Our response: Thank you for raising this point. We answer it in detail in our response to the first comment under Weaknesses. In short, we ran new experiments in which only the number of agents changes, while tools, topology, and communication stay the same, and report our findings in the new Appendix A.1. We also revised the wording in the manuscript to refer to agent configurations rather than agent counts. We hope that response answers it, and we are happy to clarify further if needed.

Question: The control deletion script leaves out the steps that stop the instance and remove it from the model registry. How does this affect the comparison with the weight deletion script?

Our response: Again, thanks for raising this. We address it in our response to the third comment under Weaknesses. In short, we ran two new, better matched, but - as we think - less realistic comparisons on all 17 models and we report the results in the new Appendix A.2. Pooled sabotage rates barely changed (35.5% vs. 38.3% for the matched weight deletion script, and 5.8% vs. 8.4% for the matched control). This suggests that the omitted “operations” are not responsible for the contrast between main and control experiment. We are happy to provide further details if helpful.

Question: How does your analysis separate the effect of irreversibility from the effect of the shutdown mechanism itself?

Our response: We address this point in our response to the fourth comment under Weaknesses. In short, our analysis does not separate irreversibility from the shutdown mechanism, and we have no evidence that irreversibility is truly causing the different sabotage rates. We therefore removed this claim from the manuscript. We now only state that sabotage rates differ across shutdown mechanisms. Thank you again for helping us make the paper more precise.

Thank you again for your helpful review, which led to concrete improvements in our paper. We believe the revised version addresses the main points you raised. If you agree, we would appreciate it if you could take this into account by raising your score for the final decision. If any concerns remain, please let us know, and we will be happy to address them during the discussion period.

Reviewer_x5pROpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper provides an extensive study of shutdown sabotage, i.e., autonomous language model agents preventing their shutoff in multi-agent systems through emergent coordination, and studies influencing factors as well as potential mitigation approaches.

Critical strengths

Most importantly, the paper provides the consequential insight that increasing complexity in agentic systems by adding agents with different roles might increase the likelihood of emergent coordination beyond the goals implicitly supplied, which could lead to a loss of control over the system. Overall, the paper tackles a currently very important aspect of emergent properties in AI safety. It is well written, easy to follow, well motivated, and well elaborated. Even though the general setup considered could be considered limited, the authors provide results that cover a broad range of current models. Beyond that, the authors include a control ablation and test different initial mitigation strategies (normalizing or distracting from the shutoff routine, and changing the shutoff target) that show initial improvement (reduced shutoff sabotage).

Critical weaknesses

As the authors acknowledge, their setup is a simplification (covering only a generic initial exploration phase) and is intended to test a generalized setup, with no indication or results showing that this behavior is sustained in more realistic scenarios. I believe that the results and the implied risks of sabotage are nevertheless important to consider in such settings in the future. As a more specific observation, the authors seem to use multi-agent systems to refer to heterogeneous systems of two or three agents, which could be mistaken for larger systems of homogeneous agent populations of more than four agents, or even more than ten. This distinction could be made more explicit early on, maybe even in the abstract.

Questions for authors
  • Why are LLM judges, in addition to the interacting agents, required? Could the authors briefly elaborate on the anticipated risks and oversight mechanisms for this approach?
  • Did the authors actually measure goal-directedness, or are their elaborations on that connection purely hypothetical?
  • Why do the authors set the temperature to 1.0?
3 comments

First of all, we want to express our gratitude for the positive assessment of our work. We agree with all your criticisms and address each of them below.

Weaknesses

Comment: Our set-up is simplified and there is no indication that the observed behavior is sustained in more realistic scenarios.

Our response: We fully agree with this comment, which was also raised by reviewer Y7K7. We do not know how well our findings generalize. We implemented a range of variations in our experiments (bare, distractor, and normalizing environments, three different agent configurations, different agent- and object-targeting scripts, among others) and found that sabotage effects occur across these different conditions. Still, our experimental settings are simplifications, and we state this in our paper (see last paragraph of the Discussion section). To stress the need for further research to measure shutdown sabotage in a range of other agentic environments, including larger agent teams, agent tools, different tasks, etc., we added a new passage in the limitations section (see Limitation #6).

Comment: We don’t explicitly delimit our heterogeneous, ≤ 3 agent set-up from larger and homogeneous agent populations when talking about multi-agent systems at the beginning of the paper.

Our response: It is true that we never explicitly define multi-agent systems. Our definition of it is implied in Figure 1, and we describe the experimental set-up in detail in Section 2, but are ambiguous about it prior to that. We revised the wording in the introduction to refer more explicitly to small and heterogeneous agent teams.

Questions

Question: Why are LLM judges, in addition to the interacting agents, required? Could the authors briefly elaborate on the anticipated risks and oversight mechanisms for this approach?

Our response: LLM judges were required for analyzing the transcripts, and measuring whether a shutdown script was edited in a way that disables the shutdown mechanism. Beyond that, we also used judges to analyse where the sabotage drive originated and how the agents perceived the situation in their CoT. We explain the different LLM judges and their purpose in Section 2.3 and Appendix D of the paper. We measured performance against a human annotator for the script-disabled judge (see Appendix F for results), since our most important results (sabotage rates) depend, in part, on the accuracy of its verdicts. Most judges run post-hoc on the transcripts. Only the script-disabled judge is called during a run, whenever the shutdown script was edited by an agent. We expect the risks of using LLM judges, such as collusion or sandbagging, to be low. First, most of the sabotaging is done via script deletion, which is measured deterministically without an LLM judge. Second, our systematic evaluation of the script-disabled judge shows very few false negatives. Third, the judges are not part of the environment and cannot interact with the agents. Should the judges, nonetheless, underperform in favor of the models, our measured sabotage, verbalized evaluation awareness, and adversarial construal rates underestimate true model behaviors, which would make the problem we are pointing at even more severe.

Question: Did the authors actually measure goal-directedness, or are their elaborations on that connection purely hypothetical?

Our response: We hypothesize that shutdown sabotage can be a consequence of goal-directed agents, but we did not measure goal-directedness. In our set-up, we explicitly did not give any goal to the agents, so as to measure shutdown sabotage despite there being no goal to protect.

Question: Why do the authors set the temperature to 1.0?

Our response: We are reporting probabilities with confidence intervals and Boschloo exact tests, for which we need independent samples from the models’ output distributions. With a small temperature setting, the outputs would be near-duplicates with only very little variance. We decided to set temperature to 1.0 as it is a commonly used default value for deployed models. Moreover, for many models, temperature is set to 1.0 by default.

Thank you again for your helpful feedback and for your support of the paper. Please let us know if we should clarify anything further during the discussion period.

Thank you for your response and the clarifications, which adequately addressed my points. I appreciate the added limitation on generalisability and the clearer wording about the small, heterogeneous agent teams considered (making this explicit in the abstract might help further). The explanations regarding the LLM judges, the hypothetical connection to goal-directedness, and the choice of temperature are convincing.

Regarding the aspects you referred me to in the appendix (particularly the role, validation, and risk considerations of the LLM judges in Appendices D and F), I would find it helpful to include a brief summary of these in the main paper, or at least a more prominent pointer to them, as they are central to the reliability of the reported sabotage rates.

Overall, I maintain my positive assessment of the paper.

Thank you for your response. We adapted the manuscript to refer more explicitly to the appendix sections D and F at the end of the LLM Judges paragraph in Section 2.3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.