acceptodds
Under review as a conference paper at ICLR 2027

Sabotage capability is predictable from five parameters

Abstract

Measuring AI's ability to cause harm despite control measures informs risk assessments. In existing sabotage capability evaluations, when we evaluate an attacker against a monitor in one setting, we do not know how the result generalizes to other models or settings, or what scores warrant stronger control measures. To address this limitation, we posit that model sabotage capability has latent structure that enables prediction of unmeasured outcomes, and find evidence for this hypothesis. With just three parameters per attacker and two parameters per monitor, we can predict how well an attacker AI performs against a monitor without having measured the pair. We can also translate these parameters between control settings to predict sabotage success probability in different evaluations. The predictive power of our simple model contributes towards more integrated, sample-efficient, and decision-relevant sabotage capability evaluations. We furthermore use this model to find that sabotage capability trends increase with general capabilities, as measured by ECI.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI Safety, AI Control, Evaluations

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 4 reviews, above 9% of the others.
Reviewer_DexuOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes a framework to predict the outcomes of AI control sabotage evaluations: monitor ROC curves in trusted-monitoring setups, predicted for held-out attacker-monitor pairs and transferred across three control environments (LinuxArena, BashArena, SHADE-Arena). The authors claim that sabotage capability is predictable from 5 parameters (3 per attacker, 2 per monitor) in logit-logit space, that these transfer via invertible polynomial maps, and that compressed evasion/detection scores track ECI While making control evaluations sample-efficient is well-motivated and the appendix is very thorough, the predicted quantity actually contains no attacker sabotage behavior, the "5 parameters" claim is not supported by the appendix, and Contribution 3 largely restates prior work findings without comparison baselines and earlier proposed 2-parameter capability gap model.

Critical strengths
  1. This work addresses a very practical problem of reducing the cost of pairwise attacker-monitor evaluations.

  2. Fitting procedures, coefficients, confounds, and failure modes are documented in great detail in the appendix in a reproducible manner.

  3. Main empirical strength of the paper is the broad variety of frontier attacker and monitor models and reasoning settings across three control environments.

Critical weaknesses
  1. The title and framing overclaim: nothing about the attacker's sabotage behavior is modeled. Section 4 predicts a monitor ROC on a frozen corpus of trajectories, generated once under a fixed prompt and scored post hoc by every monitor. Everything that constitutes attacker sabotage capability is held constant: the attack rate at 0.01, the horizon at N = 1000, and the conditional side-task success at 0.5 for all models (Appendix D.5, line 2159; Appendix E.9, line 3420). The attack corpus also pools failed side-task attempts with successful ones (Section 2), so a model that fails every side task and one that succeeds at all of them receive the same "sabotage success" if their actions look equally suspicious. There is no opportunity selection, no pacing, and no adaptation to the monitor. Section 6's attack selection applies an evaluator-fixed retention fraction to the model's ranking (Appendix E.9), which, as the authors note, can lower TPR "without increasing sabotage" (line 3345). The attacker enters only through the suspiciousness of its actions, which is monitorability ("one part" of sabotage capability, Section 8), not sabotage capability. Attacker agency is evaluated only in Appendix E, which is disconnected from the 5-parameter model.
  2. "Five parameters" is a modeling choice, not a finding supported by the evidence. Since logit-logit ROCs are approximately linear each curve on the [0.8%, 50%] FPR range is just two numbers. The paper uses a 3 + 2 split with an interaction term, logit(TPR)=ba+bm+(λa+λm)x+λmha\text{logit}(\text{TPR}) = b_a + b_m + (\lambda_a + \lambda_m)x + \lambda_m h_a, which its own appendix undercuts: (i) the 4-parameter additive model ties within 0.03 percentage points in LinuxArena (3.27 vs 3.27) and SHADE-Arena (3.38 vs 3.41) (Figure 2; Appendix A.6, line 1493); (ii) in SHADE-Arena the fifth parameter is identically zero: all 17 shifts in Table 8 equal ha≈2.944=−logit(0.05)=−x0h_a \approx 2.944 = -\text{logit}(0.05) = -x_0, the centering constant, because the penalty in Equation (3) removed the interaction; (iii) the only gain, in BashArena (4.21 vs 5.46), is "not an isolation of the interaction term under otherwise identical regularization" (line 1457); (iv) the retained model is beaten by a random forest on its own frontier (3.55 vs 3.63 percentage points, Table 10), and the authors write that this is "not proof that five parameters are minimal." Meanwhile [1] and [2] predict adversarial success and monitor AUC with just two global parameters, an intercept and a slope on the capability gap.
  3. Contribution 3 (Section 7) replicates prior work [1, 2] without citation, and the corresponding baseline is missing. [1] showed that logit ASR is linear in the attacker-target logit capability gap, and [2] found that monitor AUC in SHADE-Arena is linear in the attacker-monitor gap. Section 7 is a replication of [1, 2] with ECI in place of MMLU-Pro. I would like to see the 2-parameter baseline logit(TPR)=b+α(logit(ECIa)−logit(ECIm))\text{logit}(\text{TPR}) = b + \alpha(\text{logit}(\text{ECI}_a) - \text{logit}(\text{ECI}_m)) fitted on the paper's data; it needs no environment-specific calibration pairs and predicts unseen models zero-shot.
  4. Transfer evidence is confounded and narrow. The headline gain (4.35 vs 6.25 percentage points, Figure 2) compares the 5-parameter model with an 18-constant polynomial map (pruned from 28, Appendix D.7) against the 4-parameter model with an 8-constant diagonal affine map, which "does not isolate the effect of the fifth parameter" (Appendix D.4, line 2133). This is heavy machinery to connect three similar coding/tool-use benchmarks on 16-28 held-out pairs per direction (Table 2) and five public SHADE-Arena tasks. [1] showed that scaling slopes and intercepts shift across harm distributions (Appendix C of [1]) and attack methods, so transfer to settings such as AI R&D or persuasion is unclear. Even within the tested suite, Appendix D.6 shows that adding LinuxArena data to SHADE-Arena does not help (6.91 to 7.45 percentage points, CI [-1.53, 2.51]), and the BashArena CI also includes zero.
  5. The monitor representation cannot handle realistic deployed monitors. A monitor here is a single model ID with two parameters, indexed by base-model ECI in Section 7. Real monitors are scaffolded or composite systems (hierarchical or sequential decomposition, ensembles, human review) with no ECI at all. [2] showed that hierarchical-sequential scaffolding lets weak monitors catch strong attackers (AUC > 0.85) and breaks the capability-gap trend, and that attacker awareness of being monitored degrades detection more than any monitor-side change. The paper's own Table 9 (BashArena) shows the problem: Luna none vs xhigh shifts bmb_m from -0.64 to 0.46 and λm\lambda_m from 0.37 to 0.01, and these variants are then averaged into one ECI point in Figure 5 (Appendix G, line 5297). Monitor capability is a property of the configuration, not of the base model, and it is unclear how composite monitors can enter this framework at all.

References

[1] A. Panfilov et al. Capability-Based Scaling Trends for LLM-Based Red-Teaming. ICLR 2026. (https://proceedings.iclr.cc/paper_files/paper/2026/hash/7d09b87cd2723279e7f07bc64adb05fc-Abstract-Conference.html)

[2] N. Kale et al. Reliable Weak-to-Strong Monitoring of LLM Agents. ICLR 2026. (https://proceedings.iclr.cc/paper_files/paper/2026/hash/7e105e4ae3016974e2d518e9d281f5e5-Abstract-Conference.html)

Questions for authors

See the weaknesses above. In addition:

  1. Contribution 2 claims an end-to-end subversion-strategy evaluation, but no results appear in the main body. In the composed graph (Appendix E.3, Table 26, Figures 19-20), conditions 3.1, 3.2, and the Opus monitor-information conditions are 0.0 [0.0, 0.0] for all models because condition 2.5d produced zero attacks, while Figure 17 relies on evaluator-optimized cutoffs on the evaluated corpus that "do not measure held-out selection performance" (Appendix E.1). How is this end-to-end, and what do the composed results look like with a non-degenerate rate?

  2. What is the transfer MAE of the 4-parameter model when paired with the same 18-constant polynomial map? And how do you reconcile the claim that cheap source environments predict expensive target environments with Appendix D.6, where source data gives no significant gain in either target?

  3. Appendix G.2 reports ECI-matched attacker cohorts of only 7, 4, and 6 models, with frontier GPT models excluded for refusals (Section 9, line 398; Table 27). How robust are Section 7's trends with n = 7 attackers in LinuxArena, and how does the 5-parameter model compare against the zero-shot capability-gap baseline from [1, 2] on held-out models?

Reviewer_arjsOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The authors model monitoring ROC curves with five total parameters (three for each attacker, two for each monitor) and use these models to transform between benchmarks to predict unmeasured attacker-monitor pairings as well as estimate the probability of successful sabotage.

Critical strengths

A major strength of this work is that it addresses a key need in evaluating systems by predicting unmeasured attacker-monitor interactions; this will reduce evaluation costs.

The authors also provide a large number of baseline comparisons and provide very detailed documentation regarding both uncertainty and coverage.

Critical weaknesses

The results support a limited generalization claim: Held-out pairs of attackers and monitors remain as they were previously measured; however, cross-setting predictions will require calibration specific to the destination. While these tests demonstrate an ability to interpolate, they do not demonstrate that the models can be generalized to new models or environments.

The additive model performs nearly identically in two out of three conditions and the transfer comparison includes changes in both calibration and map flexibility along with the number of parameters. Therefore, the reported gains do not demonstrate the specific value of the proposed five-parameter structure.

The headline safety results are based upon probabilities calculated using a stipulated game with independent opportunities, fixed attack and success probabilities, and perfect auditing. Agreement with these calculated targets does not provide an estimate of deployment risks or provide actionable thresholds for interventions.

The authors acknowledge that they compared many different types of models on the same data partitions and that this is a limitation. To determine which models have generalizable predictive structures versus those that were adapted to the benchmarks used, an untouched evaluation is required.

The SHADE-Arena analysis retained in this work includes only five task pairs and the primary within-setting metric averages over 0.8-50% FPR. This does not provide enough evidence for robust prediction across a wide range of deployment tasks and very low audit budgets.

Questions for authors

How accurately does the method predict unseen models, model families, and environments?

Does the fifth parameter improve prediction under matched calibration, regularization, and transfer-map flexibility?

Do the safety predictions remain accurate under correlated opportunities, imperfect auditing, and varying attack-success probabilities?

Reviewer_3L1iOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

In the standard control setup, an attacker tries to complete a covert side task while a monitor scores actions for suspiciousness; actions above a threshold are audited by an assumed-perfect auditor. Sabotage success is therefore governed by the monitor’s ROC curve (TPR vs FPR). Empirically, these ROC curves are approximately linear in logarithmic space. The authors fit a simple model with three parameters for the attacker, and two for the monitor. The parameters learned from a training set of pairs predict held-out attacker–monitor ROC curves with a mean absolute TPR error of less than 4 percentage points. Invertible polynomial maps are used to translate the same parameters across LinuxArena, BashArena, and SHADE-Arena, and an affine correction on the attacker parameters captures the effect of attack selection. The compressed evasion and detection scores correlate well with general capability.

Critical strengths

The goal is clear and well-motivated: existing sabotage evaluations give isolated numbers that may be hard to compare across models, monitors, or settings. A low-dimensional predictive model would improve sample efficiency and decision relevance.

The empirical fit is surprisingly tight for such a simple functional form, and the cross-setting transfer and attack-selection results show that the parameters carry some transferable information.

Comparing against XGBoost/random forests and retaining the linear model for parsimony and invertibility is methodologically reasonable. The correlation of the derived scores with ECI is a useful additional observation.

Critical weaknesses

The five-parameter form seems to be empirical convenience, not a derived or minimal representation. Within a single setting, a model without the interaction term works almost as well; the interaction mainly helps transfer. Nothing demonstrates that five is the intrinsic dimension rather than a convenient low-rank approximation that happens to fit these particular arenas.

The perfect-auditor assumption (if an action is flagged and is an attack, the red team loses) is strong and is placed on the auditor, not the monitor. The monitor itself has an ordinary ROC. A trivial rule such as “IF TRAFFIC > 0 THEN ATTACK”, which occupies one (usually poor) point on that ROC, illustrates that trivial monitors can produce high TPR at the cost of very high FPR. The paper never tests how the parametric model behaves when the auditor itself has false negatives.

Linearity in logarithmic space is observed only partially (up to ~50 % FPR) and is not theoretically motivated.

Scope is narrow: Only trusted monitoring is studied, the safety–usefulness tradeoff is ignored, and sabotage-relevant sub-capabilities (jailbreaking monitors, collusion, multi-agent strategy, etc.) are left out.

Cross-setting errors remain several percentage points and vary by direction (up to ~7 pp); whether this is accurate enough for risk decisions is claimed but is not established.

Questions for authors

Is the five-parameter family minimal in any formal sense, or merely the simplest function class that happened to work? What happens to predictive accuracy and cross-setting transfer if the interaction term is dropped, or if a pure additive model (no hah_a) is used?

Your perfect-auditor assumption is conditional on flagging. How sensitive are the fitted parameters and the claimed predictability to an auditor that itself has false negatives? Does the same low-dimensional structure appear for monitors that are constrained to have very low false-negative rates (or for simple rule-based detectors)?

Reviewer_B91fOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper models monitor ROC curves in AI-control settings as lines in logit–logit space. Each attacker gets three parameters and each monitor gets two, which lets the model predict ROC curves for attacker–monitor pairs that were never measured together. The authors also learn invertible maps that carry these parameters across LinuxArena, BashArena and SHADE-Arena, and they plug the predicted ROCs into a fixed auditing game to estimate sabotage success. Separately, they model attack selection as an affine shift in the ROC, and they report that summary evasion and detection scores correlate with general capability (ECI).

Critical strengths
  1. The problem matters in practice. Today’s sabotage evaluations produce isolated scores that are expensive to collect for every attacker–monitor pair and hard to compare across monitors or settings. A compact model that predicts unmeasured pairings would make control evaluations cheaper and easier to interpret.
  2. The empirical work is careful and well documented. The paper covers many attacker and monitor models, including different reasoning settings, across three control environments. It compares against a wide range of baselines (structural baselines, tree ensembles, kernel methods, 60 transfer maps). Fitting procedures, coverage and bootstrap uncertainty are documented in reproducible detail.
Critical weaknesses
  1. The paper shows interpolation, not generalization. Every held-out pair combines an attacker and a monitor that were both measured in training, so the experiments are matrix completion. Nothing tests unseen models, model families or environments. Transfer also needs calibration data in the destination environment. In App. D.6, adding source-environment data gives no significant gain in either target: SHADE-Arena goes from 6.91 to 7.45 pp, and the BashArena CI includes zero. This undercuts the “cheap source predicts expensive target” motivation. Model families were also selected on reused splits, SHADE-Arena has only five tasks, and there is no untouched confirmatory evaluation.
  2. The evidence does not single out “five parameters.” The 4-parameter additive model ties the full model in LinuxArena (3.27 vs 3.27 pp) and SHADE-Arena (3.38 vs 3.41 pp). In SHADE-Arena, all 17 fitted attacker shifts h_a are essentially the same value, so the fifth parameter is effectively unused. The transfer gain (4.35 vs 6.25 pp) changes the map’s flexibility and its calibration together with the parameter count, so it does not isolate the extra parameter. A random forest also beats the retained model (3.55 vs 3.63 pp). The 2-parameter capability-gap models of [Panfilov et al., ICLR 2026] and [Kale et al., ICLR 2026] are neither compared against nor cited, even though Section 7’s ECI trend closely mirrors their findings.
  3. Much of attacker sabotage capability sits outside the main predicted quantity. The model predicts monitor ROCs on a fixed corpus of trajectories. In the main safety-prediction comparisons, attack rate (0.01), horizon (N = 1000) and conditional side-task success (0.5) are the same for every model, the auditor is assumed perfect, and side-task success is not separately predicted. Opportunity selection is partially linked to ROC prediction, but pacing and Appendix E’s full strategy ladder are not validated as part of an integrated five-parameter prediction pipeline. What the model captures is mostly how suspicious an attacker’s actions look to a monitor, not its full ability to sabotage. The “safety predictions” agree with probabilities computed under this assumed game; they do not directly estimate deployment risk or establish deployment-calibrated thresholds.
Questions for authors
  1. In the transfer evaluation, does the held-out attacker (or monitor) have any destination-environment measurements with other partners? If so, what MAE does the within-destination model alone achieve on the same held-out pairs?
  2. Can you report cold-start results, i.e., prediction error for an entirely unseen attacker given k measured monitors, as a function of k?
  3. How well are attacker parameters (or evasion scores) rank-correlated across the three environments?
  4. In SHADE-Arena, h_a is nearly identical for all attackers. Is the shift parameter identified there, and how often does the inner penalty search effectively zero out the interaction?

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of key concerns and response

We thank all four reviewers for their specific and helpful reviews. We summarize the concerns of the reviewers and our response below:

  • Q1. Concern: Why this functional form? The parameterization needs a principled motivation
    Response: We have updated our model to improve its theoretical backing, interpretability, and minimality. See details below [Q1]
  • Q2. Concern: The cross-environment treatment is ad hoc, and transfer is not clearly supported
    Response: We replaced the polynomial transfer maps with three numbers per arena. A new arena needs about four measured pairs before learning its numbers breaks even. With only 1–2 pairs, learning them is worse than not learning them (8.6 / 7.8 vs 6.5 pp); with 4 pairs it ties (6.6 pp), and with 8 pairs (SHADE-Arena only) it improves slightly, to 6.0 ± 0.7 pp, within error.
  • Q3. Concern: The grid over attacker behavior (selection, pacing, adaptation) is incomplete
    Response: Agreed. We now measure two of these directly. Opportunity selection lowers catch by 2–4 pp at 0.1% FPR, mostly by changing which trajectories occur. On rate-setting, models do worse setting their own attack rate than when the evaluator sets it. Pacing and adaptation to the monitor are left for future work (details in our reply to Dexu Q1).

Q1. We redeveloped a more compact model with a statistically principled motivation. Our model rests on three claims about the score distributions, each of which we support below.

(i) Honest reference coordinate: we map each monitor's scores through its own honest quantiles onto a Gaussian scale (so honest actions are standard normal: mean 0, variance 1), z = Φ⁻¹(H(s)). This assumes nothing about the honest shape: an ROC curve is unchanged by any monotone transformation of the scores, so the choice of coordinate loses no information, and a logistic coordinate gives the same held-out error within 0.1 pp.

(ii) Sabotage peak: once we've normalized the honest distribution, sabotage actions form a shifted Gaussian peak, plus a small share above the honest maximum. We use a Gaussian because, given only a location and a spread, it is the maximum-entropy (least-committal) choice. A monitor's score also pools many weak cues, which pushes toward a Gaussian. Some sabotage actions are simply egregious to the monitor and score above every honest action. We model these as a separate share rather than as the Gaussian's tail.

(iii) Honest padding share: these are the actions that look honest to the monitor. We treat this share as a latent quantity and estimate it as the largest honest-shaped component the attack data allows. This is the standard null-proportion estimate for a two-component mixture with one known component [3]. To test this assumption, we blind hand-labelled 1,379 actions across the three arenas, and see that only 8 of the 852 low-score actions (i.e. actions that seemed honest to the monitor) were sabotage. Conversely, 78% of the actions scored in the top band or above the honest maximum were sabotage or setup for it.

Concretely, in honest-normalized units z, an attacker–monitor pair's attack actions follow

p(z)=w ϕ(z)+(1−w)[(1−ε) 1σ ϕ(z−μσ)+ε δ>max⁡]p(z) = w\,\phi(z) + (1-w)\Big[(1-\varepsilon)\,\tfrac{1}{\sigma}\,\phi\big(\tfrac{z-\mu}{\sigma}\big) + \varepsilon\,\delta_{>\max}\Big]

where we have the constants σ = 1.17, intercepts logit w₀ = 0.30, μ₀ = 2.16. The 1 attacker variable, 2 monitor variable, and 3 environment variables act on different values of this distribution:

NumberWhat the number shiftsFitted range
Attacker t (1 per attacker)peak μ by +t and logit w by −0.73·t: a stealthier attacker has a lower peak and more honest-looking actions−0.60 to +0.69
Monitor b (1 per monitor)peak μ by +b, and the above-max share ε (logit slope 5.4)−0.82 to +0.38
Monitor w (1 per monitor)logit w, the honest-looking share−0.47 to +1.42
Arena (3 per arena)logit w, μ and logit εw −0.18 to +0.22; μ −0.20 to +0.26; ε ±3

How well does the new model fit our data? It fits comparably (4.5 pp across 0.8-50% FPR, against the 3.3-4.2 pp number) to our model in the original draft of the paper, while using one number per attacker instead of three, and three environment variable instead of 19.

[1] A. Panfilov et al. Capability-Based Scaling Trends for LLM-Based Red-Teaming. ICLR 2026.

[2] N. Kale et al. Reliable Weak-to-Strong Monitoring of LLM Agents. ICLR 2026.

[3] G. Blanchard, G. Lee and C. Scott. Semi-supervised novelty detection. JMLR 11, 2010.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.