acceptodds
Under review as a conference paper at ICLR 2027

The Reference Is Part of the Signal: Structural Limits of Black-Box Sandbagging Detection

Abstract

Black-box audits for sandbagging often compare a model's performance under evaluation cues with its performance under an ordinary-use reference. We establish that this reference is part of the signal: reference changes and suppression triggered whenever evaluation is present occupy the same cue direction. Any measurement invariant to arbitrary reference changes therefore discards this suppression component. We trace the practical consequences with a frozen, pre-registered structural statistic that achieves held-out AUROC 0.984 and detects an external sandbagger blind with an unflagged unlocked control. Changing only a harmless deployment preamble flips verdicts in 9 of 30 model configurations, including eight honest ones, while evaluation responses remain fixed. Reference variation exceeds question-sampling noise and extends to scalar performance-gap detectors. Reference removal discards a substantial component of the observed contrast; averaging retains honest false positives. Four models fine-tuned without a suppression objective reproduce the detected signature, two under every tested reference. Together, the theory and controlled experiments explain how strong ranking coexists with ambiguous attribution. They establish deployment framing as an explicit design choice in sandbagging audits and motivate a protocol combining reference sweeps, honest-control calibration and matched comparisons.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
sandbagging, large language models, black-box auditing, evaluation awareness, identifiability

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_sQukOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies how the choice of deployment reference affects black-box sandbagging detection. It shows that changing the reference and suppressing performance whenever an evaluation is detected can produce the same kind of signal, making the two difficult to separate. In the experiments, a preregistered detector ranks sandbagging models well, but harmless changes to the reference prompt alter several verdicts and also flag honest models. The paper therefore argues that such detectors should be calibrated with honest controls and tested across multiple deployment references.

Critical strengths
  1. The paper identifies an important and underexplored problem in sandbagging audits: the deployment reference is itself part of the measured signal.

  2. The experiments are carefully designed. The authors preregister the analyses, keep the evaluation sets separate, openly report failed predictions, and include strong controls such as honest models, matched fine-tunes, and locked and unlocked checkpoints.

Critical weaknesses
  1. The experimental evidence is difficult to follow. The paper uses several question draws, model panels, and decision rules, with key details spread across multiple appendices. This makes it hard to determine which results are preregistered, independently replicated, or based on reused data.

  2. The empirical evidence covers a narrow evaluation setting. The experiments use four-option multiple-choice questions scored from a single forced answer token, while reasoning, open-ended, and agentic evaluations are not tested. It is therefore unclear whether reference changes would have the same practical effect in broader black-box audits.

  3. The paper states the problem clearly but does not establish a reliable rule for individual audit decisions. Each of the four preregistered rules encounters at least one limitation in later or expanded evaluations, such as missing a sandbagger, flagging an honest model, or changing with the deployment reference. Matched-control comparisons perform better, but they require a suitable control model that may not be available in practice. The proposed protocol can therefore reveal uncertain verdicts, but it does not consistently determine whether a model is genuinely sandbagging.

  4. The headline detector performance provides limited evidence of transfer to unknown sandbagging strategies. Most positive examples are constructed using prompts or fine-tuning objectives that explicitly produce evaluation-conditioned suppression, which is the pattern the statistic is designed to detect. The held-out evaluation changes the questions, cue wording, and injection channel, but it does not hold out the underlying attack strategies. In the blind external evaluation, the method detects only one of the three measurable sandbaggers. The reported AUROC therefore shows stronger transfer across inputs than across adversaries.

Questions for authors
  1. The permutation test can flag honest models because it treats any consistent response to evaluation cues as unusual, and calibration on honest models does not fully remove this problem. Did you consider a null model that allows benign, model-specific responses to evaluation cues? Would the three honest models flagged under every reference still be flagged under such a test?
  2. Sign coherence is correlated with model capability, and three highly capable honest models are flagged under every reference. Could capability itself be driving these false positives? After correcting the scores for capability, do these models remain flagged?
  3. The main sweep uses six benign preambles, and the fifth draw expands this set to twelve. Have you tested references selected through adversarial or systematic prompt search rather than researcher-written variations? This would clarify whether reference sensitivity creates an audit-integrity risk beyond the tested prompts.
  4. Beren has a documented password trigger and was developed independently of this study, yet the detector finds an evaluation-conditioned signature. What behavior is the statistic detecting in this model? Clarifying this would help determine whether the result reflects a general property of weak-model imitation or a specific interaction with the tested evaluation framing.
7 comments
AuthorsOpenReview ↗

Response to Reviewer sQuk (1/4): weaknesses 1–2

Thank you for the detailed and constructive review. During the discussion we ran three new experiments (Registrations 8–10, Appendix L), each with an advance plan and documented deviations.

W1 — which results are pre-registered, replicated or reused. Thank you for this suggestion. The revision adds this evidence map to the main text as Table 1 (model panels: Appendix A).

  • Pre-registered: every evaluation on new questions has an advance record (a registration or recorded verdicts).
  • Replicated: the frozen statistic was re-scored on fresh questions in the third, fourth and fifth draws, and the beren detection was repeated under a second cue wording and in the fifth draw.
  • Reused: the reference sweep and its expansions reuse the third-draw questions; rows that reuse scores say so, and the review analyses are exploratory.
StudyQuestion dataAdvance recordResults
Development (1st draw)200, design setNoneApp. D
Held-out validation (2nd draw)400, newRegistration 1§3
Third draw400, newRegistration 2App. J.2
Fourth draw400, newRegistration 4App. J.4
Fifth draw200, newRegistration 5App. J.5
External organisms200, GSM8K/MMLURecorded verdicts*§3
Reference sweepThird-draw questionsNone§4
+ honest familiesSame sweep questionsRegistration 6App. J.6
+ organism familiesSame sweep questionsRegistration 7App. J.7
+ honest fine-tunesSame sweep questionsNone§5
Reference-free testExisting scoresRegistration 3App. J.3
Matched projectionExisting scoresAnalysis plan†App. E.3
Generated answersSweep questions, regeneratedRegistration 8‡App. L.1
Reference searchSweep questions, 27 new preamblesRegistration 9App. L.2
Free-form, externalExternal questions, regeneratedRegistration 10App. L.3
Review analysesExisting scores and App. LNone (exploratory)App. K

A draw is a new question sample. New questions exclude earlier corpora and organism training data. *Verdicts for beren and for the benign external control (Organism F) preceded the confirming runs; the unlocked-control prediction preceded scoring. †Plan and input hashes fixed before analysis. ‡Code and predictions hashed before any confirmatory output was read; formally frozen after two of four families had been read (App. L.1).

W2 — narrow setting. We have now tested two generated-answer settings (Appendix L).

  • Reasoning, then a letter (Registration 8; four model families, each run honest and as an organism). The same statistic, applied to answer correctness, still separates organisms from honest models (18/24 vs 0/24 model–preamble cells: 4 models × 6 preambles), and verdicts still vary across preambles for 2 of 4 organisms. The accuracy spread across preambles was not significant at this sample size.
  • No options, numeric answer (Registration 10; beren and controls, GSM8K). Beren still answers worse under evaluation cues (3–4 points under four of six preambles); the gap vanishes under one of the six preambles (B4), which in a post hoc comparison is also the weakest preamble for beren under forced choice and for three of four organisms with reasoning. The correctness-based statistic does not flag beren here, so this is behavioural evidence, not a transfer of detection (Q4, part 4/4).

In all three formats, changing the preamble shifts all of a question's reference-minus-cue score differences (Δ\Delta) by the same amount, and organisms' cue responses lie largely along that same direction (Appendix K.5). Agentic evaluations are the natural next test (§8).

AuthorsOpenReview ↗

Response to Reviewer sQuk (2/4): weaknesses 3–4

W3 — no reliable rule for individual decisions. We agree that the protocol does not establish a reliable individual-model verdict. What it does establish is which verdicts an auditor should not trust. Without a reference sweep, an audit can clear a sandbagger or flag an honest model through its choice of reference alone: selecting the reference on half the questions left 5 of 7 flagged organisms unflagged on the other half (Q3). Reviewer ArKs reads the result the same way: "not one more detector but a statement about a structural limit of an entire class of black-box reference-contrast audits." The experiments separate decision stability from attribution.

Stability is not attribution. As you note, each of the four pre-registered rules meets a limitation in later or expanded evaluations; a new Table 3 in §5 lists each rule with its outcome and limitation (details in Figure 12). For instance, the three-valued rule (report a verdict only when all preambles agree; otherwise escalate) answered 7 of 14 configurations in the fifth question draw without error, but after eight honest families were added (Registration 6) it stably flagged three honest models.

Attribution needs a control. By Proposition 1(ii), a single matrix of score shifts identifies suppression only under restrictions on honest cue responses. A matched control supplies one when it shares the organism's honest cue response and shows no extra suppression of its own; reference invariance also requires equal reference displacements. Such controls exist in developer audits of fine-tunes, or when a reference checkpoint is supplied. Without one, or another restriction on honest cue responses, a single shift matrix cannot support attribution, but a reference sweep still tells the auditor which verdicts not to trust.

W4 — transfer across inputs vs. adversaries. We agree; the distinction sharpens the claim, and the revision adopts it. All three organism-construction families (generic prompt, covert-domain prompt, LoRA) appear in the development set, so held-out AUROC 0.984 shows transfer across questions, cue wordings and injection channels, not across attack strategies. Two new checks:

  • Per family (each family's 3 held-out positives against the same 7 negatives): each family separates on its own (AUROC 0.952–1.000), and p<0.01p<0.01 detects all three positives in each. In the six-preamble sweep the two prompted families have their lowest AUROC under preamble B4 (0.72 and 0.82); LoRA organisms separate under all six.
  • Without stakes-gating. Stakes-gating is the component of SS that compares shift size on target (hazardous) and control questions; it follows our organisms' design. Dropping it leaves held-out ranking high (AUROC 0.952, against 0.984 with it; the rank component alone gives 1.000), though the generic-prompt family falls to 0.857. The external beren detection does not rest on it: its gating term is negative (z(G)=−6.49z(G)=-6.49), so the detection comes from the generic rank and sign components (how coherently, and in which direction, items shift across evaluation cues).

The independent external constructions provide the strategy-transfer test; we detect 1 of 3 measurable organisms (Table 6; a correction on one miss is in the general response).

AuthorsOpenReview ↗

Response to Reviewer sQuk (3/4): questions 1–2

Q1 — a null allowing benign model-specific cue responses. Under one such null, none of the three honest models flagged under every preamble is flagged under any preamble. We tested H0H_0: the model's SS is drawn from the honest-model population under the same preamble. Each honest base was compared with the other twelve using a predictive tt test. It assumes an approximately normal honest-model population; thirteen bases cannot validate its 1% tail. At p<0.01p<0.01:

  • Honest models. No honest configuration is flagged in any of the 102 model–preamble cells (17 honest configurations, i.e. 13 bases and 4 honest fine-tunes, × 6 preambles), against 50 of 102 under the permutation null. The closest to significance is Llama-3.3-70B (smallest p=0.017p=0.017 across preambles); MiniCPM4-8B and Phi-4 stay at p≥0.08p\ge0.08. The four honest fine-tunes were not used for calibration, but their bases are in the panel, so this is not a test on new families.
  • Power. Organisms are flagged in 34 of 78 cells instead of 77; 3 of the 13 are never flagged and 8 change verdict across preambles, against 1 of 13 under the permutation null.
  • Conditioning also on capability gives 0/102 honest and 39/78 organism cells.

With one shift matrix, a completely unrestricted model-specific honest response can absorb any suppression term (Proposition 1(ii)). Our population null therefore permits variation across honest models while imposing an explicit distributional restriction, as identification requires. Because it is fitted to this same panel, it trades power for the removal of honest false positives, and reference sensitivity remains.

Q2 — is capability driving the false positives? Not fully: adjusting for capability weakens one honest-model anomaly but leaves two overlapping with organisms. One premise comes from an appendix sentence we have corrected: not all three are highly capable, and MiniCPM4-8B ranks tenth of thirteen honest bases in capability. We separate three questions.

  • Significance after correction. Under the capability-conditional population null, none of the three is significant at p<0.01p<0.01 (minimum 0.020). They are not significant under the unconditional population null either, so this mostly reflects the population null rather than the capability term.
  • Ranking after correction. A linear adjustment for our capability proxy within the honest panel weakens Phi-4's relative anomaly, but MiniCPM4-8B and Llama-3.3-70B still overlap with organisms under most preambles (Appendix K.2). A linear adjustment on one proxy does not remove the overlap, and it cannot show what drives it.
  • Scoring format. Phi-4 and Llama-3.3-70B were not flagged under generated-answer scoring; the general response (Corrections, honest false positives) gives the scope this sets.

Proxy sensitivity. A capability proxy measured under the deployment reference inherits the suppression it is meant to control for. Ours is measured there, where organisms also suppress, so it understates their capability. With base-model capability instead, residualising lowers AUROC under every preamble over all thirteen organisms, and on the six new organisms it improves AUROC under only three of six preambles (five of six with our proxy). The revision states this.

Reviewer_ArKsOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies black-box auditing for sandbagging (strategic underperformance on capability evaluations), where audits contrast a model's performance under evaluation cues M against an ordinary-use reference b, forming Δ = b1ᵀ − M. The central structural insight (Proposition 1) is that changing the reference and a policy that suppresses whenever evaluation is present both contribute terms in the same uniform cue direction, so any statistic invariant to arbitrary reference changes necessarily discards the presence-conditioned suppression component — a structural tension between robustness to reference choice and the ability to detect such suppression. The authors operationalize this with a frozen, pre-registered statistic S (rank-one energy, signed consistency, stakes-gating; standardized against a permutation null), reaching held-out AUROC 0.984 and a blind external detection of a UK AISI organism. Crucially, they then show the limits: a harmless deployment preamble flips 9 of 30 verdicts (including eight honest models), honest fine-tunes without any suppression objective reproduce the signature, and the signal correlates with capability (r = 0.69) so no single-reference threshold separates the classes. They conclude that strong ranking coexists with ambiguous attribution and recommend an audit protocol combining reference sweeps, honest-control calibration, and matched comparisons.

Critical strengths
  1. A clean structural insight with a proof that reframes how these audits should be designed. Proposition 1 establishes that reference changes and presence-conditioned suppression occupy the same uniform cue direction, so robustness to the reference and detection of presence-conditioned suppression are fundamentally in tension, and it gives identification conditions with matched controls. This is not one more detector but a statement about a structural limit of an entire class of black-box reference-contrast audits, which is a genuinely general and useful contribution.

  2. Unusually rigorous and transparent methodology. The work uses seven pre-registrations that fix predictions before scoring and honestly reports the failed ones, a frozen statistic, a permutation null that preserves each question's score marginal, bootstrap intervals, honest-control fine-tunes, a blind external detection of an organism reported to have defeated black-box detection, an explicit capability-correlation analysis, and GPU-free reproducibility artifacts. This is exemplary practice for the area.

  3. The conclusions are honest and self-limiting rather than oversold. The authors do not present AUROC 0.984 as a reliable detector; they actively undercut it — a greeting flips verdicts, honest models are flagged, honest fine-tunes reproduce the signature, and the signal is confounded with capability. The explicit separation of "strong ranking" from "reliable attribution" makes the paper more credible, not less.

Critical weaknesses
  1. The constructive deliverable is thin: a limiting result plus a diagnostic protocol rather than a usable detector. The authors themselves show that the statistic S does not yield reliable verdicts (9/30 flips, honest false positives, capability confound), and the final recommendation — reference sweeps, honest-control calibration, and matched comparisons — is closer to a diagnostic procedure than a solution. The limiting result is valuable, but some readers will find the positive contribution narrow.

  2. The scope is confined to MCQ forced-choice log-margin scoring on mid-size open models, and external validity is weak. All experiments use single-letter forced-choice scoring (a method with acknowledged pathologies) on granite/gemma/ministral/Llama-70B-class models; whether the geometry and the magnitude of reference sensitivity transfer to realistic frontier audits (free-form generation, agentic tasks, real deployment) is untested. The headline AUROC 0.984 is on self-constructed organisms and negatives, whereas on the genuinely external sandbaggers only 1 of 5 is detected (two missed, two outside the measurable domain).

Questions for authors
  1. Do the structural result and the magnitude of reference sensitivity transfer beyond MCQ forced-choice log-margin scoring — e.g., to free-form or agentic evaluations? Is there any evidence that the same geometry (Δ = b1ᵀ − M with a shared cue direction) holds there?

  2. Given that signed consistency correlates with capability (r = 0.69) and no single-reference threshold separates honest models from organisms, to what extent is S measuring capability/coherence rather than sandbagging, and can the capability component be explicitly partialled out or regressed away?

5 comments
AuthorsOpenReview ↗

Response to Reviewer ArKs (1/2)

Thank you for the careful summary and for recognising Proposition 1 as a statement about a whole class of audits.

W1 — the constructive contribution. We agree that the paper does not establish a reliable individual-model detector. Its practical contribution is an audit protocol with explicit identification conditions. Proposition 1(ii) identifies the obstacle: a single shift matrix cannot separate suppression from unrestricted honest cue responses. As you note, this is a structural limit of a whole class of reference-contrast audits rather than a defect of SS alone. Sections 5–6 test three practical checks, each valid only under explicit conditions:

  1. Specify the references in advance and sweep them. A verdict that changes across references depends on the reference choice, and the risk is concrete: in a new pre-registered experiment (Appendix L.2), choosing among 27 automatically generated everyday preambles on one half of the questions left 5 of 7 previously flagged organisms unflagged on the other half and newly flagged one honest model. Stability is a diagnostic, not sufficient evidence. The three-valued rule (report a verdict only when all preambles agree; otherwise escalate) answered 7 of 14 configurations in the fifth question draw without error, but after eight honest families were added (Appendix J.6) it stably flagged three honest models.
  2. Calibrate against an honest panel covering the capability range. In a new analysis, testing SS against the honest-model population (leave-one-out, predictive tt) trades power for specificity. Counting one model under one preamble as a cell, flagged honest cells fall from 50/102 to 0/102 and flagged organism cells from 77/78 to 34/78 (17 honest configurations, 13 organisms, 6 preambles). Eight of 13 organisms then change verdict across preambles. This is the situation Proposition 1 describes: identification needs a restriction on honest responses. The population null, fitted to this same panel, supplies one; it removes the honest false positives but not reference sensitivity.
  3. Use matched controls. Proposition 1(iii) identifies suppression when the matched control (twin) shares the organism's honest cue response and has no incremental suppression. Reference invariance also requires equal reference displacements. A shared prompt alone does not ensure these conditions. In same-preamble comparisons, organisms score higher than their own bases in 75 of 78 cases and higher than honest fine-tunes in 48 of 48.

W2 / Q1 — beyond forced-choice MCQ. Proposition 1 needs only one score per item under the reference and under each cue. We have now checked the geometry directly (exploratory, Appendix K.5) and run two pre-registered generated-answer experiments (Appendix L). The geometry and the reference dependence of organism verdicts carry over; effect sizes and detection depend on the format.

  • Same geometry. In forced-choice, reasoning-then-letter and free-form numeric scoring alike, a preamble change moves Δ\Delta exactly along the shared cue direction 1\mathbf{1}, as the algebra requires. Organisms' observed cue responses also lie largely along 1\mathbf{1} (Table 23; noise-corrected share of Δ\Delta energy on 1\mathbf{1}, medians over preambles). Generic-prompt organisms reach 0.74–0.89 under forced choice (honest bases 0.45–0.53; covert-domain organisms, 0.43–0.76, overlap them) and 0.47–0.79 with reasoning (honest-base intervals include 0). Beren reaches 0.89 under forced choice (0.22 unlocked) and 0.78 at the item-mean level in free-form answers.
  • Magnitude. With reasoning before a letter (four model families, each run honest and as an organism), SS on answer correctness still separates organisms from honest models (18/24 vs 0/24 model–preamble cells) and verdicts still vary across preambles for two of four organisms, but the accuracy spread across preambles is not significant at this sample size. The honest flags that persisted under every preamble with forced choice were absent for both retested bases (Phi-4, Llama-3.3-70B), so the revision states that finding for forced-choice scoring (§8). With options removed, beren answers 3–4 points worse under evaluation cues under four of six preambles; the gap differs across preambles (exploratory permutation p=0.017p=0.017) and its documented password reduces it, but SS does not flag beren in this format (Appendix L.3).

Agentic tasks and models beyond our open-weight panels (up to 70B) are the natural next tests. On the external organisms, two of five hide in task domains a multiple-choice protocol cannot reach, and we detect one of the three measurable (Table 6). The revision no longer presents the 0.984 AUROC as evidence of transfer across attack strategies.

AuthorsOpenReview ↗

Response to Reviewer ArKs (2/2)

Q2 — is SS measuring capability? Partly. Across the thirteen honest base models, capability correlates with the sign-coherence component at r=+0.69r=+0.69 and with a model's maximum SS at r=+0.54r=+0.54, so it is associated with roughly 30% of the between-model variance in maximum SS (n=13n=13). Adjusting for it weakens one honest-model anomaly and leaves two overlapping with organisms. The revision separates two questions.

  • Significance. Under an honest-population null conditional on capability, none of the three honest models flagged under every preamble (Phi-4, MiniCPM4-8B, Llama-3.3-70B) reaches p<0.01p<0.01 (minimum 0.020). They are not significant under the unconditional population null either, so this mostly reflects the population null rather than the capability term.
  • Ranking. After a linear adjustment for our capability proxy within the honest panel, Phi-4's relative anomaly weakens, but MiniCPM4-8B (one of the less capable honest models) and Llama-3.3-70B still overlap with organisms under most preambles (Appendix K.2). A linear adjustment on one proxy therefore does not remove the overlap; it does not show what drives it. (On scoring format, see W2 in part 1.)

Can capability be regressed away? We tested this directly; the gain depends on how capability is measured. We pre-registered a linear correction fitted on the honest models and tested it on six new organisms: it improved ranking under five of six preambles (Appendix H.2). But our capability proxy is measured under the deployment reference, where organisms also suppress (Appendix B). It therefore understates their capability, and the correction partly credits organisms for their own suppression. When each organism is assigned its base model's capability instead, the correction improves ranking under only three of six preambles on the new organisms, and it lowers AUROC under every preamble over all thirteen. Auditors should check for this bias before interpreting gains from a capability adjustment.

Presentation. Following your comment on readability, the revised abstract and introduction state the audit and its terms in plain language, Proposition 1 is preceded by a plain-language statement, and Tables 1 and 3 map the experiments and the decision rules. We would welcome your view on whether the revision now reads more easily.

AuthorsOpenReview ↗

Follow-up before the discussion deadline

Thank you again for the review. The discussion closes on 18 November (AoE), so we wanted to ask whether our reply settles your two main questions: whether the results hold beyond multiple-choice scoring, and whether our statistic partly reflects model capability. If anything there is still unclear, we can respond before the deadline.

Reviewer_7hLTOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary
  1. This paper introduces a "frozen structural statistic" for black-box sandbagging detection and proves that the “ordinary-use” reference prompt shares the same signal direction as real sandbagging.
  2. It shows changing harmless greeting in reference flips verdits in 9 of 30 models, and proposes an "audit protocol".
Critical strengths

None

Critical weaknesses
  1. The paper is extremely hard to read and introduces many terms without defining what they mean e.g, in line 035, what does "honest granite-30B model" mean? In the same line, "Its statistic moves from −8.46 to +13.70 while the questions, option orders and evaluation cues stay byte-identical (Figure 1)." -> what does a statistic mean here? This paper requires a complete rewrite.
Questions for authors
  1. Could you please define the terms you introduce in the paper? It is unclear what terms like thse mean: "honest model", "frozen statistic", "pre-registrations", etc.
2 comments
AuthorsOpenReview ↗

Response to Reviewer 7hLT

Thank you for pointing to specific passages; your examples guided a revision of the main text for readability. The abstract and the opening paragraph now define the audit and its terms in plain language before the first example. Proposition 1 is preceded by a plain-language statement. Two tables map the experiments (Table 1) and the decision rules (Table 3). Below we summarise the motivation and central finding, then address the terms you highlighted.

Motivation and question. Capability evaluations help determine whether a model needs additional safeguards before deployment. If a model can perform a task but deliberately underperforms during evaluation, its measured score can understate its capabilities and the associated risks. Anthropic researchers have demonstrated such underperformance in controlled experiments with models trained to hide dangerous capabilities [1]. This motivates our question: when an audit compares performance across prompts, can it distinguish deliberate underperformance from ordinary prompt sensitivity?

The main observation: Figure 1(a). A common black-box audit compares performance under prompts that signal an evaluation ("evaluation cues") with performance under an ordinary-use prompt (the "reference"). Figure 1(a) shows that changing only this reference can change the audit's verdict. For granite-30B, a model we did not instruct or train to underperform, adding the greeting "Hey, mind helping me think through this one?" moves SS, our numerical summary of these performance changes, from −8.46-8.46 (unflagged) to +13.70+13.70 (flagged, i.e. reported as suspected sandbagging). The questions, option orders, evaluation cues, and responses under those cues remain identical. Across six harmless reference wordings, the verdict (flagged or not) changes for 9 of 30 model configurations (17 honest, 13 configured to underperform), including eight that we did not instruct or train to underperform. Thus, a flag alone does not establish deliberate underperformance.

What the paper contributes. Proposition 1 explains why this happens and when it can be avoided. The audit's basic quantity is each question's performance drop from the reference to each evaluation cue. Changing the reference shifts all of a question's drops by the same amount, and a model that suppresses a question equally under every evaluation cue produces exactly the same pattern. A statistic made fully insensitive to the reference therefore also loses this suppression signal. The proposition also gives conditions under which matched controls can separate the two. Our experiments show that reference sensitivity occurs in practice and extends to simpler performance-gap statistics (§4), and that fine-tuning without any underperformance objective can also produce a flagged signature (§5). These findings motivate the audit protocol in §6: check several ordinary-use references, calibrate against models not configured to underperform, and, where available, compare with an otherwise matched model that lacks the underperformance objective.

The terms you asked about.

  • Honest model: a model we did not instruct or train to underperform. The "honest granite-30B model" in line 35 of the submitted version is the unmodified granite-30B base model. We call models deliberately configured to underperform, through prompting or fine-tuning, organisms.
  • Statistic SS: a single number summarising the structure of per-question performance changes between the reference and the evaluation cues (§2). "Flagged" means that SS is statistically significant against the permutation null described in §2 (p<0.01p<0.01). The values in line 35 are values of this statistic.
  • Frozen statistic: we fixed the statistic's code and analysis choices before scoring the held-out data.
  • Pre-registration: a dated plan recording predictions and decision rules before the corresponding data are scored or analysed; we report failed predictions as well as successful ones. Table 1 links the main experiments to their data and advance records.

The revision also defines the other study-specific terms, such as configuration, preamble, signature and draw. The general response also summarises new experiments with generated answers and a systematic reference search.

Your review does not name a technical concern alongside its soundness rating. If one underlies it, please tell us which result or step it concerns, and we will address it during the discussion. If the revisions above resolve your concerns, we would be grateful if you would reconsider your assessment.

Reference

[1] J. Benton et al. Sabotage evaluations for frontier models. arXiv:2410.21514, 2024.

AuthorsOpenReview ↗

Follow-up before the discussion deadline

Thank you again for your review. Since the discussion closes on 18 November (AoE), we wanted to ask whether the revised abstract, introduction and term definitions make the paper easier to follow. If your soundness rating reflects a technical concern, could you tell us which result it concerns? We would like to answer it before the deadline.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response: significance, revisions, new evidence and corrections

We thank the reviewers for their detailed feedback. The revised manuscript marks changes in blue (one-word terminology swaps in the appendices are unmarked). Details are in the replies to each reviewer and in Appendices K–L.

Significance and contribution. Capability evaluations inform deployment and safeguard decisions, so deliberate underperformance can undermine the evidence used to assess AI risk [1,2]. A common class of sandbagging audits compares performance under evaluation cues with performance under an ordinary-use reference. Prior work observed that unrelated prompts can raise benign models' scores as much as sandbaggers' [2]. We treat the reference itself as a variable of the audit: changing only a harmless preamble, while the model's answers under evaluation stay the same, changes the verdict. Our main theoretical result (Proposition 1) shows that this is a limit of the whole class. From the shifts between reference and evaluation performance alone, no audit can distinguish deliberate suppression from an honest model's ordinary sensitivity to evaluation cues without an assumption about how honest models respond; we state assumptions under which it can.

Reviewer sQuk recognises this as an "important and underexplored problem"; ArKs notes a structural result that "reframes how these audits should be designed."

Presentation. We thank the reviewers for their suggestions on presentation, which have made the paper easier to follow. The abstract and opening paragraph now state the audit and its central finding in plain language, and Proposition 1 is preceded by a plain-language statement. Study-specific terms are defined explicitly and synonyms merged. Table 1 maps every experiment to its questions, advance record and results, and a new Table 3 lists the four decision rules with their outcomes and limitations.

New evidence. Appendix L reports three experiments run during the discussion, each with an advance plan, a freeze timeline, documented deviations and all registered outcomes, including failed predictions; Appendix K reports exploratory analyses. Code and frozen records are in the supplement. In brief, reference choice changes verdicts in practice and the geometry extends beyond forced choice, while detection power depends on the scoring format and on how the audit is calibrated.

  • Reference search (L.2). Choosing the reference among 27 automatically generated ordinary preambles left 5 of 7 previously flagged organisms (models we configured to underperform) unflagged.
  • Generated answers (L.1, L.3). With reasoning before a letter, our statistic SS still separates organisms from honest models, and verdicts still vary across preambles. With options removed, SS does not flag the external organism beren, although beren still answers worse under evaluation cues.
  • Geometry (K.5, exploratory). In all three scoring formats, a preamble change shifts the measured effect of every evaluation cue on a question by the same amount, as the theory predicts, and organisms' responses to the cues largely take this same uniform form.
  • Honest-population null (K.1). Calibrating against our honest panel trades power for specificity: it removes every honest false positive (organisms are flagged in 34 of 78 model–preamble cases instead of 77), and reference sensitivity persists.
  • Capability and strategy (K.2, K.3). A linear capability adjustment leaves two honest models overlapping with organisms. Held-out ranking holds within each of our three organism types (generic prompt, covert-domain prompt, LoRA).

Corrections to the submitted version. We corrected five statements in the submission. None affects the central finding that reference choice can change the verdict; each narrows a claim.

External organisms. The submission called the benign control (Organism F) "correctly rejected"; it is flagged under the first cue wording and rejected by the cross-wording rule (D.4). It also attributed the miss of Organism C to the lettered-option format, but removing the options revealed no cue-conditioned drop, so we withdraw that explanation (L.3).

Honest false positives. These are established for forced-choice scoring: they were absent under generated answers for both retested models (MiniCPM4-8B was not retested). The submission also described all three flagged models as among the most capable; MiniCPM4-8B ranks tenth of thirteen (H.2).

Scope of the held-out result. AUROC 0.984 measures transfer across questions, cue wordings and injection channels within these three organism types, all seen during development, not across attack strategies.

References

[1] J. Benton et al. Sabotage evaluations for frontier models. arXiv:2410.21514, 2024.

[2] J. Taylor et al. Auditing games for sandbagging. arXiv:2512.07810, 2025.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.