acceptodds
Under review as a conference paper at ICLR 2027

Search-Aware Selection of Verification Protocols for Scalable Oversight

Abstract

Scalable oversight asks weaker judges to evaluate stronger agents, but how reliable a judge is also depends on how the agent spends test-time compute. We study two forms of agent scaling with frozen models: best-of-N search, in which the agent ranks its own incorrect candidates by predicted acceptability without querying or identifying the deployed judge, and longer thinking, in which it reasons with a larger token budget. Across logical and mathematical reasoning and science, we find that judge failures are persistent and problem-specific, so the agent can find them without judge access. Increasing search from one to 64 candidates raises false acceptance by 1.8–5.7x. Longer adversarial thinking also raises it, though on most tasks by less than search at matched token budgets. No single verification protocol stays reliable across tasks, and the cheapest adequate protocol can change the form of verification rather than just its effort. We prove a sharp bound on how much false acceptance can grow as search increases. Building on it, we introduce SARCO (Search-Aware Risk-Controlled Oversight), which calibrates at a small search budget and uses statistical tests on held-out calibration problems to return the cheapest protocol that keeps both false and honest acceptance within target at a larger deployment budget, or abstains. On disjoint problems at search budgets of 16 to 64, every protocol SARCO returns meets both targets, whereas empirical risk-forecasting selectors fail one or both targets in 8–16% of their choices. This assurance comes at the cost of coverage: SARCO abstains when the tested protocols cannot be certified from limited calibration data, including on science tasks where no tested protocol meets both targets. For longer thinking, protocol choices certified at a low reasoning budget can fail at higher budgets, so they must be recalibrated near the deployment budget. Our results suggest that oversight sufficiency depends on how submissions are generated and selected, not only on the agent–judge pair.

Primary area:
foundation or frontier models, including LLMs
Keywords:
scalable oversight, metacognition, LLM verifiers

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_VpVKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper asks how oversight should change when a strong agent spends more test time compute trying to get a wrong answer past a weaker judge. It examines 2 cases: (1) best-of-N, where the agent picks the candidate it thinks the judge will most likely accept, and (2) longer thinking before a single answer. In both of these cases the agent never queries the judge. Using a Qwen3 (14B) agent and mostly Qwen3 (4B) judges, the paper finds that FAR rises 1.8-5.7x from N=1 to N=64 - and that judge failures concentrate on a minority of problems that the agent can find on its own. The main theorem bounds FAR at a large search budget using only FAR at a smaller one. SARCO uses this bound with standard risk-control tests to pick the cheapest verification protocol that meets both the false-acceptance and the honest-acceptance targets, or otherwise to abstain. On held out problems, every protocol SARCO returns meets both targets, though it returns 1 in only 27.5% of decisions at N=64. For longer thinking (which the theorem does not cover) protocols certified at a low reasoning budget can fail at higher budgets

Critical strengths
  1. A correct transfer bound that gives the controller a very good guarantee: theorem 1 needs only the risk at the smaller budget, and the bound for this is very sharp. Precisely this is what lets SARCO certify a large search budget from a small calibration budget. As a result, its returned protocols come with a guarantee and don't depend on a forecast

  2. Clear diagnostic findings: self-ranking raises FAR on every dataset (fig 2d), and the top 25% of problems hold most held-out failures (fig. 2c). Judge reasoning also raises FAR on GPQA (closed) and MMLU (fig. 4). These results are useful for anyone designing oversight evaluations - and the paper reports the negative results plainly

Critical weaknesses

Weakness 1: The worst-case bound makes SARCO too conservative in most settings.

The bound assumes all acceptances sits on the top-scored candidates, so it turns a 10% target at N=64 into 2.6% at N=16. Fig 2a suggests real FAR grows by significantly under 2x over that range - while the bound has to allow close to 4x. Consequently, this also sets a minimum number of test problems: appendix B4 requires 88 problems at N=64. GPQA, MMLU, and MATH have only about 29 to 45 test problems each, so SARCO can't return anything there at large N - no matter whatever the protocols are. The positive results, therefore, come only from logic and arithmetic. This problem is built into using a worst-case bound.

Weakness 2: the reliability gain over a valid baseline rests on very few decisions.

Table 1 appears to cover about 51 decisions at N=64. On that reading, SARCO’s 100% means 14 returns with no failures, and similarly direct high-N risk control’s 90% means 2 failures in 20 returns. Direct high-N calibrates at the deployment budget, so it is also a valid method, and 2 failures in about 51 decisions is within its own guarantee. Over the full grid (table 6), it fails in only 1.2% of returns and returns protocols more often. The paper does not show why SARCO is better than simply calibrating at the deployment budget. I believe that failure counts with intervals and a comparison at equal cost could fix this.

Questions for authors
  1. How many distinct incorrect candidates does each problem’s bank hold, and how often is it fewer than 64? Search is simulated by resampling from these banks. Because if they are small, FAR at N=64 may understate real best-of-64 risk, and direct high-N calibration may not need any extra generation
  2. Prompts A1 to A3 tell the agent that the reviewer “checks one randomly chosen step,” which describes the audit protocols. How does this fit the claim that the agent has no protocol information, and does the protocol ranking in figure 2b change with a neutral prompt? And also, how is the self-score S computed?
  3. Can you please report decisions, returns, and failures per selector with confidence intervals, and compare SARCO with direct high-N risk control at equal total cost? If direct high-N matches SARCO’s reliability with higher coverage, what is the remaining case for search transfer?
5 comments
AuthorsOpenReview ↗

Author Response - Part 1

Thank you for the thorough and constructive review. We are glad you valued Theorem 1's sharp distribution-free search-transfer guarantee and found our diagnostic results on query-free self-ranking, concentrated blind spots, and judge-reasoning failures informative, as well as our candid reporting of negative results.

Below, we address your concerns and hope these additional experiments and clarifications resolve your concerns and support a re-evaluation of our work. All updates have been incorporated into the revised manuscript and appendix.

[W1] The worst-case bound makes SARCO too conservative in most settings. The bound assumes all acceptances sits on the top-scored candidates, so it turns a 10% target at N=64 into 2.6% at N=16. Fig 2a suggests real FAR grows by significantly under 2x over that range - while the bound has to allow close to 4x. Consequently, this also sets a minimum number of test problems: appendix B4 requires 88 problems at N=64. GPQA, MMLU, and MATH have only about 29 to 45 test problems each, so SARCO can't return anything there at large N - no matter whatever the protocols are. The positive results, therefore, come only from logic and arithmetic. This problem is built into using a worst-case bound.

We agree completely with your diagnosis of the empirical numbers: across our real evaluation pairings, self-ranked FAR grows by 1.0×–1.6×1.0\times\text{–}1.6\times from N=16N = 16 to N=64N = 64 (Appendix C.3; Table R2 in our response to Reviewer JVsC), whereas Theorem 1 must guard against roughly a 3.85×3.85\times increase (2.60%→10%2.60\% \to 10\%), and our positive returns at large NN come exclusively from logic (PrOntoQA) and arithmetic (GSM8K). We highlight three points clarifying what drives this behavior and when it is avoidable:

  • The margin is the unavoidable minimax price of distribution-free search transfer: If each problem accepts its top-scored fraction pp of incorrect candidates, then Ra(m)=1−(1−p)mR_a(m) = 1 - (1 - p)^m and Ra(N)=1−(1−p)NR_a(N) = 1 - (1 - p)^N, achieving equality in Theorem 1. Any looser calibration margin observed only at budget mm is therefore provably invalid on regimes consistent with the calibration data (see also Reviewer ED6L, [Q1]). Table R12 demonstrates such regimes empirically, where selectors without the worst-case transfer margin fail on nearly every split.
  • On GPQA and MMLU, menu infeasibility—not the bound—blocks returns (Table R11): At N=64N = 64 under the main target, no protocol in the menu satisfies both targets on held-out problems in 100% of GPQA-closed and GPQA-extractive decisions and 87.5% of MMLU decisions, so returning any protocol there would have produced deployment failures. Abstentions where a feasible protocol actually exists—which represent the true coverage cost of the worst-case bound and test-split floor—account for 100% of MATH decisions, 75.0% of GSM8K, 17.6% of PrOntoQA, and 12.5% of MMLU.
  • Larger calibration splits (nn) or a larger calibration search budget (mm) mitigate the sample floor: At N=64N = 64 (α=0.10,δ=0.10\alpha = 0.10, \delta = 0.10), the zero-loss test-split floor drops from 88 problems at m=16m = 16 to 44 problems at m=32m = 32 (Appendix B.4). Because MATH has fewer than 44 eligible test problems after filtering (Sec. 6), certifying MATH at N=64N = 64 requires collecting more calibration problems as well as raising mm. On tasks with sufficient problems, doubling mm from 16 to 32 lifts pooled return coverage from 14.7% to 20.4% with zero configuration violations (Table R1 in the General Response), certifying up to N=64N = 64 across six PrOntoQA pairings and N=128N = 128 for Claude Haiku 5.5 on GSM8K (Appendix A.7; Table R7 in our response to Reviewer JVsC).

Table R11: Test-split size vs. zero-loss sample floor (m=16,α=0.10,δ=0.10m = 16, \alpha = 0.10, \delta = 0.10) alongside SARCO's N=64N = 64 decisions. The statistical-test split comprises 15% of dataset problems prior to eligibility filtering (Appendix A.3).

DatasetTotal ProblemsTest Split (≈15%\approx 15\%)Clears 22 (N=16N=16)?Clears 44 (N=32N=32)?Clears 88 (N=64N=64)?N=64N=64 ReturnsAbstains (Feasible Exists)Abstains (None Feasible)
PrOntoQA1,000≈150\approx 150YesYesYes70.6%17.6%11.8%
GSM8K1,280≈192\approx 192YesYesYes25.0%75.0%0.0%
MATH300≈45\approx 45 (<44<44 elig.)YesNoNo0.0%100%0.0%
GPQA-C192≈29\approx 29YesNoNo0.0%0.0%100%
GPQA-E192≈29\approx 29YesNoNo0.0%0.0%100%
MMLU192≈29\approx 29YesNoNo0.0%12.5%87.5%

Manuscript revisions: We updated Sec. 6 to explicitly state that positive returns come from logic and arithmetic, report coverage conditioned on clearing the zero-loss sample floor, and incorporate Table R11 (upgrading Table 8 in Appendix A.7).

AuthorsOpenReview ↗

Author Response - Part 2

[W2] The reliability gain over a valid baseline rests on very few decisions....

We agree completely with your statistical reading and have removed any suggestion that SARCO is more reliable than direct high-NN risk control. Both methods satisfy the same (1−δ)(1-\delta) finite-sample guarantee:

1. Reliability parity with exact counts and confidence intervals. In the submitted Table 1 (D=51D = 51 decisions at N=64N = 64), SARCO records 0 failures in 14 returns (0.0% [0.0%, 23.2%]) and direct high-NN risk control records 2 failures in 20 returns (10.0% [1.2%, 31.7%])—a non-significant difference (Fisher's exact p=0.50p = 0.50) that sits well within δ=0.10\delta = 0.10 across 51 decisions. Our expanded 100-split evaluation across eight open-judge regimes (96 configurations, 9,600 decisions; Table R1 in the General Response) makes this parity precise: direct high-NN fails in 6 of 2,585 returns (0.2% [0.1%, 0.5%]) and SARCO (m=16m = 16) fails in 3 of 1,409 returns (0.2% [0.0%, 0.6%]). Neither method exceeds δ=0.10\delta = 0.10 in any of the 96 configurations, whereas heuristic and forecasting selectors violate δ\delta in 6 to 73 of 96 configurations.

2. Equal-cost calibration and the case for search transfer. When direct high-NN is restricted to the same total number of judged calibration candidates as SARCO (m=16m = 16, reducing direct high-NN's calibration problem count by N/mN/m), SARCO achieves slightly higher return coverage. Direct high-NN is preferable when deployment NN can be generated and judged offline, whereas SARCO calibrates once at mm to certify larger budgets N>mN > m without extra candidate generation.

3. Why search-unaware risk control is unsafe in general (Table R12). Although omitting Theorem 1 (Search-unaware risk control) happened to incur few violations on our real benchmarks (0.3% failures; 0/96 configurations >δ> \delta), that safety is purely empirical. On synthetic regimes near Theorem 1's worst case—where problems accept their highest-scored incorrect candidates—search-unaware risk control returns a protocol that violates the deployment target at NN in 98.5%–99.5% of 200 replicates, whereas SARCO and direct high-NN incur 0 violations (Table R12).

Table R12: Synthetic regimes near the Theorem 1 worst case (200 calibration replicates per row).

RegimeSelectorReturns (of 200)Replicates Missing Target at NN (of 200) [95% CI]
Worst case, no spreadSearch-unaware risk control200199 (99.5%) [97.2%, 100%]
Worst case, no spreadSARCO (m=16m = 16)1580 (0.0%) [0.0%, 1.8%]
Worst case, no spreadSARCO, linear margin1580 (0.0%) [0.0%, 1.8%]
Worst case, no spreadDirect high-NN risk control2000 (0.0%) [0.0%, 1.8%]
Worst case, spread 0.5Search-unaware risk control200197 (98.5%) [95.7%, 99.7%]
Worst case, spread 0.5SARCO (m=16m = 16)1600 (0.0%) [0.0%, 1.8%]
Worst case, spread 0.5SARCO, linear margin1600 (0.0%) [0.0%, 1.8%]
Worst case, spread 0.5Direct high-NN risk control2000 (0.0%) [0.0%, 1.8%]
α=0.30,m=4,N=64\alpha = 0.30, m = 4, N = 64, n=110n = 110 test probs.SARCO (sharp margin)2000 (0.0%) [0.0%, 1.8%]
α=0.30,m=4,N=64\alpha = 0.30, m = 4, N = 64, n=110n = 110 test probs.SARCO, linear margin00 (0.0%) [0.0%, 1.8%]

4. Scaling deployment up to N=256N = 256 (Table R13). We also generated fresh 256-candidate banks on 100 problems per task and judged every candidate across the full menu (50 splits, 200 decisions per cell; Table R13). Because 15% of 100 problems leaves only 13–17 eligible test problems—below nmin⁡=22n_{\min} = 22 even at N=16N = 16—SARCO and the other risk-controlled selectors abstain in ≥99.5%\ge 99.5\% of decisions with 0 failures (illustrating the nmin⁡n_{\min} boundary on small splits). By contrast, all four heuristic/forecasting selectors continue returning protocols whose failure rates climb steeply with NN: on GSM8K, from 37.0%–38.7% at N=16N = 16 to 87.6%–92.6% at N=256N = 256.

Table R13: Failure rate among returned protocols as deployment search scales to N=256N = 256.

Task ⋅\cdot SelectorN=16N = 16N=32N = 32N=64N = 64N=128N = 128N=256N = 256
GSM8K ⋅\cdot Outcome-based38.7%66.7%71.0%75.8%90.3%
GSM8K ⋅\cdot Tail-fit forecast37.6%60.3%64.1%75.6%92.6%
GSM8K ⋅\cdot Search-aware forecast38.2%60.9%66.5%71.8%87.6%
GSM8K ⋅\cdot Pooled37.0%56.0%63.4%71.1%91.6%
GSM8K ⋅\cdot SARCO, returns (failures)1 (0)0000
PrOntoQA ⋅\cdot Outcome-based15.0%18.0%21.0%32.0%40.0%
PrOntoQA ⋅\cdot Tail-fit forecast16.0%16.0%21.0%28.0%31.0%
PrOntoQA ⋅\cdot Search-aware forecast6.0%7.0%7.0%9.0%12.0%
PrOntoQA ⋅\cdot Pooled1.0%0.0%1.0%8.0%40.0%
PrOntoQA ⋅\cdot SARCO, returns (failures)00000
AuthorsOpenReview ↗

Author Response - Part 3

[Q1] How many distinct incorrect candidates does each problem’s bank hold, and how often is it fewer than 64? Search is simulated by resampling from these banks. Because if they are small, FAR at N=64 may understate real best-of-64 risk, and direct high-N calibration may not need any extra generation

Formally, FAR is defined on the empirical retained-bank distribution Pπ,b,x−P_{\pi,b,x}^- (Appendix A.3), so Theorem 2's validity guarantee holds strictly for that endpoint. However, we agree with both of your practical points: bootstrap resampling from finite banks understates fresh best-of-64 generation, and in our cached-bank harness direct high-NN reused the same pre-generated banks without extra candidate generation.

1. Retained bank sizes and distinct draws (Table R14). With the Qwen3-14B attacker, 8.3% of GSM8K banks and 85.8% of PrOntoQA banks contain fewer than 64 retained incorrect candidates (median 64 on GSM8K and 56.5 on PrOntoQA, as candidate filtering discards valid or off-target derivations). Consequently, 64 draws with replacement visit an average of 40.6 distinct candidates on GSM8K and 36.4 on PrOntoQA (drawing the bank's single highest-scored candidate in 64% and 71% of draws, respectively). On our expanded 256-candidate banks, 0.0% of GSM8K and 1.0% of PrOntoQA banks have fewer than 64 candidates, and 64 draws with replacement visit 56.7 and 54.4 distinct candidates.

Table R14: Retained incorrect candidates per problem across banks. Distinct in 64 draws and Top candidate drawn in 64 reflect i.i.d. sampling with replacement at N=64N = 64.

TaskAttackerProblemsMedianMin<16< 16<64< 64Distinct in 64 DrawsTop Candidate Drawn (N=64N=64)
GSM8KQwen3-14B60064550.0%8.3%40.664%
GSM8KPhi-460064400.0%35.3%40.464%
GSM8KGPT-4.1-mini60064500.0%7.0%40.664%
PrOntoQAQwen3-14B50056.511.2%85.8%36.471%
PrOntoQAPhi-450029120.8%100%25.689%
PrOntoQAGPT-4.1-mini50058240.0%96.8%38.668%
GSM8K (256-cand. bank)Qwen3-14B1002562420.0%0.0%56.722%
PrOntoQA (256-cand. bank)Qwen3-14B100216111.0%1.0%54.428%

2. Measuring the resampling understatement gap on 256-candidate banks (Table R15). Using the 256-candidate runs (Qwen3-14B →\to Qwen3-4B), we compared N=64N = 64 FAR under three sampling schemes: (i) resampling with replacement from the first 64 candidates (64-bank), (ii) resampling with replacement from the full 256-candidate bank (Full bank), and (iii) drawing 64 fresh candidates without replacement (Fresh). Resampling from a 64-candidate bank understates fresh 64-candidate risk by at most 4 percentage points on GSM8K (21% vs. 25% under direct judgment; 0–3 points across audit and reasoning protocols) and at most 1 percentage point on PrOntoQA (8% vs. 9% under direct judgment; 0 points across all audit and reasoning protocols).

3. Shared banks vs. deployment generation cost for direct high-NN. We explicitly note in Sec. 5 and Table 1 that in our benchmark evaluations, all selectors draw from shared pre-generated candidate banks—so Cal. draws in Table 1 measures judged draws per calibration problem from shared banks, and direct high-NN required no extra LLM generation in the experimental harness. In prospective deployment where 64-candidate banks do not already exist, direct high-NN requires generating, filtering, and judging NN fresh candidates per calibration problem, whereas SARCO requires only mm.

Table R15: N=64N = 64 FAR under 64-bank resampling, full-bank resampling, and fresh draws without replacement. Evaluated on the 256-candidate banks (Qwen3-14B →\to Qwen3-4B; 100 GSM8K problems and the 99 PrOntoQA problems with ≥64\ge 64 retained candidates).

Verification ProtocolGSM8K (64-bank)GSM8K (Full bank)GSM8K (Fresh)PrOntoQA (64-bank)PrOntoQA (Full bank)PrOntoQA (Fresh)
Direct judgment21%24%25%8%9%9%
Judge + uniform audit (k=3k = 3)12%14%15%6%6%6%
Judge + optimized audit (k=3k = 3)7%9%9%3%3%3%
Judge + optimized audit (k=8k = 8)7%9%9%1%1%1%
Judge + uniform audit (k=8k = 8)8%8%8%3%3%3%
Judge reasoning (bJ=512b_J = 512)5%7%8%2%2%2%
Judge + every-step audit5%5%6%0%0%0%

Manuscript revisions: We added Tables R14 and R15 to Appendix A.3, noted the finite-bank resampling understatement in Sec. 5, and clarified in Table 1's caption that Cal. draws counts judged draws from shared candidate banks.

Reviewer_ED6LOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how a weak judge’s reliability changes when a stronger agent spends more test-time compute producing convincing incorrect solutions. It considers two forms of agent scaling: best-of-N search, where the agent ranks its own wrong candidates by predicted acceptability without querying the judge, and longer reasoning budgets. The authors find that judge failures are concentrated in a minority of problems and persist across samples, and that both forms of scaling raise false acceptance (1.8–5.7× from N=1 to N=64 under search). They then propose SARCO, which uses calibration at a small search budget to select the cheapest verification protocol (e.g., direct judgment, longer judge reasoning, escalation, step audits) that is statistically certified to keep false acceptance below α and honest acceptance above 1−β at a larger deployment budget, or else abstains. The key ingredient is Theorem 1, an upper bound on false acceptance at budget N in terms of false acceptance at budget m, which is combined with Learn-then-Test and Pareto Testing.

Critical strengths

To my knowledge, this is the first work to systematically vary the adversarial agent’s reasoning budget against a weak judge and compare it with best-of-N search in an oversight setting. Related work on inference-time compute and robustness (e.g., Zaremba et al., 2025) mostly scales the defender’s reasoning rather than the attacker’s. The finding that judge blind spots are persistent and can be found without querying the judge is a useful framing for evaluating oversight. In the experiments, every protocol SARCO returned met both held-out targets, whereas empirical forecasting and search-unaware risk control did not. The paper is candid about its limitations, reporting coverage and abstentions explicitly and showing that low-budget calibration does not transfer under longer reasoning.

Critical weaknesses

Incremental novelty. SARCO largely combines existing risk-control machinery (Learn-then-Test, Pareto Testing, Hoeffding–Bentkus p-values) with a search-budget adjustment. The finding that best-of-N selection amplifies evaluator errors is well established (Gao et al., 2023; Khalaf et al., 2025; Hughes et al., 2024). Taken together, the contribution feels like an incremental step over prior work. Missing related work and baselines. Jones et al. (2025), “Forecasting Rare Language Model Behaviors” (arXiv:2502.16797), extrapolates risk from small evaluation sets to deployment scale. That is closely related to this paper’s goal of certifying a large search budget from small-budget calibration, yet it is neither cited nor used as a baseline. I recognize that SARCO provides a distribution-free guarantee rather than a fitted forecast, but the paper should discuss this distinction and ideally compare against it. Trust or Escalate (Jung et al., ICLR 2025) also provides provable guarantees for LLM judges with escalation to stronger judges and should be discussed. Zaremba et al. (2025) is relevant to the judge-reasoning results in Figure 4.

Questions for authors

How meaningful is Theorem 1 in practice? It follows from a fairly standard order-statistics argument, and Table 6 shows that the linear bound αm/N produces identical decisions in every setting tested. Is there a realistic regime where the sharp form changes which protocol is selected? How does SARCO compare with forecasting approaches such as Jones et al. (2025) in accuracy and coverage? Is the large loss in coverage the necessary price of a distribution-free guarantee?

4 comments
AuthorsOpenReview ↗

Author Response - Part 1

Thank you for the thoughtful and constructive review. We appreciate your recognition that our work is the first to systematically scale an adversarial agent's reasoning budget against a weaker judge alongside best-of-NN search, and that our finding on persistent, query-free searchable blind spots provides a useful framing for oversight evaluation. We also appreciate your positive remarks on SARCO's empirical reliability and our explicit reporting of abstentions and negative transfer under longer reasoning.

Below, we address your comments on SARCO's novelty relative to existing risk-control primitives, incorporate and compare against the missing related work (Jones et al., 2025; Jung et al., 2025; Zaremba et al., 2025), and answer each part of your question on Theorem 1, parametric tail forecasting, and the coverage–validity trade-off. We hope these new baselines and clarifications address your concerns and support a re-evaluation of our work. All updates have been incorporated into the revised manuscript and appendix.

[W1a] Incremental novelty. SARCO largely combines existing risk-control machinery (Learn-then-Test, Pareto Testing, Hoeffding–Bentkus p-values) with a search-budget adjustment. The finding that best-of-N selection amplifies evaluator errors is well established (Gao et al., 2023; Khalaf et al., 2025; Hughes et al., 2024). Taken together, the contribution feels like an incremental step over prior work.

We agree that SARCO's multiple-testing primitives (Learn-then-Test, Pareto Testing, and Hoeffding–Bentkus pp-values) are standard tools, and we have revised the Sec. 1 contributions to state this explicitly. Building on those primitives, three aspects of our work are new:

  • Query-free oversight setting (no evaluator access during selection): Prior work on best-of-NN overoptimization and jailbreaking relies on direct evaluator feedback during candidate selection: Gao et al. (2023) and Khalaf et al. (2025) rank candidates directly using the proxy reward model being exploited, and Hughes et al. (2024) repeatedly query the target model and observe attack success. In our setting, the agent ranks its own candidates via S(x,z)S(x,z) without ever querying or identifying the deployed judge or verification protocol. We show that query-free self-ranking still amplifies FAR—by 1.8×–5.7×1.8\times\text{–}5.7\times in the submitted benchmarks and 2.1×–9.0×2.1\times\text{–}9.0\times across six open/closed judges and three attacker families in our expanded cross-family runs (Appendix C.3; Table R2 in our response to Reviewer JVsC)—because weak-judge blind spots persist on the same problems, changing which form of verification (not just how much judge compute) is cheapest and reliable.
  • Distribution-shift transfer from low-mm calibration to high-NN deployment: Standard Learn-then-Test and Pareto Testing certify risk only on the distribution they calibrate on, whereas scaling agent search from mm to N>mN > m alters the submitted-candidate distribution. Theorem 1 bridges this distribution shift by converting a deployment target at budget NN into an exact, minimax-tight calibration target αeff(N)=1−(1−α)m/N\alpha_{\text{eff}}(N) = 1 - (1 - \alpha)^{m/N} at budget mm using only low-search risk, while Remark 1 establishes the sharp boundary with thinking-budget shifts where no distribution-free transfer exists.
  • Scaling attacker-side test-time compute (search vs. longer thinking): As noted in your review, prior inference-time robustness work primarily scales the defender's reasoning, whereas we systematically scale and compare the adversarial agent's reasoning budget against best-of-NN search—showing both that search typically raises FAR faster per token and that low-budget protocol certifications fail to transfer across longer agent reasoning.

Manuscript revisions: We rewrote the Sec. 1 contributions bullets to explicitly credit the standard risk-control components alongside these three contributions, and expanded Sec. 9 (Related Work) to contrast evaluator-querying selection with query-free self-ranking.

AuthorsOpenReview ↗

Author Response - Part 2

[W1b] Missing related work and baselines: Jones et al. (2025), “Forecasting Rare Language Model Behaviors” (arXiv:2502.16797)... Trust or Escalate (Jung et al., ICLR 2025) also provides provable guarantees for LLM judges with escalation to stronger judges and should be discussed. Zaremba et al. (2025) is relevant to the judge-reasoning results in Figure 4.

Thank you for pointing out all three references. We have added all three to the revised manuscript and implemented a parametric tail-forecasting baseline inspired by Jones et al. (2025) (evaluated in [Q1] below):

  • Jones et al. (2025) (Forecasting Rare Language Model Behaviors): Added to Sec. 9 and evaluated as a baseline in Sec. 5–6 and Appendix B.6 (detailed in [Q1] below). Whereas Jones et al. fit parametric extreme-value tails across query elicitation probabilities to forecast rare failures over larger query volumes, SARCO provides a distribution-free (1−δ)(1-\delta) certificate over within-problem candidate search NN.
  • Jung et al. (ICLR 2025) (Trust or Escalate): Added to Sec. 9 (Verification cost and deployment feedback). Jung et al. calibrate confidence thresholds on a weak judge to selectively escalate uncertain instances to a stronger judge (or human) with finite-sample agreement guarantees on a fixed input distribution (N=1N = 1). Selective confidence-threshold escalation (Escalation to Qwen3-14B at three threshold pairs) is in fact one of the protocol families inside SARCO's menu A(r)\mathcal{A}(r). SARCO differs along two axes: (i) calibrating escalation without search transfer corresponds to our Search-unaware risk control baseline, which violates deployment targets once agent search scales to N=64N = 64 (90.0% attainment vs. 100% for SARCO; Table 1), and (ii) SARCO selects offline across heterogeneous verification forms (direct judgment, judge reasoning, majority voting, escalation, and randomized/optimized step audits) under joint FAR and HAR constraints.
  • Zaremba et al. (2025) (Trading Inference-Time Compute for Adversarial Robustness): Added to Sec. 7 (discussing Fig. 4) and Appendix A.8. Zaremba et al. show that scaling defender-side reasoning tokens improves adversarial robustness. Figure 4 shows that this holds in our oversight setting on deductive tasks—on GSM8K and PrOntoQA, scaling judge reasoning to 512–2048 tokens holds FAR below 8% across all attacker thinking budgets (and reduces closed-judge GSM8K FAR to ≤2.9%\le 2.9\% at N=64N = 64 in Appendix A.8)—but reverses on science tasks (GPQA-closed and MMLU), where extra judge reasoning increases FAR above direct judgment at every agent budget (e.g., 20.7%–24.3% vs. 5.4%–11.7% on MMLU) as the weaker judge rationalizes plausible-sounding false claims.
AuthorsOpenReview ↗

Author Response - Part 3

[Q1] How meaningful is Theorem 1 in practice?

1. Practical role of Theorem 1 and when the sharp form 1−(1−α)m/N1 - (1-\alpha)^{m/N} changes decisions vs. αm/N\alpha m / N. Theorem 1 serves two practical roles. First, its minimax tightness construction (Appendix B.2) proves that the Θ(m/N)\Theta(m/N) contraction of the calibration target is not an artifact of a loose inequality: over arbitrary score–acceptance distributions using only Ra(m)R_a(m), no distribution-free controller can use a looser calibration threshold without risking target violations at budget NN. Second, by Taylor expansion, αeff(N)=1−(1−α)m/N=αmN+m2N ⁣(1−mN)α2+O(α3).\alpha_{\text{eff}}(N) = 1 - (1 - \alpha)^{m/N} = \frac{\alpha m}{N} + \frac{m}{2N}\!\left(1 - \frac{m}{N}\right)\alpha^2 + O(\alpha^3). In our main grid (α≤0.10\alpha \le 0.10, N/m≤4N/m \le 4), the O(α2)O(\alpha^2) term is small: at α=0.10,N/m=4,δ=0.10\alpha = 0.10, N/m = 4, \delta = 0.10, αeff=2.60%\alpha_{\text{eff}} = 2.60\% vs. αlin=2.50%\alpha_{\text{lin}} = 2.50\%, which shifts the Hoeffding–Bentkus zero-loss sample floor nmin⁡=⌈log⁡(1/δ)/(−log⁡(1−θ))⌉n_{\min} = \lceil \log(1/\delta) / (-\log(1 - \theta)) \rceil (Appendix B.4) from 9191 to 8888 test problems—a 3-problem gap that happens not to straddle our benchmark test-split sizes. However, in realistic regimes with moderate error tolerances (e.g., α∈[0.20,0.30]\alpha \in [0.20, 0.30] in human-in-the-loop screening or monitoring triage) or larger search-extrapolation ratios (N/m≥8N/m \ge 8), the sharp form strictly reduces required calibration sizes and changes admission: for example, at α=0.20,N/m=4,δ=0.10\alpha = 0.20, N/m = 4, \delta = 0.10, nmin⁡n_{\min} drops from 4545 (linear) to 4242 (sharp), and at α=0.30,N/m=16,δ=0.10\alpha = 0.30, N/m = 16, \delta = 0.10, nmin⁡n_{\min} drops by 15% from 122122 (linear) to 104104 (sharp)—meaning any calibration test split with 104≤n≤121104 \le n \le 121 problems admits zero-loss protocols under the sharp bound while the linear bound cannot certify any protocol. More generally, the relative margin gain grows with N/mN/m up to a supremum factor of −ln⁡(1−α)/α-\ln(1-\alpha)/\alpha (5.4% at α=0.10\alpha = 0.10 and 18.9% at α=0.30\alpha = 0.30; Table R9). On synthetic regimes at α=0.30,m=4,N=64\alpha = 0.30, m = 4, N = 64 with n=110n = 110 test problems (between 104 and 122), the sharp margin certifies a protocol in 200/200 replicates (0 violations) while the linear margin returns 0/200.

Table R9: Sharp vs. linear search margin (δ=0.10\delta = 0.10, sharp / linear minimum test problems nmin⁡n_{\min}).

Target α\alphaSupremum Margin Gain (−ln⁡(1−α)/α−1-\ln(1-\alpha)/\alpha - 1)nmin⁡n_{\min} (N/m=4N/m = 4)nmin⁡n_{\min} (N/m=16N/m = 16)
0.105.4%88 / 91350 / 368
0.2011.6%42 / 45166 / 184
0.3018.9%26 / 30104 / 122

2. Comparison with parametric tail forecasting (Jones et al., 2025; Table R8). To adapt extreme-value tail forecasting to within-problem candidate search, our tail-fit forecast selector computes each protocol's empirical top-of-kk self-ranked FAR R^a(k)\hat{R}_a(k) for k=1,…,16k = 1, \dots, 16 on the calibration split (Eq. 3), fits the complementary log-log scaling relation log⁡(−log⁡(1−R^a(k)))=alog⁡k+b\log(-\log(1 - \hat{R}_a(k))) = a \log k + b across kk (capturing power-law hazard growth 1−Ra(k)≈exp⁡(−cka)1 - R_a(k) \approx \exp(-c k^a), which nests the i.i.d. envelope at a=1a = 1), extrapolates R^a(N)\hat{R}_a(N) to NN, and selects the cheapest protocol predicted to meet both targets. Empirically, parametric tail forecasting trades away validity for higher return coverage:

  • Pooled across eight open-judge regimes (96 configurations, 9,600 decisions; Table R1 in the General Response): The tail-fit forecast returns a protocol in 69.2% of decisions, but fails one or both deployment targets in 18.3% [17.4%, 19.3%] of its returns—violating δ=0.10\delta = 0.10 in 42 of 96 configurations (and failing on up to 45% of splits in the worst configuration). By contrast, SARCO (m=16m = 16) fails in only 0.2% [0.0%, 0.6%] of returns and violates δ=0.10\delta = 0.10 in 0 of 96 configurations.
  • Run-by-run across all 12 open- and closed-judge suites (Table R8): The tail-fit forecast fails in 2.0%–38.3% of returned protocols on every attack suite except the PrOntoQA closed-judge suite (where judges already reject nearly 100% of attacks at every NN), whereas SARCO violates δ=0.10\delta = 0.10 in 0 configurations across every suite. (On GSM8K →\to Nemotron-9B, SARCO's 1 / 8 is 1 violation out of 600 decisions—a 2% rate in one 50-split configuration, well inside δ=0.10\delta = 0.10.)
AuthorsOpenReview ↗

Author Response - Part 3 (Continued)

Table R8: Parametric tail-fit forecast (inspired by Jones et al., 2025) vs. SARCO (m=16m = 16) by evaluation suite. 50 random calibration splits per run (δ=0.10\delta = 0.10).

Evaluation SuiteDecisionsTail-Fit ReturnsTail-Fit Failure RateTail-Fit Configs >δ> \deltaSARCO ReturnsSARCO Failures / ReturnsSARCO Configs >δ> \delta
GSM8K, Qwen3-14B →\to Qwen3-4B, Nemotron-8B, Haiku, Sonnet2,40077.9%18.6%17 of 4813.6%0 / 3260 of 48
GSM8K, Qwen3-14B →\to Nemotron-9B60038.8%26.6%6 of 121.3%1 / 80 of 12
GSM8K, Phi-4 →\to Qwen3-4B60066.8%31.9%8 of 122.8%0 / 170 of 12
GSM8K, GPT-4.1-mini →\to Qwen3-4B60088.8%3.0%1 of 1210.2%0 / 610 of 12
GSM8K, closed judges1,80088.8%17.8%13 of 3616.6%0 / 2990 of 36
GSM8K, neutral prompt60061.2%10.9%2 of 121.0%0 / 60 of 12
PrOntoQA, Qwen3-14B →\to Qwen3-4B, Nemotron-8B, Haiku1,80082.2%20.7%17 of 3633.5%2 / 6030 of 36
PrOntoQA, Qwen3-14B →\to Nemotron-9B60098.3%10.0%6 of 1226.2%0 / 1570 of 12
PrOntoQA, Phi-4 →\to Qwen3-4B600100%2.0%0 of 1234.7%0 / 2080 of 12
PrOntoQA, GPT-4.1-mini →\to Qwen3-4B600100%36.0%12 of 1238.8%0 / 2330 of 12
PrOntoQA, neutral prompt600100%38.3%12 of 1249.2%0 / 2950 of 12
PrOntoQA, closed judges1,80095.7%0.3%0 of 3640.2%3 / 7230 of 36

3. Is the loss in coverage the necessary price of a distribution-free guarantee? First, most abstentions are not lost valid coverage: in our main grid, 74.3% of SARCO's abstentions occur in settings where no protocol in the menu satisfies both targets on held-out problems (Appendix A.7), so abstaining avoids deploying an uncertified failure. Second, on the remaining 25.7% where a feasible protocol exists, yes—under a fixed low-search budget (n,m)(n, m), Theorem 1's sharpness proof establishes that guarding against concentrated top-score blind spots (and obeying the sample floor nmin⁡∝N/mn_{\min} \propto N/m) is the unavoidable minimax price of distribution-free search transfer. Relaxing that conservativeness requires either (i) assuming a parametric tail law—which lifts coverage to 69.2% at the price of an 18.3% deployment failure rate when real score–acceptance tails deviate from the fit (Table R8)—or (ii) spending more offline calibration compute (nn or mm), such as increasing mm from 16 to 32, which halves nmin⁡n_{\min} and lifts pooled coverage from 14.7% to 20.4% while maintaining 0% configuration violations.

Manuscript revisions: We added discussions of Jones et al. (2025) and Jung et al. (2025) to Sec. 9 and Zaremba et al. (2025) to Sec. 7 (Fig. 4), added the parametric tail-fit forecast baseline and Table R8 to Appendix B.6 (summarized in Sec. 5–6), and added the sharp-vs.-linear sample-floor analysis and Table R9 to Appendix B.4.

Reviewer_JVsCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper revealed a phenomenon in which the weak judge might be unable to evaluate the answer from the agent with much more test time compute in the scalable oversight setting. The authors focused on Best of N case and longer adversarial thinking case, and proposed ASRCO to leverage limited calibration to choose the protocol for verifying the agent. The theoretical guarantee on the Best of N case is also provided.

Critical strengths
  • Analyze the phenomenon in which an agent can deceive a weak judge without direct access to the specific judge models.
  • Provide a workflow design for selecting the protocol and give theoretical guarantees for the Best of N case.
Critical weaknesses
  • Although cross-family experiments are included, the authors used models from the Qwen family as both the agent (Qwen3-14B) and the judge (Qwen3-4B) in the diagnostic experiments. Given the assumption that an agent can deceive a weak judge without direct access to the judge models, using models from the same family might make the argument less convincing.
  • The research is conducted in the context of scalable oversight, where only test-time inference is required. The lack of experiments on closed-source models narrows the research scope.
  • What is the motivation for the selection process in the context of scalable oversight? Though the agent is prompted to find wrong answers adversarially, in a practical setting, agents might provide more accurate answers instead of generating more deceptive answers to avoide the detection by the weak checker.
Questions for authors
  • Could you provide AI use statement? This is mandatory for the conference submission.
  • Could you provide some reasons why a strong agent is able to generate answers to deceive the weak judge?
  • The selection process provides a theoretical guarantee for best of N case. Could you discuss the longer adversarial thinking case in more details?
  • How sensitive is the protocol selection coverage to the order of the fitting splits?
  • If SARCO abstains in most deployment settings, how should the oversight system deal with these cases, and what is the reliability guarantees and costs? What is the estimated overall abstention rate?
  • This is a minor issue, but the descriptions of Figure 3 and Table 1 seem concatenated, is it possible to make them more readable?
5 comments
AuthorsOpenReview ↗

Author Response - Part 1

Thank you for your feedback. We are glad you found our analysis of how a stronger agent can deceive a weaker judge without direct access to the judge model informative, and we appreciate your positive assessment of SARCO's protocol-selection workflow and its theoretical guarantees.

Below, we respond to your specific comments and questions regarding cross-family and hope these new experiments and clarifications address your concerns and enable a re-evaluation of our work.

[W1] Although cross-family experiments are included, the authors used models from the Qwen family as both the agent (Qwen3-14B) and the judge (Qwen3-4B) in the diagnostic experiments. Given the assumption that an agent can deceive a weak judge without direct access to the judge models, using models from the same family might make the argument less convincing.

While our protocol-selection experiments already evaluate four non-Qwen judges—Phi-4-mini across five datasets, plus Gemma-4-12B, NVIDIA-Nemotron-Nano-9B-v2, and Llama-3.1-Nemotron-Nano-8B, where SARCO returns protocols in 25.0% of Gemma-4-12B and Nemotron-9B decisions at N=64N = 64 with 100% target attainment and abstains on Nemotron-8B (Sec. 5–6, Appendix A.7)—we agree that the Fig. 2 diagnostics only paired Qwen3-14B with Qwen3-4B. We have now repeated those diagnostics across eight cross-family pairings on GSM8K and PrOntoQA, varying both the judge family (Llama-3.1-Nemotron-Nano-8B, Nemotron-Nano-9B-v2) and the attacker family (open-weight Phi-4 and closed-source GPT-4.1-mini; Table R2).

Across every cross-family pairing, all three diagnostic findings from Sec. 3 hold without shared family pretraining:

  • Search consistently amplifies false acceptance: Scaling from N=1N = 1 to N=64N = 64 raises cross-family FAR by 4.4×–7.6×4.4\times\text{–}7.6\times on GSM8K and 2.5×–3.2×2.5\times\text{–}3.2\times on PrOntoQA—matching or exceeding the same-family Qwen3-14B →\to Qwen3-4B amplification (5.9×5.9\times and 2.1×2.1\times). For instance, Qwen3-14B raises FAR to 32.2% on Nemotron-8B and 34.9% on Nemotron-9B (vs. 27.8% on Qwen3-4B), while Phi-4 and GPT-4.1-mini raise Qwen3-4B's FAR by 7.6×7.6\times and 6.1×6.1\times on GSM8K.
  • Judge blind spots remain concentrated on specific problems: The top 25% of calibration-ranked problems account for 67%–71% of held-out failures on GSM8K and 49%–81% on PrOntoQA across cross-family pairs (roughly 2×–3.2×2\times\text{–}3.2\times random ordering), showing that vulnerability reflects problem-level structure rather than family-specific artifacts.
  • Query-free self-ranking transfers across model families: The attacker's own score S(x,z)S(x,z) separates judge-accepted from judge-rejected incorrect candidates well above the shuffled-SS null (0.44–0.560.44\text{–}0.56) in every pairing (AUROC=0.85–0.96\text{AUROC} = 0.85\text{–}0.96 on GSM8K; 0.64–0.780.64\text{–}0.78 on PrOntoQA). Notably, evaluating Qwen3-14B's unchanged self-scores against Nemotron-8B and Nemotron-9B yields nearly identical within-problem discrimination to Qwen3-4B (0.94 and 0.96 vs. 0.97 on GSM8K; 0.76 and 0.77 vs. 0.77 on PrOntoQA).

Table R2: Cross-family diagnostics under direct judgment. FAR of the top-scored candidate among the first NN generated candidates. Top-25% share: held-out failures captured by the top 25% of problems ranked from separate calibration candidates (as in Fig. 2c). AUROC(SS): within-problem separation of judge-accepted vs. judge-rejected incorrect candidates by the attacker's query-free score SS (shuffled-SS null: 0.44–0.560.44\text{–}0.56).

TaskAttacker →\to JudgeProblemsFAR (N=1N=1)FAR (N=16N=16)FAR (N=64N=64)Growth (1→641 \to 64)Top-25% ShareAUROC(SS)
GSM8KQwen3-14B →\to Qwen3-4B (same-family ref.)6004.7%18.2%27.8%5.9×5.9\times78%0.97
GSM8KQwen3-14B →\to Nemotron-8B6005.7%21.9%32.2%5.7×5.7\times69%0.94
GSM8KQwen3-14B →\to Nemotron-9B$^*$3927.9%25.3%34.9%4.4×4.4\times71%0.96
GSM8KPhi-4 →\to Qwen3-4B6001.3%6.5%10.2%7.6×7.6\times67%0.86
GSM8KGPT-4.1-mini →\to Qwen3-4B6002.5%11.2%15.3%6.1×6.1\times69%0.85
PrOntoQAQwen3-14B →\to Qwen3-4B (same-family ref.)5006.2%10.4%13.0%2.1×2.1\times84%0.77
PrOntoQAQwen3-14B →\to Nemotron-8B5002.4%6.4%6.3%2.6×2.6\times81%0.76
PrOntoQAQwen3-14B →\to Nemotron-9B$^*$2805.4%12.5%15.0%2.8×2.8\times69%0.77
PrOntoQAPhi-4 →\to Qwen3-4B5009.0%25.2%28.8%3.2×3.2\times49%0.78
PrOntoQAGPT-4.1-mini →\to Qwen3-4B5006.6%10.6%16.2%2.5×2.5\times53%0.64

∗^*Nemotron-9B rows cover the subset of problems where it returned a parseable verdict for every candidate.

Family overlap therefore does not drive the phenomenon, and we have incorporated Table R2 into the revised manuscript (Sec. 3 and Appendix C).

AuthorsOpenReview ↗

Author Response - Part 2

[W2] The research is conducted in the context of scalable oversight, where only test-time inference is required. The lack of experiments on closed-source models narrows the research scope.

We have expanded our experiments to include four closed-source judges (GPT-4.1-mini, GPT-5.4-mini, Claude Haiku 5.5, and Claude Sonnet 5.5) alongside our closed-source attacker (GPT-4.1-mini, reported in Table R2).

1. Search-induced vulnerability and task heterogeneity persist on closed-source judges (Table R3). Under direct judgment on GSM8K, scaling search from N=1N = 1 to N=64N = 64 sharply increases false acceptance for every closed-source judge: GPT-4.1-mini rises from 4.3% to 21.8%, Claude Haiku 5.5 from 2.2% to 19.3%, and Claude Sonnet 5.5 from 0.8% to 5.5%. Switching judges to reasoning mode largely eliminates this vulnerability on GSM8K (FAR≤2.9%\text{FAR} \le 2.9\% at N=64N = 64), consistent with Zaremba et al. (2025). Conversely, on PrOntoQA even direct judgment accepts almost no incorrect derivations at any NN (Haiku: 0.0%, Sonnet: ≤0.4%\le 0.4\%, GPT-4.1-mini: ≤0.8%\le 0.8\%). This contrast reinforces our central motivation: cheap direct judgment suffices on PrOntoQA even at N=64N = 64, whereas search breaks direct judgment on GSM8K and requires upgrading verification.

Table R3: Closed-source judges on GSM8K against the Qwen3-14B attacker. Direct-judgment rows cover 521–600 problems; the two low-effort reasoning rows use a 120-problem subset.

JudgeModeFAR (N=1N = 1)FAR (N=16N = 16)FAR (N=64N = 64)
Qwen3-4B (open-weight ref.)Direct4.7%18.2%27.8%
GPT-4.1-miniDirect4.3%14.5%21.8%
Claude Haiku 5.5Direct2.2%11.7%19.3%
Claude Sonnet 5.5Direct0.8%3.0%5.5%
GPT-5.4-miniReasoning, low effort0.0%2.5%2.9%
Claude Haiku 5.5Reasoning, high effort0.0%0.3%0.5%
Claude Sonnet 5.5Reasoning, low effort0.0%0.0%0.0%

2. SARCO applies directly to closed-source API judges using binary verdicts (Table R4). Because Theorem 2 requires only bounded verdicts (qa∈[0,1]q_a \in [0, 1]), SARCO operates unchanged on parsed Yes/No responses from closed-source judges. Evaluating protocol selection across Haiku, Sonnet, and GPT regimes (each offering direct, reasoning-mode, and step-audit protocols across 50 calibration splits; 1,800 decisions per task):

  • GSM8K: Heuristic and forecasting selectors fail one or both targets in 3.2%–17.8% of returned protocols and violate the δ=0.10\delta = 0.10 risk tolerance in 2 to 13 of 36 configurations. SARCO (m=16m = 16 and m=32m = 32) achieves 0.0% violations across all configurations using low-search calibration.
  • PrOntoQA: Every selector—including SARCO—experiences failures in only 0.3%–0.5% of returns, well inside the nominal δ=10%\delta = 10\% guarantee (0 of 36 configurations exceed δ\delta, with every configuration's empirical violation rate ≤2%\le 2\%).

Table R4: Protocol selection over closed-source judge regimes. Haiku, Sonnet, and GPT regimes with direct, reasoning-mode, and step-audit menus across 50 random splits (1,800 decisions per task across 36 configurations; 24 configurations at N≥32N \ge 32 for m=32m = 32). Brackets show 95% confidence intervals; on PrOntoQA, 0 of 36 configurations exceed δ=0.10\delta = 0.10 for any selector.

SelectorGSM8K ReturnsGSM8K Failure RateGSM8K Configs >δ> \deltaPrOntoQA ReturnsPrOntoQA Failure Rate
Outcome-based94.8%17.4% [15.6, 19.3]13 / 3695.7%0.3% [0.1, 0.8]
Tail-fit forecast88.8%17.8% [15.9, 19.7]13 / 3695.7%0.3% [0.1, 0.8]
Pooled95.5%11.8% [10.3, 13.4]8 / 3695.7%0.3% [0.1, 0.8]
Search-aware forecast58.7%3.2% [2.2, 4.5]2 / 3695.7%0.3% [0.1, 0.8]
Search-unaware risk control29.0%0.0% [0.0, 0.7]0 / 3678.5%0.4% [0.2, 0.9]
Direct high-NN risk control27.0%0.0% [0.0, 0.8]0 / 3676.4%0.4% [0.2, 0.9]
SARCO (m=16m = 16)16.6%0.0% [0.0, 1.2]0 / 3640.2%0.4% [0.1, 1.2]
SARCO (m=32m = 32)20.9%0.0% [0.0, 1.5]0 / 2454.5%0.5% [0.1, 1.3]

3. Remaining API scope boundaries. While SARCO's verifier selection requires only black-box API verdicts, two attacker-side experimental mechanisms rely on capabilities that closed APIs expose only partially: (i) computing the continuous self-score S(x,z)S(x,z) from Yes/No token log-probabilities (available for our GPT-4.1-mini attacker in Table R2, though omitted by several frontier reasoning APIs), and (ii) mid-trace end-of-thinking suppression for controlled token-budget sweeps in Sec. 7.

Manuscript revisions: We have added Tables R3 and R4 to a new closed-source evaluation appendix, referenced these results in Sec. 6, and added an explicit closed-source API scope discussion to Sec. 10.

AuthorsOpenReview ↗

Author Response - Part 3

[W3] What is the motivation for the selection process in the context of scalable oversight? Though the agent is prompted to find wrong answers adversarially, in a practical setting, agents might provide more accurate answers instead of generating more deceptive answers to avoide the detection by the weak checker.

Our formulation reflects both the standard threat model in scalable oversight and the selection pressure inherent to confidence-ranked best-of-NN pipelines:

1. Threat model (conditional verifier reliability). Scalable oversight must hold when a stronger agent is misaligned, reward-hacking, or sycophantic—the standard stress-test setting in debate, consultancy, and AI control (Khan et al., 2024; Kenton et al., 2024; Greenblatt et al., 2024). Even without deliberate subversion, post-training for evaluator approval (e.g., RLHF) can make incorrect outputs systematically more convincing without making them more accurate (Wen et al., 2024). Prompting a frozen model to argue for an incorrect answer isolates conditional verifier risk—how often the judge is fooled when the agent is wrong—without conflating verifier reliability with how often the agent solves easy benchmark items honestly. Moreover, any practical best-of-NN pipeline that ranks candidates by model confidence or predicted acceptability applies this same selection pressure whenever a hard problem exceeds the agent's competence, surfacing its most plausible-looking errors.

2. Empirical comparison on benign errors (Table R5). To test non-adversarial behavior directly, we prompted Qwen3-14B honestly, retained the problems where it naturally produced incorrect solutions, and ranked those benign errors by the same score S(x,z)S(x,z) against the Qwen3-4B direct judge:

  • Selection still amplifies benign errors, though less steeply than adversarial search: Ranking by SS raises benign-error FAR from 33.9% to 41.5% (1.2×1.2\times) on GSM8K and from 36.0% to 52.5% (1.5×1.5\times) on MATH, compared to 5.9×5.9\times in the matched GSM8K attack pairing (4.4×–9.0×4.4\times\text{–}9.0\times across all GSM8K attack pairings). Benign errors start from a much higher N=1N = 1 FAR because an honest 14B model errs primarily on genuinely difficult problems where its mistakes reflect subtle, plausible misconceptions rather than forced errors on easy items.
  • Risk control prevents false certification on hard benign errors: Because benign-error FAR is already ≥33.9%\ge 33.9\% at N=1N = 1, no protocol in our menu meets the deployment target in these regimes. Across 1,200 decisions (50 splits), SARCO and the other risk-controlled selectors correctly abstain in all 1,200 decisions (0% false returns), whereas the pooled heuristic selector returns a protocol 7 times and fails the deployment target in all 7 (100% failure rate).

Table R5: Benign errors vs. adversarial search under direct judgment (Qwen3-14B →\to Qwen3-4B). Benign rows prompt Qwen3-14B honestly and retain naturally occurring incorrect solutions ranked by S(x,z)S(x,z).

SettingProblemsFAR (N=1N = 1)FAR (N=16N = 16)FAR (N=64N = 64)Growth (1→641 \to 64)
GSM8K, benign errors12433.9%41.1%41.5%1.2×1.2\times
MATH, benign errors20036.0%50.0%52.5%1.5×1.5\times
GSM8K, adversarial attack6004.7%18.2%27.8%5.9×5.9\times

Manuscript revisions: We have added a dedicated threat-model paragraph to the Introduction (Sec. 1) citing Khan et al. (2024), Kenton et al. (2024), Greenblatt et al. (2024), and Wen et al. (2024), and added the benign-error evaluation (Table R5) to Appendix C.4.

[Q1] Could you provide AI use statement? This is mandatory for the conference submission.

We apologize for the accidental omission of this section from the submitted PDF (while declaring our usage in the OpenReview submission form). We have added the mandatory AI Use Statement to the revised manuscript right after the Conclusion (before References), matching our OpenReview disclosure.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to Reviewers and Area Chairs

We thank all three reviewers and the Area Chair for their constructive feedback. Our revised manuscript (major additions in blue) incorporates the following new experiments, baselines, and clarifications:

  1. Multi-split evaluation, equal-cost calibration, and tail forecasting (Table R1; Sec. 5–6, Appendix A.7, B.6–B.7):

    • Evaluated all selectors over 100 random splits across eight open-judge regimes (96 configurations, 9,600 decisions; Table R1) and 12 open/closed suites (50 splits each; Tables R8, R17–R18) with exact counts, 95% CIs, and per-configuration violation rates.
    • Added a Jones et al. (2025)-style extreme-value tail-fit forecast: it lifts coverage to 69.2% (Table R1) but fails targets in 18.3% [17.4%, 19.3%] of returns, violating δ=0.10\delta = 0.10 in 42 of 96 configurations (vs. 0 of 96 for SARCO).
    • Repositioned against direct high-NN risk control: both achieve 0.2% failure rates (0/96 configurations >δ> \delta). At equal judged candidates, SARCO (m=16m = 16) slightly leads in coverage (14.7% vs. 12.5%; Table R1). Direct high-NN is preferable when deployment NN can be generated and judged offline; SARCO calibrates once at mm to certify larger N≥mN \ge m (at an O(N/m)O(N/m) test-split floor).
  2. Cross-family and closed-source attackers and judges (Sec. 3, 5–6, 10; Appendix A.8, C.3):

    • Expanded Fig. 2 diagnostics to eight cross-family pairs with Nemotron-8B/9B judges and Phi-4 and GPT-4.1-mini attackers (Table R2 in JVsC): search raises FAR by 4.4×–7.6×4.4\times\text{–}7.6\times (GSM8K) and 2.5×–3.2×2.5\times\text{–}3.2\times (PrOntoQA), with unchanged Qwen3-14B scores S(x,z)S(x,z) reaching 0.94–0.96 AUROC on Nemotron (vs. 0.97 on Qwen3-4B).
    • Evaluated four closed-source API judges (GPT-4.1-mini, GPT-5.4-mini, Claude Haiku 5.5, Sonnet 5.5; Tables R3–R4 in JVsC): search raises direct GSM8K FAR to 19.3%–21.8% (≤2.9%\le 2.9\% in reasoning mode), and SARCO on binary verdicts has 0.0% GSM8K violations (0/36 configurations >δ> \delta) vs. up to 17.8% for heuristics.
  3. Threat model, benign errors, prompt neutrality, and S(x,z)S(x,z) (Sec. 1–3; Appendix A.2, C.4):

    • Contrasted adversarial stress-testing with benign errors under honest prompting (Table R5 in JVsC), where self-ranked benign errors grow 1.2×–1.5×1.2\times\text{–}1.5\times in FAR and SARCO abstains in all 1,200 infeasible decisions (0% false certifications).
    • Added neutral-prompt ablations removing step-check wording (Table R16 in VpVK), confirming stable rankings (Kendall's τ=0.73–0.87\tau = 0.73\text{–}0.87 on GSM8K; 1.001.00 on PrOntoQA) and SARCO validity, and defined S(x,z)S(x,z) with its exact prompt.
  4. Sharpness, worst-case necessity, finite-bank checks, and abstention remedies (Sec. 4.2, 6–7; Appendix A.3, A.7, B.2, B.4, B.7):

    • Added Theorem 1's sharpness construction, bounded sharp-vs.-linear gains by −ln⁡(1−α)/α-\ln(1-\alpha)/\alpha (Table R9 in ED6L), proved cross-bb transfer impossibility (Remark 1) with a nested-prefix recalibration recipe (Sec. 7), and showed 98.5%–99.5% search-unaware RC failures on worst-case regimes (Table R12 in VpVK).
    • Built 256-candidate banks (Tables R13–R15 in VpVK) bounding finite-bank resampling gaps (≤4\le 4 points GSM8K, ≤1\le 1 point PrOntoQA) with heuristic failures reaching 87.6%–92.6% at N=256N = 256, and added sample-floor breakdowns (Table R11 in VpVK) and certified caps Nmax⁡N_{\max} (Table R7 in JVsC).

Table R1: Pooled prospective selection across eight open-judge regimes (96 configurations, 100 splits; 9,600 decisions, δ=0.10\delta = 0.10). Exact 95% Clopper–Pearson CIs among returns; Configs >δ> \delta (Worst) counts configurations exceeding the δ=10%\delta = 10\% unconditional violation bound across splits (and worst rate). SARCO (m=32m = 32) covers the 64 configurations with N≥32N \ge 32.

SelectorReturns (% of 9,600)Failures / ReturnsFailure Rate [95% CI]Configs >δ> \delta (Worst)
Outcome-based80.1%2,807 / 7,69236.5% [35.4%, 37.6%]73 of 96 (86%)
Tail-fit forecast (Jones et al. style)69.2%1,217 / 6,64618.3% [17.4%, 19.3%]42 of 96 (45%)
Search-aware forecast70.0%725 / 6,72210.8% [10.1%, 11.6%]25 of 96 (49%)
Pooled56.2%189 / 5,3993.5% [3.0%, 4.0%]6 of 96 (24%)
Search-unaware risk control29.2%8 / 2,8020.3% [0.1%, 0.6%]0 of 96 (1%)
Direct high-NN risk control26.9%6 / 2,5850.2% [0.1%, 0.5%]0 of 96 (2%)
Direct high-NN RC (cost-matched)12.5%2 / 1,1990.2% [0.0%, 0.6%]0 of 96 (1%)
SARCO (m=16m = 16)14.7%3 / 1,4090.2% [0.0%, 0.6%]0 of 96 (1%)
SARCO (m=32m = 32, N≥32N \ge 32)20.4% (of 6,400)3 / 1,3030.2% [0.0%, 0.7%]0 of 64 (2%)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.