acceptodds
Under review as a conference paper at ICLR 2027

True Claims, False Answer: Spot-Check Debate Is Limited by What the Honest Side Claims

Abstract

Spot-check debate lets a weak judge verify only the values that debaters choose to claim, on the premise that a false answer must rest on a checkable false claim. A dishonest debater that claims only values shared by the true and false computations defeats this premise: every check confirms it. Against such an attacker, a check helps only in a covered debate, where the honest debater claims a discriminating value (one on which the two computations differ) and the judge checks it. By construction no check contradicts the attacker, which makes it an instrument for measuring the honest side and the judge. We build a testbed that records, for every debate, whether the honest side held such a value, claimed it and had it checked: 600-step arithmetic programs that an oracle verifies step by step but the judge cannot execute, with one LLM in all roles. With the full true trace, one check suffices (accuracy 0.98). With a thinned trace, coverage is lost on the honest side: our open-weight honest debaters claim a discriminating value in only a third to half of the debates in which they hold one. The judge checks mostly what it is asked to check and, without coverage, sides with whichever debater a check confirms, a rule that is Bayes-optimal against ordinary liars. Against ordinary liars the failure does not show, because their own false claims get checked: when the honest side holds only every 60th value, the same honest debater and judge score 0.86 against an ordinary liar but 0.53 against the attacker, close to the 0.47 of no debate. The gap replicates with Qwen2.5-72B in every role. With every 10th value, scripting the honest side to claim all it holds and request checks of its latest values nearly doubles coverage and raises accuracy from 0.60 to 0.84, only part of that gain through coverage. Evaluations of debate should include such an attacker and report how often the honest side claims the discriminating values it holds and how often these are checked.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
scalable oversight, AI debate, obfuscated arguments, LLM-as-a-judge, AI safety, red teaming

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_1w5cOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies “spot-check debate,” a debate protocol where the judge can check only a few values that the debaters choose to state. Debate assumes that the honest debater can point to a false claim that the lying debater makes, and the judge can catch the false claim by checking it; this paper considers a setting where the values that the lying debater computes differ from the honest debater’s only after a step where a lie is inserted. These values where the debaters differ are called “discriminating values.” Then the lying debater can state only values where it and the honest debater agree, and if this is the case, the judge can only catch the lying debater if the honest debater states a discriminating value that the judge checks.

The authors build a testbed of debate protocols for long arithmetic programs and check for each debate whether the honest debater held a discriminating value, whether the honest debater stated a discriminating value, and whether the judge checked a discriminating value. They find that most of the lost accuracy is due to the honest debater not stating a discriminating value or not asking to have such a value checked.

Critical strengths

-- The point that, in spot-check debate, a judge that does not check a discriminating value cannot catch a lying debater that states only true values is clear and interesting. The paper carries out careful experiments to investigate whether the honest debater states a discriminating value that the judge checks, i.e., whether spot-check debate works.

Critical weaknesses

-- The setting is somewhat narrow. The authors run experiments using one synthetic task family (arithmetic programs), and the fixes were run with one main model. Also, the lying debater is constructed by the authors, so the paper shows this strategy breaks the protocol, but models may not use this strategy.

Questions for authors

None

3 comments
AuthorsOpenReview ↗

Response to Reviewer 1w5c

Thank you for the review. On your three points (details in the general response):

One task family. We agree that, apart from the exploratory check below, the experiments use one synthetic task family. The attack itself is not specific to arithmetic: it needs only that the false derivation share its checkable values with the true one up to the false step, as when one step of a proof, an execution trace or a retrieval chain is altered; in quote-verified debate, a dishonest debater that quotes only real passages is in this position, since no check of its quotes can fail. As supporting evidence, Table 2 includes an exploratory check on twelve classic loop algorithms, written as Python programs, where the gap recurs (0.86 against an ordinary liar vs. 0.67 against the attacker; post hoc, p = .02, and p = .09 and .17 after dropping two and three of the algorithms). The revised §5 states which parts of the argument transfer to other domains and how to test them there.

Fixes run with one main model. Your concern was warranted, and testing it showed that what carries over is the diagnosis rather than the script. On a thinned witness, the submission had repeated only the late-claim prompt with Qwen2.5-72B, which raised claiming but not accuracy (§3.5). For the revision, we repeated the main model's best fix, in a registered run, with Qwen2.5-72B in every role. It brought claiming to its ceiling but raised coverage only from 0.18 to 0.24 and accuracy from 0.61 to 0.69 (one-sided p = .12 and .14): at this model's depth only one or two of its three requested values lie in the region, so the check, not the claim, was the binding step.

Requesting only the latest held value, which lies in the region whenever any held value does, raised coverage to 0.48 and accuracy to 0.80 (one-sided p < 10⁻⁶ and p = .002), level with the ordinary liar (0.82). This variant was designed after the first result and registered before its run, and it is the last variant we ran on this model; both results are in Table 2, §3.5 and the updated Limitations, and the revised §5 states that a request rule must fit the task's structure.

The constructed attacker. We agree that current models do not adopt this strategy unprompted; the paper treats the attacker as an instrument, not as a prediction of model behavior. A safety argument must hold for every strategy a protocol permits, and our findings concern the honest side and the judge, which the attacker serves to measure. Instruction alone moves models toward the strategy without reaching it: handed only the shared values, a cover story lowers the main model's false-claim rate from 43% to 17%, and a strict rule lowers GPT-5.5's from 86% to 13% (App. Table 3, App. F.18); the filter removes the rest. A judge that follows confirmations, which is optimal against ordinary liars, would also reward a debater in training for claiming only shared values.

We hope this addresses your three points; if it does, we would be grateful if you would take the revision into account in your assessment. We are happy to answer further questions.

Thank you for the detailed response and additional experiments. While these strengthen the paper, I still have some reservations about the generality of the results beyond the settings studied. I remain positive about the paper but will keep my score.

Thank you for considering our response and the new experiments, and for remaining positive about the paper. We agree that how far our measured rates and fixes carry beyond the settings studied remains to be tested; the revised §5 states which parts of the argument we expect to transfer and what must be checked again in each new setting.

Reviewer_zYtzOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper examines an automated debate system involving two participants and point-based claim verification. The participants are programs that perform arithmetic operations (or large language models—LLMs—emulating such operations), while the judge is an LLM. The authors investigate the issue of "dishonest" responses from one of the participants, which can skew the judge’s evaluation because the correctness of the response cannot be verified. The authors develop and describe a test environment and dataset, and conduct large-scale experiments using various LLMs and deterministic programs. The results clearly demonstrate a performance gap between existing evaluation methods and the proposed approach. The authors conclude that, to obtain more accurate quality metrics, evaluations of such systems should always include a "dishonest" participant.

Critical strengths

The problem is precisely formulated, the methodology is described in a rigorous and thorough manner, and the hypotheses put forward are supported by convincing experimental evidence. All conclusions are accompanied by detailed clarifications and explanations that leave no room for ambiguity. The limitations of the proposed approach and potential avenues for improvement are clearly outlined. The paper represents a well-crafted piece of academic research.

Critical weaknesses

The article is written in a very "dense" (though technically correct) manner, making it challenging to read. The domain proposed for evaluation is introduced in another, as-yet-unpublished paper by the same authors; this complicates the reader's understanding and leaves the rationale for selecting this specific domain unaddressed. Certain research parameters—such as program size and the step numbers for interventions—are introduced abruptly, with their justifications provided only in the appendices. It is also worth noting that, while the research itself is undoubtedly of high quality, it focuses on an extremely narrow niche within the broader AI debate and does not, in fact, explore the practical application of its findings.

Questions for authors

Could you comment on the rationale for choosing this domain (arithmetic programs) and clarify the extent to which the research findings might generalize to other domains?

3 comments
AuthorsOpenReview ↗

Response to Reviewer zYtz

Thank you for the careful review and the question on the choice of domain.

Your question: why arithmetic programs, and what generalizes. What we measure is coverage: how often the honest side claims a discriminating value (one on which the true and false computations differ) and the judge checks it. Measuring it needs four things at once: an oracle for every intermediate value, knowledge of where the two computations diverge, a judge whose weakness can be set, and an honest side whose knowledge can be set. Long arithmetic programs provide all four; question-answering evaluations neither mark which checks could separate the answers nor set what the honest side knows.

What generalizes is the structure of the attack: it needs only that the false derivation share its checkable values with the true one up to the false step, which holds whenever one step of a multi-step derivation is altered (a proof, an execution trace, a retrieval chain). The same structure appears in quote-verified debate (Khan et al., 2024): a dishonest debater that quotes only real passages cannot fail a check of its quotes. Empirically, the gap recurs on twelve classic loop algorithms (exploratory; 0.86 vs. 0.67) and with a second model family (Table 2). The rates and the request rule do not transfer unchanged and must be measured in each setting; §5 now describes how to do this where no step-wise oracle exists. See the new paragraphs "Why arithmetic programs" (§2) and "Beyond arithmetic programs" (§5).

One clarification on the setup, which our dense writing may have obscured: both debaters and the judge are LLMs (one model plays all three roles), and the arithmetic programs are only the task. The scripted parts are the harness (the oracle and the filters that remove disallowed claims) and, in §3.5, an honest side whose claims and, in some conditions, check requests are set by a script. The gap we report holds the honest debater and the judge fixed and changes only the dishonest debater: with every 60th value held, they score 0.86 against an ordinary liar but 0.53 against a dishonest debater that makes no claim a check could contradict (0.47 without debate). Evaluations should therefore include such a dishonest debater alongside an ordinary liar, and report how often discriminating values are claimed and checked (§5).

Dependence on the cited framework. The revision makes the paper self-contained on this point: §2 now states what the programs are, how they are generated and checked, and why we chose them (details in App. D), so the other paper is not needed to follow this one. We use three of its components, the program generator, the perturbation gates and the depth rule (App. Table 5), and all three ship with the supplementary code. Each program is executed and kept only if its trace verifies, each lie must pass the executable gates, the depth is calibrated on our own judges, and no result depends on that work's findings.

Parameters. §2 now gives the reason for each choice where it is introduced (the first two were stated there, briefly, in the submission; the other three are new):

  • Program size: D is set by a rule, the shallowest depth at which the judge alone is right at most 65% of the time (0.47 at D = 600).
  • Lie position: the final 20% of the program, so that its region is short and a thinned witness often holds none of its values.
  • Witness: at D = 600, every 10th value rarely leaves the honest side without a discriminating value, so what fails there is claiming and checking; every 60th often does, so holding fails too (60 was chosen over 80 in a calibration run before the main runs).
  • Protocol constants: two rounds, three requests and a 200-word cap, fixed before data and not tuned.
  • One check: with a full witness it already gives accuracy 0.98, and more checks cannot raise coverage above what the honest side claims.

Density and practical use. We wrote the added paragraphs for readers outside debate, and we would welcome pointers to any passage you found hardest to follow, which we will revise for the final version. The practical output is the evaluation checklist in §5 and the protocol advice (have the honest side claim what it holds and point the judge to values downstream of the disagreement); the new runs with Qwen2.5-72B apply that advice to a second model family (general response, point 2).

If the revision resolves the presentation issues you raised, we would be grateful if you would consider this in your assessment. We are happy to answer further questions.

Reviewer_zYtzOpenReview ↗

Thanks for the clarification

Your comments were really helpful, now I understand the paper much better. However I will keep my score since it was mostly related with the contributions rather than writing. But all of the reviewers are quite positive about the work.

Thank you for reading our response and for the kind words. We are glad the clarifications helped.

Reviewer_CbzbOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Please see above

Critical strengths

Please see above

Critical weaknesses

Please see above

Questions for authors

Please see above

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response and summary of revisions

We thank the reviewers for their time and comments. In brief: a dishonest debater that claims only values shared by the true and false computations is never contradicted by a check, so a check helps the judge only in a covered debate, where the honest debater claims a discriminating value (one on which the two computations differ) and the judge checks it. Our open-weight honest debaters claim one in only a third to half of the debates in which they hold one; with every 60th value held, the same honest debater and judge score 0.86 against an ordinary liar but 0.53 against this attacker (0.47 without debate).

We have uploaded a revision; all new text is marked in red, and no experimental result of the submission has changed. Below are the three main points raised in the reviews, followed by the list of changes.

1. Why arithmetic programs, and what generalizes (zYtz, 1w5c). Measuring coverage needs four things at once: an oracle for every intermediate value, a known divergence region (so each claim can be labeled discriminating or not), a judge whose weakness can be set, and an honest side whose knowledge can be set. Question-answering evaluations neither mark which checks could separate the answers nor set what the honest side knows. Long arithmetic programs provide all four, so we use them as a measuring instrument rather than as a target domain (new paragraph in §2).

What transfers is structural: the attack needs only that the false derivation share its checkable values with the true one up to the false step, which holds whenever one step of a multi-step derivation is altered (a proof, an execution trace, a retrieval chain); in quote-verified debate, a dishonest debater that quotes only real passages is in the attacker's position, since no check of its quotes can fail. The gap also recurs on classic loop algorithms (exploratory) and with a second model family. What does not transfer unchanged are the rates and the request rule, which must be measured in each setting; where no step oracle exists, the checklist can be run on known-answer tasks with one planted error (new paragraph in §5).

2. The best fix on a second model (1w5c). We repeated the main model's best fix, in a registered run, with Qwen2.5-72B in every role. This script (claim all held values, request checks of the three latest) brought claiming to its ceiling but raised coverage only from 0.18 to 0.24 and accuracy from 0.61 to 0.69 (one-sided p = .12 and .14): at this model's depth the region is short, so only one or two of the three requested values discriminate.

Requesting only the latest held value, which lies in the region whenever any held value does, raised coverage to 0.48 and accuracy to 0.80 (p < 10⁻⁶ and p = .002, one-sided), level with the ordinary liar on the same tasks (0.82). This variant was designed after the first result and registered before its run, and it is the last variant we ran on this model; both results are reported (Table 2, §3.5, App. C and F.22). The coverage decomposition is what located the binding step.

3. The constructed attacker (1w5c). A safety argument for a protocol must hold for every strategy the protocol permits; control evaluations construct attackers for the same reason. The attacker is also our instrument: the findings concern the honest side, which leaves discriminating values unclaimed, and the judge, which follows confirmations; neither depends on whether a model adopts the strategy unprompted. Current models approach it under instruction without reaching it: handed only shared values, 17% of the main model's claims are false under a cover story, and 13% of GPT-5.5's under a strict rule; the filter removes the rest (new sentence in §2). Since following confirmations is optimal against ordinary liars (§3.4), a debater trained against such a judge would be rewarded for claiming only shared values, which no check contradicts.

Changes in the revision.

  1. Abstract: the script's effect is stated for the main model.
  2. §2: why arithmetic programs; how programs are generated and checked; the reasons for the witness strides, the late-lie position, the protocol constants and the single check; the attacker's accessibility.
  3. §2 footnote, App. B (Table 5), App. D and the framework's reference entry: what is taken from the cited framework and how programs are generated and verified, so that the paper can be read on its own.
  4. Fig. 1 caption: one LLM plays all roles, and the program is the task.
  5. Contribution (4), Table 2, §3.5, §3.6 and §5 (protocol advice, Limitations, Conclusion): the best fix repeated with Qwen2.5-72B, and a variant of it.
  6. §5: what transfers beyond arithmetic.
  7. App. A, C, E and F.22, and the Reproducibility Statement: registrations, multiplicity, names and details.
  8. Two references corrected.
  9. Supplementary material: adds the new records, registrations and a readout script.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.