acceptodds
Under review as a conference paper at ICLR 2027

Cryptographic Techniques for AI Safety via Debate

Abstract

AI debate aims to let a weak judge evaluate stronger AI systems by having competing agents expose each other’s errors. We study two mechanisms for strengthening debate: cryptographic commitments and private communication. Building on classical competing-prover protocols, we give debate protocols whose communication and judging costs scale polylogarithmically with computation length, in addition to input and security-parameter costs. These guarantees extend the standard debate model from PSPACE to EXP and accommodate black-box calls to tools, databases, or human feedback. With cryptographic commitments and random access to the input and transcript, the judge checks only a constant number of locations and makes at most one black-box call. We evaluate private communication in LLM debates where the agent defending an incorrect answer is stronger than its opponent and the judge; experiments show that privacy in debate can increase the protocol's accuracy and its robustness.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
debate, AI safety

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_9Qw3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether private communication and cryptographic commitments can reduce the cost of AI debate. Building on competing-prover protocols, it develops efficient debate systems for deterministic computations with oracle access. The cryptographic construction uses collision-resistant hashing and Merkle commitments to locate and check disagreements, and claims constant-operation judging under random access to the input and transcript.

Critical strengths
  • The paper addresses a meaningful problem in scalable oversight. Private communication and binding commitments offer concrete ways to restrict how a dishonest debater can adapt its claims.

  • The theoretical results cover both the possibilities and the limitations of the approach. The treatment of deterministic oracle computations, the communication barrier for computations with witnesses, and the use of longer transcripts fit together well.

  • The experiments provide encouraging evidence for private communication.

Critical weaknesses
  • The constant-operation judging result needs a more explicit protocol and security analysis.

  • The cryptographic assumptions and the scope of the cryptographic contribution require clarification.

  • The cryptographic distribution-verification extension is not fully developed in the submitted version.

Questions for authors
  • Theorem 4.2 is a central contribution, but the description in Section 4 and the proof in Appendix C leave several steps implicit. The proof states that an honest debater can identify an incorrect auxiliary computation or a violation of the protocol, after which the judge checks a constant number of locations. More detail is needed to establish this for arbitrary malicious behavior, including malformed messages, inconsistent transcript references, and false challenges. Merkle binding prevents inconsistent openings of a commitment; it does not establish that the committed computation is correct. The analysis should explain how every successful cheating strategy either violates binding or produces an error covered by a specified local check. The RAM model also needs a precise definition of word size, permitted operations, and the interface for submitting an oracle query. These details matter because constant-operation judging is stronger than polylogarithmic verification.

  • Section 4 permits debaters with exponential computational budgets. Its security requires collision resistance against the stated debater runtime, including oracle access, rather than ordinary polynomial-time security alone. Although the paper acknowledges this requirement, it should explain more clearly how the security parameter relates to the computation and adversary budgets, and how this affects the claimed efficiency. The construction also draws heavily on existing competing-prover and refereed-delegation protocols. The paper should identify precisely what is new in the cryptographic composition. A direct comparison with the closest constructions would help distinguish an application of established tools from a new protocol-level result. The concern is not that the paper must introduce a new cryptographic primitive, but that its incremental contribution needs to be established more clearly.

  • Appendix E gives a concrete information-theoretic protocol, whereas the cryptographic version in Appendix E.5 relies on distribution commitments, authenticated sampling, and succinct proofs about committed histograms. The latter also requires the property circuit to be fixed or preprocessed, and leaves further details to a future version. These conditions should appear explicitly in the theorem statement. In particular, online verification costs should be separated from preprocessing costs, and the assumptions needed for the succinct proof system should be distinguished from collision-resistant hashing. The submitted version should include the full protocol and its soundness analysis, or narrow the claim to reflect the construction outline currently provided.

2 comments

Thanks again for the review. We'd love to hear if our response addressed your concerns.

Thank you for the author's explanation of the RAM model and security assumptions, but the proof for Theorem 4.2 is still insufficient. I tend to maintain the original score.

Reviewer_DAB8OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies private communication and cryptographic commitments for AI debate. It proves that these mechanisms can reduce communication and judging costs for formal computation tasks. Preliminary image-classification experiments also show that private debate can improve accuracy.

Critical strengths
  1. The theoretical results explain how a weak judge can check complex computations with limited work.
  2. The paper identifies both useful protocols and limits on what efficient debate can achieve.
Critical weaknesses
  1. The formal guarantees concern deterministic computations and oracles, leaving a gap to noisy judgments and open-ended reasoning.
  2. The experiments test privacy in a narrow image task rather than the full cryptographic protocols.
Questions for authors
  1. Which guarantees remain valid when tool responses or human judgments are noisy?
  2. How much of the improvement comes from privacy rather than differences in the arguments or evidence selected?
  3. What practical costs would arise from implementing the commitment-based protocol with LLM debaters?
2 comments

Thanks again for the review. We'd love to hear if our response addressed your concerns.

Thank you for your detail response. This has address all my question I decide to raise my score.

Reviewer_1wF3OpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper builds on competing-provers protocols and leverages private communication and cryptographic commitments to enhance AI debate. The contributions of this paper are as follows: (1) the protocols proposed in this paper make it possible to realize AI debate through private communication or cryptographic commitments; (2) the proposed protocols enable the judge to reach a verdict in polynomial time; (3) cryptographic commitments reduce the judge’s workload; and (4) the paper proves that the proposed scheme is a deterministic computational scheme. According to the data in Table 2 of the experimental results, the private protocols proposed in this paper indeed achieve an improvement in honest win rate compared with public protocols.

Critical strengths
  1. The paper employs cryptographic commitments to fix the computational state claimed by debaters, using the binding property to prevent them from modifying submitted data after being challenged, thereby providing an effective safeguard for the consistency and accountability of the debate process.
  2. The scheme fully exploits the local opening and verifiable splitting capabilities of Merkle commitments, using binary search to progressively locate disputed computational steps and data positions, thereby avoiding the transmission of the full computational state and effectively reducing the communication cost of debate.
  3. The paper combines the commitment mechanism with interactive checking by both parties, so that disputes over complex computations are ultimately reduced to the verification of local state transitions. Under a model that supports random access to inputs and debate transcripts, the judge only needs to inspect a constant number of positions, demonstrating the application value of commitment techniques in reducing supervision costs.
Critical weaknesses
  1. The overall writing style of the paper is not sufficiently accessible to readers with a purely AI background; it is more characteristic of a cryptography research paper.
  2. The Preliminaries section is intended to introduce background knowledge so that readers can better understand the paper, but this section does not introduce the key concept of AI debate.
  3. In AI papers, “model” generally refers to models such as Sonnet and Mistral, which is inconsistent with the notion of “model” in cryptography; the authors are advised to distinguish between the two in the paper.
  4. The experiments are relatively limited and cannot adequately support the contributions claimed in the paper, such as the claims regarding reaching a verdict in polynomial time and reducing the judge’s workload.
  5. The scheme implements commitments using a tree structure. For data of length LL, a single-point opening still requires transmitting and verifying an authentication path of logarithmic length, resulting in O(λlog⁡L)O(\lambda \log L) bits of communication overhead and O(log⁡L)O(\log L) hash computations. Although the paper reduces the judge’s direct workload through interactive checking, this overhead is still borne by the protocol participants. The authors are advised to further discuss the performance bottlenecks of the commitment layer and to examine recent developments in vector commitments, polynomial commitments, and batch openings, analyzing their applicability to the current protocol as well as the trade-offs among proof length, computational cost, and security assumptions, so as to identify possible directions for further improving the performance of the scheme.
Questions for authors
  1. Does “These protocols” at the beginning of page 2 refer to the protocols proposed in this paper or to existing protocols in the field?
  2. In the experimental validation section, only one example, “cat vs. dog,” is provided, which I consider insufficient; more examples should be added. In addition, I did not find a comparison with baselines.
  3. Table 2 is presented directly, but Table 1 cannot be found. What is the content of Table 1?
  4. In the part shown in Table 2, the judge model is either the same as the honest debater or the same as the dishonest debater. What would happen if the judge model were different from both?
  5. Is there a relationship between the amount of information revealed and the honest win rate?
1 comment

Thanks again for the review. We'd love to hear if our response addressed your concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.