- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates whether private communication and cryptographic commitments can reduce the cost of AI debate. Building on competing-prover protocols, it develops efficient debate systems for deterministic computations with oracle access. The cryptographic construction uses collision-resistant hashing and Merkle commitments to locate and check disagreements, and claims constant-operation judging under random access to the input and transcript.
Critical strengths
-
The paper addresses a meaningful problem in scalable oversight. Private communication and binding commitments offer concrete ways to restrict how a dishonest debater can adapt its claims.
-
The theoretical results cover both the possibilities and the limitations of the approach. The treatment of deterministic oracle computations, the communication barrier for computations with witnesses, and the use of longer transcripts fit together well.
-
The experiments provide encouraging evidence for private communication.
Critical weaknesses
-
The constant-operation judging result needs a more explicit protocol and security analysis.
-
The cryptographic assumptions and the scope of the cryptographic contribution require clarification.
-
The cryptographic distribution-verification extension is not fully developed in the submitted version.
Questions for authors
-
Theorem 4.2 is a central contribution, but the description in Section 4 and the proof in Appendix C leave several steps implicit. The proof states that an honest debater can identify an incorrect auxiliary computation or a violation of the protocol, after which the judge checks a constant number of locations. More detail is needed to establish this for arbitrary malicious behavior, including malformed messages, inconsistent transcript references, and false challenges. Merkle binding prevents inconsistent openings of a commitment; it does not establish that the committed computation is correct. The analysis should explain how every successful cheating strategy either violates binding or produces an error covered by a specified local check. The RAM model also needs a precise definition of word size, permitted operations, and the interface for submitting an oracle query. These details matter because constant-operation judging is stronger than polylogarithmic verification.
-
Section 4 permits debaters with exponential computational budgets. Its security requires collision resistance against the stated debater runtime, including oracle access, rather than ordinary polynomial-time security alone. Although the paper acknowledges this requirement, it should explain more clearly how the security parameter relates to the computation and adversary budgets, and how this affects the claimed efficiency. The construction also draws heavily on existing competing-prover and refereed-delegation protocols. The paper should identify precisely what is new in the cryptographic composition. A direct comparison with the closest constructions would help distinguish an application of established tools from a new protocol-level result. The concern is not that the paper must introduce a new cryptographic primitive, but that its incremental contribution needs to be established more clearly.
-
Appendix E gives a concrete information-theoretic protocol, whereas the cryptographic version in Appendix E.5 relies on distribution commitments, authenticated sampling, and succinct proofs about committed histograms. The latter also requires the property circuit to be fixed or preprocessed, and leaves further details to a future version. These conditions should appear explicitly in the theorem statement. In particular, online verification costs should be separated from preprocessing costs, and the assumptions needed for the succinct proof system should be distinguished from collision-resistant hashing. The submitted version should include the full protocol and its soundness analysis, or narrow the claim to reflect the construction outline currently provided.
2 comments
Thank you for the author's explanation of the RAM model and security assumptions, but the proof for Theorem 4.2 is still insufficient. I tend to maintain the original score.
Thanks again for the review. We'd love to hear if our response addressed your concerns.