- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies how much improvement an iterative reasoning system can obtain from feedback about a hidden specification. It defines an information budget for the feedback transcript and compares performance with a “ghost” interaction driven by an independently sampled decoy specification. Standard information inequalities yield an improvement ceiling, which the authors turn into statistical tests for information paths outside the accounted feedback. The paper also proposes a tight ceiling for bounded transcript alphabets, structural caps for verbose feedback, and training-time conservation results. Validation includes an exactly solvable synthetic family and controlled code-repair protocols. Unrestricted text feedback is evaluated descriptively through pricing, while certificates are reported for constrained protocols and a separate residual audit.
Critical strengths
-
I liked how the paper connects an explicit ghost baseline, an information budget, and a statistical audit. Theorem 1 bounds improvement over the decoy-driven interaction, while Proposition 2 gives an exactly tight ceiling for declared feedback alphabets. Together, these results provide a precise way to assess whether accounted feedback can explain observed gains.
-
I think the residual audit is a useful contribution because it accounts for information that legitimate feedback has already disclosed. Appendix G, Propositions 6–8, develops the corresponding posterior and e-value construction. The comparison in Table 6 between legal overfitting and exploiting an undisclosed test provides a meaningful controlled demonstration of why this conditioning matters.
Critical weaknesses
-
I think the practical evidence remains limited to controlled settings. The declared-budget experiments use deliberately constrained protocols, and the successful residual-audit exploit in Appendix G is scripted around model-generated programs. The model itself did not meaningfully exploit the supplied leak. These results support the proposed mechanisms, but provide limited evidence about detecting leakage arising naturally in deployed reasoning systems. I view this as a limitation of the demonstrated practical scope rather than a failure of the formal guarantees.
-
I had a concern about the broader generalization in Appendix D, lines 1073–1075. This passage appears to extend the uniform-prior success bound by replacing (m\ln 2) with general conditional entropy. High entropy alone can coexist with a dominant answer that is easy to guess without feedback, so this substitution seems to require additional assumptions. I would appreciate clarification.
Questions for authors
-
Could you clarify the assumptions behind the entropy-based extension in Appendix D, lines 1073–1075? In particular, is an additional uniformity assumption intended, or is a different bound needed for nonuniform specifications? This would help establish the scope of the claimed generalization.
-
In Theorem 4, should explicitly include both the feedback transcript and the final output, as used in Appendix C’s proof? I think this interpretation addresses the policy-randomness concern, but the main text previously defines as feedback alone. Making the definition consistent would clarify exactly which information quantity the conservation guarantee bounds.