The Verifier-Bit Ledger: A Leakage Ceiling and an Executable Audit for Verifier-in-the-Loop Reasoning
Abstract
Verifier feedback drives much of the recent progress in iterative reasoning, yet the channel that carries it can also carry the answer: on hidden-test pass rate, a model that learns to repair looks identical to one fed the hidden tests, and contamination audits, which cover only static training data, cannot say with a guarantee how much of a gain the feedback could have supplied. We introduce the verifier-bit ledger. Its budget meters what each round of feedback reveals about the hidden answer; its ceiling bounds the improvement any policy can buy with that budget over a ghost baseline whose verifier is keyed to decoy answers, and is tight to first order as the budget vanishes and exactly tight for declared feedback alphabets. Inverting the ceiling yields an executable audit that certifies an unlogged information path, at a chosen confidence level, whenever improvement outruns the budget; we characterize exactly when it can fire, and it is deployable today for low-rate or structurally capped feedback. Two conservation theorems show that compliant training cannot smuggle answers and that amortized gains are prepaid. On a synthetic family where every quantity is exactly computable, the bounds close with matching constants and strong injected side channels are caught at the predicted confidence. On real code repair, free-text feedback is priced rather than tested, and a planted leak lifts repair success by five points over the ghost; on three controlled real-task protocols with declared or capped feedback, planted leaks trigger the certificate and honest runs never do. A residual audit conditioned on legally disclosed feedback extends certificates to full first-failure feedback on randomized suites, catching a one-test exploit the ceiling cannot. Verification information can be budgeted, audited, and priced like computation.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.