More Evidence Makes Advice Easier To Certify And Harder To Use
Abstract
The same observations can support a reliability certificate for external advice and update the fallback learner. As evidence accumulates, certification can improve while the advice's learner-relative value shrinks. We call this remaining value actionability, which determines whether the advice is still worth using relative to the fallback. One might expect advice to remain actionable once enough evidence supports certification. We establish that, as confidence tightens, early certification disappears, so nonvanishing success must reach a learner state that preserves enough actionability for certification to finish. The best achievable probability of reaching such a state sets a high-confidence ceiling on certified-and-actionable success. Reachability depends on the advice’s remaining advantage over the fallback, how strongly new evidence pushes the fallback toward the advice, and whether a sufficiently favorable random evidence excursion actually occurs. More informative evidence can still lower success by shrinking this advantage.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.