Certifying Tactical Mistakes in Self-Play Reinforcement Learning

Self-play scores can conceal decisions that discard a forced win because sampled opponents fail to refute them. We present a certificate-based audit that turns local tactical proofs into executable opponent responses and quantitative guarantees on their value. The audit distinguishes mistakes on the realized deal from winning actions identifiable using the player's observations. Each denial witness supplies a payoff cap for a responding coalition, reducing certified response optimization to optimal stopping under ordinary self-play. A paired martingale-dual score bounds a fixed switching rule's regret within this response class, using exact conditional values or independent successor simulation. We give a sharp coefficient-one residual bound for predictor-greedy rules and an exact-boundary absorption result. In complete games of a sampled proximal policy optimization (PPO) policy, a simulated switching rule has mean certified score $0.544$ points per game, with stopping regret below $0.0985$ points at 95% confidence. Across $64$ enumerable stochastic endgames, exact regret is $0.00109$ and the paired dual bound is $0.01003$. Refutation replay denies first place on all $1,357$ audited throws. As a further use of the audit, reusing the certified action sets in training raises sampled retention from $0.781$ to $0.957$ on a fixed Tien Len challenge set over five independent runs.