TMOT: Geometric Optimal Transport over Target Representations for Speculative Verification

Speculative decoding reduces the serial cost of autoregressive generation, but its gains depend on how draft tokens are verified. Token-identity and probability-only comparisons do not explicitly represent relations between distinct tokens. We formulate predictive-distribution comparison as probability-measure alignment under a ground geometry: binary token-identity cost recovers total variation, while a cost induced by target token representations replaces uniform mismatch with graded geometric mismatch. TMOT (Target-Manifold Optimal Transport) implements this view using truncated, entropy-regularized, finite-step transport. Its verifier factorizes candidate-specific probability admissibility and distribution-level geometric compatibility, with entropy and sharpness adapting the allowable discrepancy. The resulting module is training-free and can be integrated into a speculative decoding pipeline without changing model parameters. Code-generation and mathematical-reasoning experiments yield up to $1.90\times$ end-to-end speedup, with task-dependent accuracy recovery and a small measured verification-time share. These results support a quality–efficiency trade-off for geometric relaxation; TMOT is a lossy verifier and does not preserve the exact target sampling distribution.