Training-Path Equivalence: Validating Implementation Substitutions

Efficient neural kernels are typically validated through output and gradient agreement at fixed weights. Training introduces a different requirement: a replacement must preserve the reference behavior after repeated parameter and optimizer updates. We introduce Training-Path Equivalence (TPE), a framework that turns this requirement into an explicit substitution contract and uses short paired training paths to screen candidates. In a fresh evaluation of two previously frozen policies on 35 condition-seed units, the step-50 paired screen identifies all 20 contract rejections and all 15 passes; local checks at fixed numerical tolerances through step 200 miss three rejections and reject five passing units. A separately timed Math-to-Flash attention substitution, accepted in five fresh seeds under the lower-learning-rate 5,000-step contract, retains a 1.357x full GPU training-step speedup. Matched reference-seed calibration across nine conditions independently reproduces rejection and acceptance outcomes and quantifies the accepted pair's small functional gaps. Complementary long-horizon studies, replications up to 1.003B parameters, and optimizer-state interventions characterize how substitution decisions depend on training requirements. Together, these results establish paired-path validation as a practical complement to local numerical checks for selecting efficient training implementations.