When Success Is Invalid: Latent Validity in AI-Guided Machine Learning Research

A machine-learning artifact can satisfy its nominal evaluator without supporting the scientific claim attached to it. Latent validity is the satisfaction of claim-critical requirements left uncertified by a particular evaluation interface. Our central contribution is a claim-to-control compilation path: structure the residual from the claim and interface, map each requirement to a complete mediation cut or detection boundary, and compile it into a guard or evidence rule, a channel-relative completeness test, a trivalent witness, and a fail-closed release decision. We evaluate components of this lifecycle. A diagnostic archive finds that 113/824 artifact passes fail a separate process proxy; a frozen historically informed roster shows human-adjudicated candidate failures recurring within and across requested routes. In blinded archived packets, the human labeler resolves process evidence for 33/34 complete traces but only 5/116 incomplete traces, and every resolved incomplete-trace verdict is fail. Only 84/135 nominal intervention pairs contain an application receipt for the requested executable control. Every constructed integrity case matches the frozen oracle: intact compliant or allowed-control streams release, intact violations fail, and corruptions block. These results characterize the claim-to-control lifecycle rather than estimating prevalence, ranking agents, or establishing causal guard efficacy: whether computational success supports a scientific claim depends partly on admissible process evidence, not on the final artifact alone.