Orthogonality Is Not Realizability: Signed Generative Risks and Probability-Constrained Counterfactual Learning

Orthogonal scores remove first-order nuisance bias, but their empirical targets need not be probability laws. We characterize this gap through a failure–recovery theory for proper generative risks. We construct smooth nuisance estimates whose \(L_4\) errors vanish while their pseudo-density ratios remain negative on rare covariate regions. For categorical log risks, we give recovery conditions in minimum cell mass, observations per stratum, and nuisance-product error, with separate counterexamples isolating sampling and nuisance effects. Within a released counterfactual diffusion architecture, growing denoising-feature moments yield an exactly integrated objective that is unbounded below even with nuisance-product error \(o(n^-1/2)\). We formulate orthogonal embedding projection (OEP) as a minimum-distance probability repair of doubly robust feature estimates, preserving accuracy in the chosen distance and the nuisance-product error bound. Exact finite-feature guarantees and a Project STAR study connect the theory to computation.