From Marginal Matching to Sampler Compatibility: Auditing Independent Latent Recombination

Split-latent generators reconstruct an observation from content and style codes encoded from that same observation, yet generate by sampling the two codes independently. We formalize this train–generation shift as sampler compatibility between the encoded conditional joint and the declared sampling law . An exact expected-conditional KL identity decomposes incompatibility into style-prior mismatch, condition information in style, content-prior mismatch, and within-condition code dependence, so aggregate prior matching certifies only the first clause. We turn the identity into an audit: a direct joint test, clause-specific diagnostics, and decoder and task checks that separate latent compatibility, decoded transport, and generation competence. An exact-marginal counterexample, image and known-factor audits, and a released-checkpoint boundary case show that aggregate matching can coexist with strong leakage that the decoder actually uses. The audit also guides the repair. On retrained MNIST models, a sampler fitted to all four clauses raises generation accuracy from 33.9% to 96.4% and closes 97% of the Fréchet gap to the paired-code ceiling with higher recall and a memorization ratio near one, whereas a style-only repair recovers part of the gap and a shrunk class-style shortcut collapses diversity. A capacity-matched ablation shows that fitting the class-conditional marginals suffices, so content and style stay independent within a class but style must depend on the class. Our encoder-side contract training instead lowers the joint discrepancy by 27–38% yet worsens decoded transport in every matched pair and makes the model harder to repair.