Epistemic Representation under Predictive Compression: Separation, Excess Rate, and Collapse

Can a compressed predictive representation preserve the information needed to evaluate a future measurement? We study settings in which an epistemic functional of a parameter-indexed observation experiment is recoverable from the complete predictive marginal before compression. We characterize residual sensing-context information at the predictive rate–distortion boundary. For a coordinate-weighted Gaussian model, we prove an all-encoder stability bound and derive the exact additional rate required for prescribed log-Fisher fidelity, including achievability. We then connect this mechanism to the native predictive KL: marginalizing an uncertain target suppresses predictive sensitivity to measurement quality while leaving log-Fisher sensitivity unchanged. For Gaussian predictive-family decoders, a global natural-KL theorem gives a strictly positive epistemic excess-rate bound at positive primary predictive rate. We test the mechanism with stochastic neural bottlenecks under the same natural-KL loss. A validation-locked candidate sweep followed by ten independent replication seeds per target-uncertainty setting yields 40/40 predictive-selected models satisfying predictive fidelity but 0/40 satisfying epistemic fidelity, whereas 40/40 epistemic-aware models satisfy both. The empirical-mixture mutual-information gap is 1.05–1.16 nats, while aggregate-prior mismatch is small, and binary sensing-action error falls from approximately chance (0.499) to 0.137–0.140. These results quantify how information present in a predictive distribution can remain costly to preserve under predictive compression.