Belief Validity Horizon: Predicting World Model Reliability Under Distribution Shift

Model-based RL agents have no built-in mechanism to detect when their world model has gone stale, so they continue planning over imagined futures even after the real environment dynamics have shifted. We address this by training two lightweight heads on a DreamerV3-style RSSM to extract a validity signal directly from the latent state before failure occurs. The training proceeds in two phases where the world model and both heads train jointly in Phase 1, and then the RSSM is frozen while the heads continue training stop-gradient in Phase 2, which preserves the world model representation without interference. ValidityHead predicts tau-hat, the expected number of steps until KL divergence between prior and posterior exceeds a fixed threshold, using twohot cross-entropy over 64 symlog-spaced bins. HazardHead models shift arrival as a discrete-time competing-risk survival process and outputs a monotonically non-increasing survival curve S(t) over 16 discrete intervals. The stop-gradient isolation holds empirically, with world model KL changing by only -0.0015 nats over 100,000 Phase 2 training steps, which is well below the 0.5-nat threshold that would indicate representation drift. On SensorDrift, where noise drift is deterministic and monotonic, ValidityHead reaches a concordance index of 0.963 on 25,000 evaluation samples compared to 0.873 for a reconstruction-error baseline. On ShiftPendulum, where shift timing is geometrically distributed and therefore memoryless, the method returns 0.507, which is effectively random and consistent with what theory predicts. We report both results together because a method that correctly produces no signal in an environment where no signal is theoretically possible is behaving as intended, and establishing where the approach does not work is as integral to the contribution as establishing where it does.