Adaptive Precision Allocation Can Starve Itself: Consequence-Aware Allocation for Bounded World Models
Abstract
A recurrent world model operating under a hard per-transition representation budget must decide where to spend its available precision. We show that adaptive precision allocation can starve itself: when source-referred importance is repeatedly measured through the precision assignment it is used to update, low precision attenuates the very signal needed for a group to recover rate. We formalize this reference-point failure through multi-step predictive consequence—how strongly current representation error propagates into future latent predictions under the same action continuation—and show that the resulting feedback admits locally absorbing starvation fixed points. Measuring all groups from a common allocation-independent reference removes the assignment-dependent gain. Building on this analysis, we introduce CAP-WM, which predicts state-dependent multi-step consequence and converts it into an exact recurrent precision allocation by reverse water filling at a no-bypass interface. A linear–Gaussian characterization recovers the same groupwise allocation rule under a standardized group-block-isotropic specialization. In trained models, adaptive source-referred refresh reaches active groups from ordinary initializations with probability –, versus – for memory-referred refresh. Under a directly countable serialized interface, CAP-WM reaches retention of the no-channel DreamerV3 reference at bits/coordinate, while the tested adaptive controls that reach this criterion require ; at the tight nominal rate, CAP-WM improves DMC-GB2 IQM by over Uniform and over One-step consequence. The additional multi-step benefit is conditional rather than universal, resolving when precision is restrictive and predictive dependence extends beyond one latent transition. More broadly, the results expose a design hazard for adaptive resource allocation: an assignment can become self-reinforcing when it attenuates the importance signal used to revise itself.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.