acceptodds
Withdrawn from ICLR 2027

Adaptive Precision Allocation Can Starve Itself: Consequence-Aware Allocation for Bounded World Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

A recurrent world model operating under a hard per-transition representation budget must decide where to spend its available precision. We show that adaptive precision allocation can starve itself: when source-referred importance is repeatedly measured through the precision assignment it is used to update, low precision attenuates the very signal needed for a group to recover rate. We formalize this reference-point failure through multi-step predictive consequence—how strongly current representation error propagates into future latent predictions under the same action continuation—and show that the resulting feedback admits locally absorbing starvation fixed points. Measuring all groups from a common allocation-independent reference removes the assignment-dependent gain. Building on this analysis, we introduce CAP-WM, which predicts state-dependent multi-step consequence and converts it into an exact recurrent precision allocation by reverse water filling at a no-bypass interface. A linear–Gaussian characterization recovers the same groupwise allocation rule under a standardized group-block-isotropic specialization. In trained models, adaptive source-referred refresh reaches active groups from ordinary initializations with probability –, versus – for memory-referred refresh. Under a directly countable serialized interface, CAP-WM reaches retention of the no-channel DreamerV3 reference at bits/coordinate, while the tested adaptive controls that reach this criterion require ; at the tight nominal rate, CAP-WM improves DMC-GB2 IQM by over Uniform and over One-step consequence. The additional multi-step benefit is conditional rather than universal, resolving when precision is restrictive and predictive dependence extends beyond one latent transition. More broadly, the results expose a design hazard for adaptive resource allocation: an assignment can become self-reinforcing when it attenuates the importance signal used to revise itself.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
world models, adaptive resource allocation, representation learning, rate-distortion, predictive consequence, mixed-precision representation, model-based reinforcement learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_twYPOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies how to adaptively allocate a hard per-transition precision budget across groups of latent representations in a world model. The authors model precision allocation using a Gaussian channel, where the rate assigned to each group controls its distortion scale. They then introduce a multi-step sensitivity measure, defined as a weighted sum of squared Frobenius norms of prediction Jacobians, to quantify how perturbations to each group propagate into future latent predictions.

The authors first find a potential failure of measuring source-referred sensitivity at the current allocation: if one assigns low precision to a group, this will attenuate its measured sensitivity, and it can further reduce its allocation in later updates. Under assumptions, they show that this mechanism has locally absorbing starvation fixed-points. And those fixed-points even include ones that suppress highly consequential groups. To address this issue, their proposed method, CAP-WM, evaluates sensitivity at a common uniform reference allocation. They prove that this construction removes the allocation-dependent multiplicative term. For efficiency, they train a network that only depends on the current state to predict the reference-based sensitivity scores and use reverse water filling to allocate the precision.

The experiments compare CAP-WM with several allocation strategies on world-model benchmarks, using both the Gaussian channel and a serialized quantization interface. The results show improvements especially under tight precision budgets. The additional benefit of multi-step sensitivity over one-step sensitivity is the clearest when restrictive precision coincides with substantial predictive dependence beyond a single transition.

Critical strengths
  1. The authors theoretically identify a failure mechanism in adaptive precision allocation and propose a solution that provably removes the associated allocation-dependent attenuation.

  2. The numerical experiments are comprehensive and show performance improvements over the compared methods, especially under tight precision budgets.

  3. The experiments include targeted tests of the identified failure mechanism and the proposed correction, providing evidence beyond performance improvements alone.

Critical weaknesses
  1. The motivation for several modeling choices could be clearer, particularly the Gaussian channel added to the recurrent-state interface and the weighted multi-step Jacobian sensitivity measure. Explaining their connection to precision constraints and predictive distortion as well as motivation before introducing the formal definition would improve readability.

  2. A natural idea is to measure memory-referred sensitivity at the current adaptive allocation, which avoids the source-channel attenuation identified in Proposition 1. Appendix A.2.1 shows that this alternative largely avoids starvation by collecting and analyzing some statistics on allocation process. But I think a matched end-to-end world-model performance comparison would better support the practical advantage of the proposed method.

  3. The settings for the return comparison between common-reference and adaptive source-referred measurement in Section 5.3 and Table 31 could be more explicit. Were the variants trained from scratch under their respective measurement rules, fine-tuned from a shared checkpoint, or evaluated by modifying a pretrained model? This would help interpret the evidence for the proposed mechanism.

Questions for authors
  1. I would suggest providing more motivation and intuition for the Gaussian channel and the weighted multi-step Jacobian sensitivity measure to improve readability.

  2. Could the authors include an end-to-end world-model performance comparison against memory-referred sensitivity measured at the current adaptive allocation?

  3. I would suggest clarifying the settings for the return comparison in Section 5.3 and Table 31. In particular, please specify whether the variants were trained from scratch under their respective measurement rules, fine-tuned from a shared checkpoint, or evaluated by modifying a pretrained model.

Reviewer_V14COpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies how to allocate a fixed precision budget across groups of recurrent world-model features. It identifies a feedback failure in which reduced precision attenuates the sensitivity signal used to update the allocation, potentially leaving important groups starved of precision. CAP-WM measures multi-step predictive sensitivity at a common reference allocation, predicts this quantity from the current state, and applies reverse water filling. The paper provides a fixed-profile starvation analysis and a restricted linear-Gaussian characterization, with experiments on DMC variants, Atari, and additional world-model backbones. It reports the largest gains over one-step sensitivity under restrictive budgets.

Critical strengths

A clear analysis of a specific allocation failure. Proposition 1 and Appendix A.2 explain how allocation-dependent attenuation can sustain starvation, including starvation of consequential groups. The distinction between source- and memory-referred sensitivity is useful for diagnosing this class of allocator. This is a limited but meaningful contribution under the stated assumptions.

Critical weaknesses

1.The main experimental claims are not sufficiently supported. Atari scores do not match the stated metric. Table 39 reports a Seaquest human-normalized score of −0.207 for Uniform. With the standard random/human references of 68.4 and 42054.7, this implies a raw return of about −8,623, which is incompatible with Seaquest’s nonnegative score rewards. The authors need to explain the source of this discrepancy and whether it changes the aggregate results in Tables 24–25. github.com The bootstrap result does not match the printed returns under the usual sampling procedure. In Table 37, CAP-WM exceeds Uniform in 29 of 30 task-seed pairs by at least 2 points; the remaining difference is −2. Under task-stratified paired resampling of five seeds per task, every bootstrap IQM difference is positive. The probability of improvement would therefore be 1, rather than the 0.98 reported in Appendix B.3. A different sampling scheme might explain this, but it is not specified clearly enough in Appendix B.1. The reported point-estimate advantage itself is reproducible. The 33.1% bit-saving claim rests on an uncertain threshold crossing. At 4 bits, CAP-WM scores 739 [715, 762], against a 90% retention target of 738 (Table 6). Its advantage over multi-step value sensitivity at that budget is also unresolved: +18 [−6, 42] (Table 8). The 33.1% calculation is correct for the tested point estimates, but the paper has not established that this saving at the target performance holds once uncertainty is considered. These issues affect the reliability of the performance comparisons and a headline efficiency claim.

2.Two important method choices need stronger validation. The most direct alternative is missing from the trained-agent comparison. Measuring sensitivity directly with respect to stored memory removes the attenuation identified in Eq. (6). Figure 4 already shows that this memory-referred alternative avoids most collapse on fixed checkpoints. However, Tables 31–32 do not include an agent trained throughout with this alternative. It remains unclear whether the common reference improves control performance over this direct alternative. The learned score does not directly estimate expected consequence. Appendix A.3, Eq. (42), states that log-space regression learns a geometric mean rather than an arithmetic mean. This can give less weight to groups with rare but large consequences, which may matter when allocating precision to reduce expected damage (Eq. (11)). The paper acknowledges the distinction, but does not show whether it changes allocations or performance. This is a validation gap, not an error in the water-filling solution.

Questions for authors

1.What raw returns and normalization constants produce the Seaquest value in Table 39? Please provide the normalization code and explain whether correcting it changes Tables 24–25.

2.What exact resampling procedure produces the probability of improvement of 0.98 in Appendix B.3? Please provide the code and input arrays, and reconcile the result with Table 37.

3.Does the 33.1% saving remain supported after accounting for uncertainty in both the method scores and the no-channel reference? How sensitive is it to nearby retention thresholds?

4.How does CAP-WM compare with an agent trained throughout using adaptive memory-referred sensitivity, with the same horizons, predictor capacity, and precision budget? Does the common reference still improve return or reduce starvation?

5.Does predicting arithmetic expected consequence instead of log consequence change group rankings, intervention damage, or performance at tight budgets?

Reviewer_HJWzOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to allocate a fixed recurrent-state precision budget in world models. Its central observation is that measuring source-referred sensitivity through the current precision allocation can attenuate the estimated importance of already low-rate feature groups, creating self-reinforcing starvation. For a fixed positive intrinsic-consequence profile, the paper proves the existence of starved fixed points and a local absorbing region. To avoid this failure mode, the proposed method (CAP-WM) measures multi-step predictive Jacobian sensitivity at a common allocation-independent reference, trains a state-conditioned consequence predictor, and allocates bits via reverse water-filling. The experiments distinguish a nominal Gaussian channel from a separately trained, serialized quantization interface with explicit bit accounting. Across DMC-GB2, multi-seed ablations, action-repeat interventions, and three world-model backbones, CAP-WM shows clear gains in the low-rate regime: at 4 serialized bits per coordinate, it improves DMC-GB2 IQM over a matched one-step sensitivity baseline by 40 points (95% CI: [13, 66]), while differences at 6 bits are small and statistically unresolved.

Critical strengths
  1. The self-starvation mechanism is cleanly formulated and backed by targeted empirical diagnostics. Proposition 1 establishes the existence and local absorption of starved fixed points under a fixed intrinsic profile, and Proposition 2 shows how a common positive reference removes the assignment-dependent attenuation factor.

  2. The experimental design isolates both the multi-step horizon benefit and the physical bit budget. The one-step baseline shares the exact same reference point, predictor architecture, grouping, channel, and allocator as CAP-WM.

Critical weaknesses
  1. The headline claim of "33.1% less effective recurrent payload" relies on a fragile threshold crossing on a coarse bitrate grid. Specifically, CAP-WM's 4-bit IQM is 739 (95% CI: [715, 762]), whereas 90% of the no-channel reference score (820) is 738. Crossing the K90 threshold at 4 bits therefore depends on a point estimate that exceeds the threshold by just a single IQM point, and uncertainty in both the reference score and the 4-bit estimate is not propagated into the K90 metric. In addition, because the evaluated feature-rate grid only tests 2, 4, and 6 bits, the 33.1% figure reflects a step change between discrete grid points rather than a continuous estimate of minimum required payload. While the paired 4-bit improvement over the one-step baseline is solid, the K90 payload-reduction ratio should be framed more cautiously or accompanied by bootstrap confidence intervals (Table 1; Appendix B.2).

  2. There is a non-trivial gap between the theoretical allocation objective and the deployed consequence predictor. While the reverse water-filling allocator minimizes a local separable quadratic surrogate, the learned predictor is trained in log space (targeting a conditional geometric mean rather than the conditional arithmetic expectation required by expected quadratic damage) and uses a two-probe Hutchinson estimator that introduces bias after taking logarithms (Appendix A.3). Furthermore, in the tight nominal regime, the uniform distortion exp(-0.4) ≈ 0.67 lies outside the small-distortion regime assumed by Lemma 1. Although the empirical ablations—such as finite-perturbation correlations, probe-count checks, and eigenmode-gap diagnostics—show that the heuristic works well in practice, the drop from teacher correlation (0.82) to deployed predictor correlation (0.66) suggests noticeable approximation error, which could matter in settings with rare, high-consequence rollouts.

Questions for authors
  1. Can you report bootstrap confidence intervals for the 4-bit retention ratio relative to the no-channel reference (propagating uncertainty from both the reference and quantized runs), as well as the bootstrap probability that each tested bitrate achieves 90% retention? Clarifying in the main text that the 33.1% payload reduction is a point-estimate comparison on the {2, 4, 6}-bit grid would also make this claim more precise.

  2. In your replay data, how much do the resulting bit allocations differ when training the predictor on raw (arithmetic-mean) consequence targets versus log-space (geometric-mean) targets? Does log-space regression noticeably under-allocate bits to feature groups that experience rare but severe multi-step damage?

  3. In the offline reachability experiments, the adaptive memory-referred variant largely avoids group collapse. Have you evaluated a fully trained online world model using adaptive memory-referred sensitivity, and how does it compare to the fixed common-reference version in return and training cost? This would help clarify how much of the practical gain comes from removing source-gain attenuation versus holding the linearization reference fixed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.