acceptodds
Withdrawn from ICLR 2027

DARC: Decision-Aware Readout Calibration for Generative World Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Generative world models are increasingly used to evaluate candidate futures for planning and control, yet better numerical inference does not necessarily produce better downstream decisions. We study this mismatch by separating three notions of fidelity: numerical fidelity of the sampler, environmental fidelity to the underlying dynamics, and decision fidelity after a planner-specific readout. Across 13 DeepMind Control environments, increasing sampler refinement from 1 to 32 steps improves sample sharpness while worsening simulator-referenced PSNR and increasing planner readout error, showing that numerical refinement can amplify errors that matter to the decision. We introduce DARC, a decision-aware calibration framework for frozen generative world models. DARC uses simulator-referenced attribution to separate finite-inference error from learned-model error, then applies a readout-specific, mean-preserving dispersion transport that calibrates the statistics actually consumed by the downstream planner without retraining the world model. Across 135 latent world models and 1,620 evaluation cells, DARC removes 25.0% of planner readout error on average, with a 95% confidence interval of [16.3%, 34.5%], while harming only 1/1,620 cells. In contrast, latent-variance calibration yields −16.5% error removal and harms 330/1,620 cells. These results show that world-model inference should be evaluated and calibrated through the downstream decision functional, rather than through generative or numerical fidelity alone.

Primary area:
reinforcement learning
Keywords:
generative world models, decision-aware calibration, model-based planning, test-time inference, uncertainty calibration, embodied AI, decision fidelity

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Z2ghOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper examines the assumption in world-model planning that more sampling steps yield more accurate imagined futures. It finds that the learned models are over-dispersed and that coarse few-step solvers happen to offset this bias, so the error of the planner’s statistic grows as the solver converges. Based on this, it proposes DARC, which uses simulator reset-and-replay as the reference, applies a mean-preserving dispersion scaling to value samples, and fits one scalar per task, solver budget and statistic. On 135 latent world models, it reduces the statistic error by 25% without extra model evaluations. However, the method does not change the selected actions, with no gains in ranking, regret or closed-loop control; it also relies on a simulator with exact state reset, and the accounting behind its headline numbers is unclear. The paper’s value lies more in its diagnosis than in its method.

Critical strengths

(1) It is commonly assumed that more sampling steps make the generative model’s solution more accurate, the imagined futures more reliable, and planning better. The authors find that this can actually get worse, which is a very interesting finding, somewhat like “a person who tends to exaggerate seems more reliable when speaking vaguely; the more clearly they speak, the more obvious the exaggeration becomes.”

(2) The paper shows that the error of a coarse solver may be masking the defects of the model itself, and with a simple scaling it calibrates the planner’s statistic to the true level.

Critical weaknesses

(1) The paper uses many coined terms (“three-law formulation”, “cells”, “families”, “predicted nulls”, etc.), and key definitions are placed in the appendix, so repeated reading is needed to understand what is fitted and what each percentage refers to.

(2) The method itself offers limited novelty: it essentially adds a temperature-like scaling to the dispersion of value samples, fitting one scalar per task, solver budget and statistic. The main contribution is the diagnostic analysis rather than the method.

Minor suggestion: Find a setting where readout calibration actually changes the selected action (e.g., risk-constrained planning with a threshold on the failure probability) to demonstrate the value of the method; otherwise, the paper would be better repositioned as an analysis paper.

Questions for authors

pls see weaknesses.

Reviewer_6VTpOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the relationship between world model errors, sampling accuracy, and downstream decision making. It proposes DARC, a post-hoc calibration method that uses a simple scalar rescaling to improve the accuracy of planner readouts without modifying the world model. Experiments show that DARC reduces readout error by 25% on latent world models, but does not lead to clear improvements in action selection or closed-loop control.

Critical strengths

S1. The paper studies an interesting and important problem. Understanding how world model errors affect downstream decision making is challenging, and I think this is a meaningful research direction.

S2. The authors have clearly put considerable effort into this work, especially the detailed analysis and extensive experiments.

S3. The paper is generally well organized and clearly presents its findings. I also appreciate that the authors honestly report the negative results and limitations of their method.

Critical weaknesses

W1. The paper provides a detailed analysis of world model errors and their relationship with downstream decisions. However, the proposed method is essentially a simple scalar rescaling. I am not fully convinced by its methodological novelty, or how much the analysis actually contributes to the method design.

W2. Although DARC improves readout accuracy, these improvements do not lead to clear gains in action selection or closed-loop control. I appreciate that the authors honestly report these negative results, but this also makes me question the practical value of the proposed method.

W3. The results are less consistent on pixel-space world models. For example, DARC increases readout error in 510 out of 5,184 cells on locomotion and 477 out of 3,456 cells on Jaco. Its advantage over latent-space calibration also depends on the aggregation metric used. These results make me question how robust the reported improvements are across different world models and evaluation settings.

Questions for authors

Q1. The proposed method uses a relatively simple scalar rescaling. Could you clarify how the error analysis contributes to the design of DARC, and what makes it different from a standard post-hoc calibration method?

Q2. DARC significantly improves readout accuracy but shows almost no improvement in action selection or closed-loop control. Could you discuss why these improvements do not translate into better decisions, and under what conditions DARC could provide practical benefits?

Q3. The pixel-space experiments show a noticeable number of harmed cells, and the comparison with latent-space calibration is sensitive to the aggregation metric. Could you explain what causes these differences and how they affect the overall conclusions about DARC's effectiveness?

Reviewer_JVGPOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

DARC is a calibration method for planning with generative world models. DARC learns a scalar that rescales the sampled values around their mean. This scalar is learned against a simulator reference for each task. DARC reduces readout error by approximately 25% on average. However, despite improved readout estimation, the experiments show no demonstrated benefits for action selection or closed-loop control.

Critical strengths
  • The diagnosis on why more solver steps can degrade readout accuracy is interesting.
  • Honest evaluation including null results for action and control.
Critical weaknesses
  • Major: My main concern is practical significance. DARC does not seem to improve action choice or control in general (Section 4.6). And in general, DARC seems to have no effect on action selection. The training for DARC requires a simulator for every task. However, using the simulator directly seems preferable and is also the more straightforward approach.
  • The method novelty seems limited, being quite closely related to standard variance/temperatur calibration. Ablation-wise, simple alternatives are not rlly compared, e.g. fitting the scalar to the value-sample variance.
  • The presentation of the paper makes it very hard to read, it's dense and jargon-heavy. Also, much of the theory is standard, e.g. Propositions 2,3, 5, but not rlly presented in that way.
Questions for authors
  • Could the authors clarify what the intended benefit/use case of DARC is? From what I understood, there is no control benefit, and additionally, a simulator is required?
  • Table 16 shows that DARC results in no meaningful change in ranking agreement compared to the base, even though the readout error is lower. How often does DARC change the selected action compared to no DARC?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.