- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies that visual and geometric quality metrics for generated worlds do not necessarily reflect their reliability for downstream task planning. The authors identify a Verification Paradox, in which assimilating additional miscalibrated evidence can increase collision risk and worsen downstream performance. To address this, they propose Calibration-Aware Cross-World Verification (CWV), a training-free decision layer that forms cross-world route beliefs, prioritises task-critical verification, and combines inverse-variance fusion with lower-confidence-bound commitment. An innovation gate down-weights observations that are inconsistent with the cross-world belief. The method is then evaluated across multiple environments.
Critical strengths
1 The paper is well motivated and addresses an important practical problem: the gap between upstream generative world model performance and downstream task-specific decision-making.
2 The experiment is solid, with many statistical results supporting the reliability of the results.
Critical weaknesses
-
The author discusses one of the key reasons for the Verification Paradox: shared structural hallucinations in cross-world agreement, but does not address the problem. Adding sigma_0 (Eq. 9) increases uncertainty, while small sigma_r (cross-world agreement)can still deprioritize confident wrong routes (Eq. 13). The gate (Eq. 15) may also down-weight corrective observations, since it does not determine whether the prior or the observation is wrong. Table 3 shows Base CWV worsening as B increases in Manhattan Midtown, Phoenix Suburb, and Seattle Downtown, raising doubts about whether the problem has been addressed.
-
The writing and figure presentation make the paper difficult to read. Simple transformations introduce too many symbols, and sigma_0 and sigma_o are easily confused. Fig. 3 is too small to read without zooming; some legend entries are difficult to distinguish, and some curves lack explanations. Also, I could not find in-text references to Figs. 2 and 5 or understand where they fit into the discussion. These issues made it difficult to understand the method and results.
Questions for authors
- Does CWV really resolve the Verification Paradox? If cross-world agreement is from shared structural hallucinations, a conflicting observation may be correct. But the gate always down-weights that observation without identifying which source is wrong. Will this strengthen the error agreement? Table 3 also shows Base CWV’s collision rate increasing with the verification budget in several regions. Gated CWV also does not solve this problem(Eq. 15).
2 In the top-left subplot of Fig. 1 left, what do the colored curves, the black curve, and the red dashed line mean? Also, the left figure’s legend does not distinguish solid and dashed lines. The colors of Base CWV and InfoGain Robust in Fig. 2 are also difficult to distinguish. Also, please address Fig. 3’s size and the missing in-text references to Fig. 2 and Fig. 5. 3 What exactly is the route cost function mentioned around line 185? Also, why is TC-EMA included? I could not find an explanation for why this baseline is introduced near Tab.1/ in the main text.
4 Is the second paragraph of the Discussion discussing that upstream generation and downstream decisions should be more closely connected? Appendix H feels disconnected from the paper’s topic. How does it relate to the specific components of CWV?