- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies predictive sufficiency in coarse-grained world models by examining the value of additional temporal context and micro-level information separately. Track A compares prediction risks across history lengths, while Track B tests whether micro-level features improve prediction beyond the same macro history through residual augmentation. Two theoretical constructions illustrate that macro observations alone do not identify strong lumpability, and that a finite-dimensional recursive state does not imply a sufficient finite observation window. The main experiments use a controlled linear voxel system to compare the two empirical prediction gaps under different aggregation regimes.
Critical strengths
1.The paper addresses a useful question. Low prediction error alone does not establish that a representation retains sufficient information. Examining the gains from additional history and micro-level features separately could help determine how a model should be improved.
2.The theoretical examples clearly explain the limits of inference from macro observations. In particular, the construction of identical macro laws with different strong-lumpability properties shows why macro prediction performance cannot by itself establish properties of the underlying micro dynamics.
Critical weaknesses
W1. The dependence on predictor and input choices is not evaluated, leaving the practical value of the diagnostics unclear. Both tracks depend on whether the chosen predictor can use the additional information, but the experiments mainly use regularized linear models. The paper does not compare predictors with different modeling capabilities or systematically examine the effect of micro-level feature selection. It is therefore difficult to tell whether the absence of a gain means that the existing information is sufficient or that the predictor cannot use the additional information effectively. The authors explicitly limit their conclusions to the chosen models and features, but the practical impact of this restriction still needs to be evaluated. Track A also fixes the reference history length at 8 without showing whether the conclusions hold with longer reference windows. There is no comparison with simple alternatives, such as selecting history length on a validation set or directly comparing models with and without micro-level features. It remains unclear whether the two-track audit provides reliable guidance or adds value beyond standard model selection. W2. The experiments are insufficient, with limited validation in the intended applications and little visual analysis. The main synthetic experiments are limited to linear dynamics on a 4 × 4 × 4 voxel grid. Both tracks yield non-positive estimates in all four joint settings in Section 6.4 and Table 3, and the other synthetic Track B experiments also provide no positive examples. These results do not establish that the method can detect additional history or micro-level information when it is genuinely predictive. Nor do they show that the diagnostic recommendations lead to effective model improvements. The experiments therefore do not substantiate the applications to 3D scenes, BEV representations, or voxel world models emphasized in the paper. The results are also presented mainly as numerical tables, with few visual explanations connecting the diagnostic values to actual prediction behavior. Missing analyses include error curves with uncertainty intervals across history lengths, visualizations of how the diagnostics change with spatial aggregation, and side-by-side views of micro states, coarse representations, and predictions. There are also no concrete success or failure cases showing what changes after extending history or retaining more spatial detail. This makes it difficult to interpret the numerical differences in a practical setting. At a minimum, the paper needs validation on controls with known gains and known null effects, together with an application showing that the diagnostics correspond to actual improvements.
Questions for authors
- How do the diagnostic results change when linear predictors are replaced with nonlinear predictors, or when different micro-level features are used? Can you distinguish a lack of additional predictive information from a predictor’s inability to use it?
- Are the Track A conclusions stable with reference histories longer than 8? What additional information or practical benefit does the audit provide over selecting history length and feature inclusion directly on a validation set?
- Can you provide controlled cases with known positive gains and known null effects to establish the detection ability of each track? How does performance change with sample size?