- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper presents Visual Bayesian Regression World Model (ViBR-WM), which combines visual features, physical histories and known covariates for interpretable regression. The modular architecture supports trend, seasonal, and cycle dynamics. Visual compression leverages PCA to select features, while predictive uncertainty is achieved with Bayesian regression. In addition, Bayesian variable selection is introduced.
Experiments are conducted on several tasks such as PointMaze, Wall, PhenoCam. Comparison methods include TemporalStraightening, ConvLSTM, PredRNN, and SimVP, most of which are before 2022. The proposed method beats them w.r.t. Mean NRMSE.
Critical strengths
This paper presents ViBR-WM as a modular and Bayesian framework for world modeling.
- It combines visual features, physical histories and known covariates through interpretable regression.
- Modular design with trend, seasonal and cycle dynamics are input to the model.
- Visual compression, Bayesian variable selection, and Bayesian inference are designed.
- Experiments show the advantanges over simple baselines listed in this paper (all before 2022).
Critical weaknesses
This paper straightforwardly combines several simple modules such as PCA, Bayesian inference, Trend/Seasonality/Cycle in a joint framework for world model prediction.
- These modules are mature ideas from existing literatures, limiting the novelty and new insights of this paper.
- No clear ablation study is conducted to verify and analyse each modules' effect.
- Baseline comparison model (TS in Figure 1) barely use only original features, which is a over weakened baseline.
Experiments are less convincing and less comprehensive, w.r.t. baseline method, ablation study, visualization, etc.
- All baseline methods are quite simple, ConvLSTM (Shi et al., 2015), PredRNN (Wang et al., 2017), and SimVP (Gao et al., 2022). The superior performance is thus less convincing and less evident.
- The tasks and data used in this paper are mainly PointMaze, Wall, PhenoCam, which are less real-world.
- No ablation study to evaluate the designed modules.
- No visualization of the dataset, or results.
Questions for authors
Please see above weaknesses.
3 comments
Thank you for the rebuttal
Thank you for the clarification and additional experiments. I acknowledge that the revision strengthens the empirical evaluation. However, these additions do not change my overall assessment of the paper. My main concern remains the limited methodological novelty, as the framework is largely built from established components and the new results do not substantially alter this view.
Therefore, I remain unconvinced and still lean towards rejection.
Clarification on the Methodological Novelty of ViBR-WM
Thank you for the follow-up and for acknowledging that the revision strengthens the empirical evaluation. We respect the reviewer's assessment, but would like to clarify one point regarding methodological novelty.
Our contribution is not the individual use of PCA, regression, or Bayesian inference, each of which is of course well established. Rather, the methodological contribution is to formulate visual world modeling itself within a Bayesian regression framework. In ViBR-WM, learned visual representations are incorporated together with physical histories and known actions/covariates into a probabilistic state-transition model, replacing the conventional neural transition predictor with Bayesian regression over joint visual–physical dynamics. This formulation supports both recursive world-model rollouts and direct physical-target forecasts.
Importantly, the Bayesian formulation is not simply an implementation choice: it provides posterior inference over dynamics, Bayesian variable selection, interpretable predictor effects, and posterior predictive distributions that propagate parameter and future-disturbance uncertainty through forecasting. We therefore view the contribution as introducing and validating a different paradigm for visual world modeling—one that connects learned visual representations with interpretable Bayesian dynamics and explicit uncertainty quantification—rather than claiming novelty for the individual statistical components themselves.
We have revised the paper to make this distinction and positioning clearer, and we appreciate the reviewer highlighting the need to articulate it more explicitly.
Thank you for your suggestions on the comparator methods and component tests.
1. Methodological contribution and ablation results
ViBR-WM's contribution is an interpretable Bayesian architecture for joint visual–physical forecasting, supporting recursive state prediction and direct physical-target prediction with uncertainty. The trend, seasonal and cycle modules give users the freedom to enhance interpretability by separating temporal patterns from covariate effects.
To test the contribution of Bayesian estimation, we compare the Bayesian regression module with ridge regression. With identical predictors, Bayesian regression has lower error than ridge on PointMaze and SKIPP'D; 95% intervals for Bayesian-minus-ridge error differences lie below zero. Removing image inputs improves both regressions on PointMaze and SKIPP'D, with no clear change on Wall or PhenoCam. Numerical histories may already capture much of the predictive information; extra visual predictors may increase overfitting.
Beyond the regression comparison, component tests show that Bayesian inference improves interval quality by accounting for parameter uncertainty, while predictor selection improves accuracy. Sampling rather than fixing parameters reduces the interval score, which penalizes interval width and observations outside it, by 7.79% on Wall and 9.91% overall across three PhenoCam sites in 2025. Selecting predictors lowers SKIPP'D error by 1.78%.
On PointMaze and Wall, we test fitting principal component analysis (PCA) to image-feature values instead of changes between frames, retaining the same number of coordinates. We also test features directly from the fixed DINOv2 image encoder instead of features learned by Temporal Straightening (TS). Neither change significantly improves overall error.
2. TS implementation and comparator selection
TS (ICML 2026) uses its trained feature mapping and predictor, not raw DINOv2 features alone. TS and ViBR-WM share this learned representation within each PointMaze and Wall repetition. In this revision, we clarify in the text that the image encoder is fixed, while the feature mapping and predictor are trained.
ViBR-WM also outperforms the added attention-based SwinLSTM-B (ICCV 2023) on all four tasks. We use the authors' implementation. ViBR-WM itself uses compact regression, motivating comparisons with established, relatively simple architectures. Across ten repetitions, ViBR-WM's mean normalized root mean squared error (NRMSE) is lower than SwinLSTM-B by 55.94% on PointMaze, 55.88% on Wall, 50.82% on PhenoCam and 59.35% on SKIPP'D. All four 95% intervals for the error differences support these improvements.
3. Real-world relevance of the tasks
PhenoCam and SKIPP'D use data collected in the real world: PhenoCam records outdoor vegetation with cameras, while SKIPP'D pairs sky images with measured solar power output. We further test performance across years and locations by extending the PhenoCam evaluation to all 52 weeks of 2025 at the two original camera sites, Mead1 and Mead3, and an additional camera site, arsmorris1.
Averaged across the three sites, ViBR-WM's prediction error for 2025 is lower than that of all five neural comparators, including a 49.05% reduction relative to SwinLSTM-B.
4. Visualization
The submitted paper includes visualizations of both the data and forecasting results. Figure 2 shows Wall images and trajectories, and Figure 3 compares forecasting errors across all four tasks. Figure 4 shows observed PhenoCam greenness alongside predictions from ViBR-WM and the four original comparators.