acceptodds
Under review as a conference paper at ICLR 2027

ViBR-WM: Visual Bayesian Regression for World Modeling

Abstract

Modeling temporal dependence and uncertainty is central to forecasting with world models. The Visual Bayesian Regression World Model combines visual features, physical histories and known covariates through interpretable regression, within a modular architecture supporting trend, seasonal and cycle dynamics. Visual compression reduces representation dimension, while Bayesian variable selection reduces active regression dimension. Posterior prediction combines forecasts across predictor subsets using their posterior probabilities as weights and accounts for parameter uncertainty and future disturbances. The model forecasts joint visual–physical states recursively and physical targets directly. Across four forecasting tasks spanning object motion, vegetation greenness and solar power, ViBR-WM achieves lower mean overall physical-target error than Temporal Straightening, ConvLSTM, PredRNN and SimVP on every task. Repeated fitting and resampling support these overall gains.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
world models, Bayesian regression, probabilistic forecasting, uncertainty quantification, variable selection, visual forecasting

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ZoL1OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents Visual Bayesian Regression World Model (ViBR-WM), which combines visual features, physical histories and known covariates for interpretable regression. The modular architecture supports trend, seasonal, and cycle dynamics. Visual compression leverages PCA to select features, while predictive uncertainty is achieved with Bayesian regression. In addition, Bayesian variable selection is introduced.

Experiments are conducted on several tasks such as PointMaze, Wall, PhenoCam. Comparison methods include TemporalStraightening, ConvLSTM, PredRNN, and SimVP, most of which are before 2022. The proposed method beats them w.r.t. Mean NRMSE.

Critical strengths

This paper presents ViBR-WM as a modular and Bayesian framework for world modeling.

  • It combines visual features, physical histories and known covariates through interpretable regression.
  • Modular design with trend, seasonal and cycle dynamics are input to the model.
  • Visual compression, Bayesian variable selection, and Bayesian inference are designed.
  • Experiments show the advantanges over simple baselines listed in this paper (all before 2022).
Critical weaknesses

This paper straightforwardly combines several simple modules such as PCA, Bayesian inference, Trend/Seasonality/Cycle in a joint framework for world model prediction.

  • These modules are mature ideas from existing literatures, limiting the novelty and new insights of this paper.
  • No clear ablation study is conducted to verify and analyse each modules' effect.
  • Baseline comparison model (TS in Figure 1) barely use only original features, which is a over weakened baseline.

Experiments are less convincing and less comprehensive, w.r.t. baseline method, ablation study, visualization, etc.

  • All baseline methods are quite simple, ConvLSTM (Shi et al., 2015), PredRNN (Wang et al., 2017), and SimVP (Gao et al., 2022). The superior performance is thus less convincing and less evident.
  • The tasks and data used in this paper are mainly PointMaze, Wall, PhenoCam, which are less real-world.
  • No ablation study to evaluate the designed modules.
  • No visualization of the dataset, or results.
Questions for authors

Please see above weaknesses.

3 comments

Thank you for your suggestions on the comparator methods and component tests.

1. Methodological contribution and ablation results

ViBR-WM's contribution is an interpretable Bayesian architecture for joint visual–physical forecasting, supporting recursive state prediction and direct physical-target prediction with uncertainty. The trend, seasonal and cycle modules give users the freedom to enhance interpretability by separating temporal patterns from covariate effects.

To test the contribution of Bayesian estimation, we compare the Bayesian regression module with ridge regression. With identical predictors, Bayesian regression has lower error than ridge on PointMaze and SKIPP'D; 95% intervals for Bayesian-minus-ridge error differences lie below zero. Removing image inputs improves both regressions on PointMaze and SKIPP'D, with no clear change on Wall or PhenoCam. Numerical histories may already capture much of the predictive information; extra visual predictors may increase overfitting.

Beyond the regression comparison, component tests show that Bayesian inference improves interval quality by accounting for parameter uncertainty, while predictor selection improves accuracy. Sampling rather than fixing parameters reduces the interval score, which penalizes interval width and observations outside it, by 7.79% on Wall and 9.91% overall across three PhenoCam sites in 2025. Selecting predictors lowers SKIPP'D error by 1.78%.

On PointMaze and Wall, we test fitting principal component analysis (PCA) to image-feature values instead of changes between frames, retaining the same number of coordinates. We also test features directly from the fixed DINOv2 image encoder instead of features learned by Temporal Straightening (TS). Neither change significantly improves overall error.

2. TS implementation and comparator selection

TS (ICML 2026) uses its trained feature mapping and predictor, not raw DINOv2 features alone. TS and ViBR-WM share this learned representation within each PointMaze and Wall repetition. In this revision, we clarify in the text that the image encoder is fixed, while the feature mapping and predictor are trained.

ViBR-WM also outperforms the added attention-based SwinLSTM-B (ICCV 2023) on all four tasks. We use the authors' implementation. ViBR-WM itself uses compact regression, motivating comparisons with established, relatively simple architectures. Across ten repetitions, ViBR-WM's mean normalized root mean squared error (NRMSE) is lower than SwinLSTM-B by 55.94% on PointMaze, 55.88% on Wall, 50.82% on PhenoCam and 59.35% on SKIPP'D. All four 95% intervals for the error differences support these improvements.

3. Real-world relevance of the tasks

PhenoCam and SKIPP'D use data collected in the real world: PhenoCam records outdoor vegetation with cameras, while SKIPP'D pairs sky images with measured solar power output. We further test performance across years and locations by extending the PhenoCam evaluation to all 52 weeks of 2025 at the two original camera sites, Mead1 and Mead3, and an additional camera site, arsmorris1.

Averaged across the three sites, ViBR-WM's prediction error for 2025 is lower than that of all five neural comparators, including a 49.05% reduction relative to SwinLSTM-B.

4. Visualization

The submitted paper includes visualizations of both the data and forecasting results. Figure 2 shows Wall images and trajectories, and Figure 3 compares forecasting errors across all four tasks. Figure 4 shows observed PhenoCam greenness alongside predictions from ViBR-WM and the four original comparators.

Reviewer_ZoL1OpenReview ↗

Thank you for the rebuttal

Thank you for the clarification and additional experiments. I acknowledge that the revision strengthens the empirical evaluation. However, these additions do not change my overall assessment of the paper. My main concern remains the limited methodological novelty, as the framework is largely built from established components and the new results do not substantially alter this view.

Therefore, I remain unconvinced and still lean towards rejection.

AuthorsOpenReview ↗

Clarification on the Methodological Novelty of ViBR-WM

Thank you for the follow-up and for acknowledging that the revision strengthens the empirical evaluation. We respect the reviewer's assessment, but would like to clarify one point regarding methodological novelty.

Our contribution is not the individual use of PCA, regression, or Bayesian inference, each of which is of course well established. Rather, the methodological contribution is to formulate visual world modeling itself within a Bayesian regression framework. In ViBR-WM, learned visual representations are incorporated together with physical histories and known actions/covariates into a probabilistic state-transition model, replacing the conventional neural transition predictor with Bayesian regression over joint visual–physical dynamics. This formulation supports both recursive world-model rollouts and direct physical-target forecasts.

Importantly, the Bayesian formulation is not simply an implementation choice: it provides posterior inference over dynamics, Bayesian variable selection, interpretable predictor effects, and posterior predictive distributions that propagate parameter and future-disturbance uncertainty through forecasting. We therefore view the contribution as introducing and validating a different paradigm for visual world modeling—one that connects learned visual representations with interpretable Bayesian dynamics and explicit uncertainty quantification—rather than claiming novelty for the individual statistical components themselves.

We have revised the paper to make this distinction and positioning clearer, and we appreciate the reviewer highlighting the need to articulate it more explicitly.

Reviewer_uy7aOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents ViBR-WM, a visual forecasting framework that combines compressed pretrained visual features with Bayesian regression. It incorporates observed physical histories and action or calendar information to predict future states or scalar quantities, using recursive prediction for simulated dynamics and horizon-specific regression for real-world forecasting. Posterior predictive distributions provide uncertainty estimates. Experiments on two simulated environments, vegetation greenness, and photovoltaic power demonstrate competitive forecasting performance, while examining the sensitivity to model configurations and baseline adaptations.

Critical strengths

1.Combining pretrained visual representations with Bayesian regression offers a compact approach to probabilistic forecasting. The framework explicitly accounts for parameter uncertainty and future disturbances, supporting both recursive state prediction and direct forecasting.

2.The supplementary analyses provide useful transparency, including inference diagnostics, predictive coverage, and adapted baseline comparisons. These results help readers assess which conclusions are supported and where limitations remain.

Critical weaknesses

1.The methodological contribution and component benefits are insufficiently established. The approach largely combines existing visual representations with standard Bayesian regression. Table S1 shows that all four selected configurations use regression only, and the two simulation tasks disable predictor selection. Matched comparisons against regularized non-Bayesian regression and the same model without visual features are missing, leaving the benefits of Bayesian inference and visual information unclear.

2.The reported superiority is sensitive to the comparison protocol. The main comparisons differ in available inputs, training objectives, and conversion from predictions to physical quantities. Appendix G shows that adapted baselines can outperform ViBR-WM: PredRNN achieves 0.5053 versus 0.5602 on PointMaze, and TS achieves 0.3275 versus 0.4367 on PhenoCam. Although these adaptations change several factors jointly, they substantially weaken the attribution of the main-table improvements to the proposed predictor.

3.The evidence for reliable uncertainty and broader applicability remains limited. Nominal 95% prediction intervals achieve only 78.87% coverage on PhenoCam and 89.03% on Wall. Evaluation is restricted to two simulation environments and two real-world forecasting datasets, with two PhenoCam sites and one held-out test year per real-world dataset. This provides limited evidence that the uncertainty estimates remain reliable across different environments and temporal conditions.

Questions for authors

1.Can you provide matched comparisons using the same features and regression design: without visual inputs, and with regularized non-Bayesian regression? Which component accounts for the observed improvements?

2.How should the main comparative conclusions be interpreted given the ranking reversals in Table S18? Can the adapted baseline comparisons also be extended to Wall and SKIPP’D?

3.What explains the substantial undercoverage on PhenoCam and Wall? How could calibration be improved while retaining informative, reasonably narrow prediction intervals?

5 comments

We appreciate your detailed feedback on our method and its evaluation.

1. Methodological contribution and ablation results

Section 3.2 states that the structural modules serve “to enhance interpretability by explicitly separating the contributions of trend, seasonality and cycles from covariate effects in the forecast.” It also explains that “extra parameters can overfit and increase prediction error.” With regression inputs and priors fixed, Table S9 (Section C.3.1) reports PhenoCam validation normalized root mean squared error (NRMSE) of 0.506103 with the trend's level component and seasonal states, 0.456370 with the trend's level alone and 0.385197 without them.

With identical predictors, Bayesian regression has lower error than ridge on PointMaze and SKIPP'D; 95% intervals for Bayesian-minus-ridge error differences lie below zero. Wall's interval includes zero; PhenoCam's errors are nearly identical. Removing image inputs improves both regressions on PointMaze and SKIPP'D, with no clear change on Wall or PhenoCam. Numerical histories may already capture much of the predictive information; extra visual predictors may increase overfitting.

Beyond the regression comparison, component tests show that Bayesian inference improves interval quality by accounting for parameter uncertainty, while predictor selection improves accuracy. Sampling rather than fixing parameters reduces the interval score, which penalizes interval width and observations outside it, by 7.79% on Wall and 9.91% overall across three PhenoCam sites in 2025. Selecting predictors lowers SKIPP'D error by 1.78%.

2. Comparison protocol and S18 adaptations

Table S26 (S18 in the submitted manuscript) is not a fair cross-method comparison: each method receives different modifications tailored to the particular task. The section examines how adapting a method changes its performance relative to its own unchanged version. The original table placed adapted neural systems alongside unchanged ViBR-WM, obscuring this purpose. The revised table places each method's unchanged and adaptation versions side by side, states exactly what changed, and emphasizes its own before-and-after result.

For PointMaze, we modified the input and output interfaces of ConvLSTM, PredRNN and SimVP to add capabilities absent from their original image-to-image implementations: accepting visual features together with actions and physical measurements, and directly predicting future visual features and physical states. For TS, forecasts stay unchanged; the ridge penalty for converting features to position and velocity is selected on separate validation trajectories instead of within training data. Each PhenoCam model trains its own greenness-history encoder and head; video models use greenness loss, which TS adds to its original loss. The existing adaptations cannot be directly extended to Wall or SKIPP'D: they are specific to PointMaze and PhenoCam, not general improvements to the underlying algorithms. Extending Table S26 would require designing a separate set of task-specific adaptations for each of those two tasks.

We have now developed task-specific adaptation versions of ViBR-WM for PointMaze and PhenoCam. On PointMaze, we tailor ViBR-WM's regression inputs to location-dependent motion and wall geometry. For PhenoCam, we add a compact encoder of greenness history, image summaries and dates, and use its output in ViBR-WM's Bayesian regression. With these adaptations, ViBR-WM's mean NRMSE changes from 0.56018 to 0.49636 on PointMaze and from 0.43670 to 0.32225 on PhenoCam, respectively 11.39% and 26.21% lower. All original S18 entries remain.

The main tables compare complete forecasting systems on the same task data and physical targets. Their architectures differ in how they use the data and construct model inputs.

3. Interval calibration and evaluation across sites and years

For each PhenoCam site and horizon, the model estimates one disturbance variance across seasons. This shared scale can understate summer uncertainty despite low point error. In this revision, we introduce a calibration step to address this undercoverage. It estimates a width correction from past prediction errors: the adjusted 95th percentile of how many original half-widths are needed to reach each past observation from its interval midpoint. Multipliers use 2024 observations preceding forecast origins and stay fixed for 2025; interval midpoints and point forecasts remain unchanged.

To assess reliability in a new year and at another camera site, we evaluate all 52 weeks of 2025 at four forecast distances for the two original PhenoCam camera sites, Mead1 and Mead3, and the additional camera site arsmorris1. To test prediction at a new site, each method applies its model trained on Mead1 data directly to arsmorris1, without retraining there. Coverage at these sites is 100%, 89.33% and 95.19%, respectively.

On the three-site 2025 evaluation, calibration raises 95% coverage from 77.40% to 94.84%. The interval score penalizes width and missed targets; lower is better. It falls from 5.0344 to 3.1662. With the same candidate predictors and calibration rule, residual-based ridge intervals score 3.1465, with no statistically significant difference from Bayesian regression.

Wall diagnostics at five frames ahead suggest that the shared disturbance covariance does not fully capture the greater error variability associated with collisions. Trajectories with and without collisions have nearly identical mean interval widths, but 95% coverage is 89.08% and 95.83%, respectively.

On the separate evaluation trajectories, calibration raises Wall coverage from 87.13% to 93.85% and PointMaze from 94.83% to 96.11%. Calibration uses trajectories separate from those used for evaluation. On Wall, Bayesian interval width rises from 1.1590 to 1.4563 while interval score falls from 2.2090 to 1.9558; calibrated ridge scores 1.9685.

PointMaze interval scores under the same calibration rule are 2.7049 for Bayesian regression and 2.8951 for ridge. The continuous ranked probability score (CRPS) evaluates the full predictive distribution against observations. On SKIPP'D, without calibration, Bayesian regression has 13.55% lower CRPS and 12.00% lower interval score than ridge. Both differences are statistically significant; lower is better.

AuthorsOpenReview ↗

Follow-up before the discussion concludes

Dear Reviewer uy7a,

Thank you again for your careful review and constructive suggestions. As the discussion period draws to a close, we would like to briefly summarize how we addressed your questions:

  • What each component contributes: We added matched Bayesian-vs.-ridge comparisons using identical predictors and evaluated both the no-vision and visual-only settings. With visual information alone, ViBR-WM again achieves the lowest mean NRMSE on all four tasks. Bayesian regression improves accuracy on PointMaze and SKIPP'D, while additional component tests show that Bayesian parameter uncertainty improves interval quality and Bayesian variable selection improves predictive accuracy.
  • Adapted-baseline comparisons: We clarified that the original adapted baselines were task-specific modifications rather than matched algorithmic comparisons, reorganized the table accordingly, and added task-specific adaptations of ViBR-WM. These improve ViBR-WM NRMSE by 11.39% on PointMaze and 26.21% on PhenoCam.
  • Uncertainty calibration and broader evaluation: We introduced an out-of-sample calibration procedure and expanded PhenoCam evaluation across all 52 weeks of 2025 and an additional camera site. Calibration increases three-site 95% coverage from 77.40% to 94.84%, while separate Wall and PointMaze evaluations also show substantially improved coverage.
  • Broader probabilistic evaluation: We added interval-score and CRPS comparisons against ridge; on SKIPP'D, Bayesian regression achieves 13.55% lower CRPS and 12.00% lower interval score, with both differences statistically significant.

If these revisions resolve your questions, we would greatly appreciate it if you could consider reflecting this in your assessment.

Best regards,

Authors

Reviewer_Vo98OpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes ViBR-WM, which combines compressed image features with past physical measurements, actions, or calendar information in a Bayesian regression model to predict future states and their uncertainty. The authors evaluate it on four tasks covering object motion, vegetation changes, and solar power, aiming to balance prediction accuracy, interpretability, and uncertainty estimation.

Critical strengths
  1. The idea is straightforward and practical. It uses existing visual representations and a statistical model to predict physical quantities, making it easier to inspect the role of the inputs and obtain prediction intervals.

  2. The experiments are carefully reported. Beyond average errors, the authors include repeated fitting, resampling that accounts for temporal dependence, and posterior sampling diagnostics to check the reliability of the results.

  3. The paper covers both simulated and real-world data, showing a workable way to combine visual representations with Bayesian prediction.

Critical weaknesses
  1. The gains may come from the comparison setup rather than Bayesian modeling. In Appendix S18, some adapted baselines beat ViBR-WM on PointMaze and PhenoCam.

  2. It’s unclear what each component adds over simpler regression. The highlighted trend, seasonal, and periodic components are unused in all four selected configurations.

  3. The uncertainty estimates are not fully convincing: on PhenoCam, the 95% prediction intervals cover only about 79% of the targets, and comparisons with probabilistic baselines are limited.

Questions for authors
  1. Why do some baselines outperform ViBR-WM in Appendix S18? How much of the advantage remains when the inputs and prediction objectives are matched?

  2. How does Bayesian regression compare with ridge regression using the same features and numerical inputs? What happens if visual features are removed? These checks would help show where the gains come from.

  3. Why is the prediction interval coverage lower than expected? Could the authors compare with a calibrated probabilistic baseline to show what their uncertainty estimates add?

I currently lean toward weak reject. The approach has practical value, but I’m not convinced that the gains come from the method itself. Addressing that question i will reconsider my decision.

4 comments

Thank you for your thoughtful questions about our comparisons and uncertainty estimates.

1. Comparison protocol and Table S18

Table S26 (S18 in the submitted manuscript) is not a fair cross-method comparison: each method receives different modifications tailored to the particular task. The section examines how adapting a method changes its performance relative to its own unchanged version. The original table placed adapted neural systems alongside unchanged ViBR-WM, obscuring this purpose. The revised table places each method's unchanged and adaptation versions side by side, states exactly what changed, and emphasizes its own before-and-after result.

For PointMaze, we modified the input and output interfaces of ConvLSTM, PredRNN and SimVP to add capabilities absent from their original image-to-image implementations: accepting visual features together with actions and physical measurements, and directly predicting future visual features and physical states. For TS, forecasts stay unchanged; the ridge penalty for converting features to position and velocity is selected on separate validation trajectories instead of within training data. Each PhenoCam model trains its own greenness-history encoder and head; video models use greenness loss, which TS adds to its original loss. These are different application-specific systems, not a common modification that establishes a ranking of the underlying algorithms across tasks.

We have now developed task-specific adaptation versions of ViBR-WM for PointMaze and PhenoCam. On PointMaze, we tailor ViBR-WM's regression inputs to location-dependent motion and wall geometry. For PhenoCam, we add a compact encoder of greenness history, image summaries and dates, and use its output in ViBR-WM's Bayesian regression. With these adaptations, ViBR-WM's mean NRMSE changes from 0.56018 to 0.49636 on PointMaze and from 0.43670 to 0.32225 on PhenoCam, respectively 11.39% and 26.21% lower. All original S18 entries remain.

The main tables compare complete forecasting systems on the same task data and physical targets. Their architectures differ in how they use the data and construct model inputs.

2. Interpretability and predictive contributions

The trend, seasonal and cycle modules give users the freedom to enhance interpretability by separating temporal patterns from covariate effects; they are not a promise of higher forecasting accuracy. Section 3.2 states: “The purpose of including these modules is to enhance interpretability by explicitly separating the contributions of trend, seasonality and cycles from covariate effects in the forecast.” It also states: “Adding modules does not guarantee better forecasts: extra parameters can overfit and increase prediction error.”

The reported four-task configurations use the regression specialization. Table S9 in Section C.3.1 of the revised manuscript reports the PhenoCam validation comparison with the regression design and priors held fixed: NRMSE is 0.506103 with the trend's level and seasonal states, 0.456370 with level alone and 0.385197 without those states. Thus the interpretation offered by structural decomposition and the predictive accuracy of a selected configuration are distinct questions.

With identical predictors, Bayesian regression has lower error than ridge on PointMaze and SKIPP'D (95% intervals below zero); Wall's interval includes zero and PhenoCam's errors are nearly identical. Removing vision improves both estimators on PointMaze and SKIPP'D, with no clear effect on Wall or PhenoCam. Numerical histories may already capture much of the predictive information, while extra visual predictors may increase overfitting.

3. Coverage, calibration and probabilistic comparison

For each PhenoCam site and horizon, the model estimates one disturbance variance across seasons. This shared scale can understate summer uncertainty despite low point error. In this revision, we introduce a calibration step to address this undercoverage. It estimates a width correction from past prediction errors: the adjusted 95th percentile of how many original half-widths are needed to reach each past observation from its interval midpoint. Multipliers use 2024 observations preceding forecast origins and stay fixed for 2025; interval midpoints and point forecasts remain unchanged.

On the three-site 2025 evaluation, calibration raises 95% coverage from 77.40% to 94.84%. The interval score penalizes width and missed targets; lower is better. It falls from 5.0344 to 3.1662. With the same candidate predictors and calibration rule, residual-based ridge intervals score 3.1465, with no statistically significant difference from Bayesian regression.

Wall diagnostics at five frames ahead suggest that the shared disturbance covariance does not fully capture the greater error variability associated with collisions. Trajectories with and without collisions have nearly identical mean interval widths, but 95% coverage is 89.08% and 95.83%, respectively.

On the separate evaluation trajectories, calibration raises Wall coverage from 87.13% to 93.85% and PointMaze from 94.83% to 96.11%. Calibration uses trajectories separate from those used for evaluation. On Wall, Bayesian interval width rises from 1.1590 to 1.4563 while interval score falls from 2.2090 to 1.9558; calibrated ridge scores 1.9685.

PointMaze interval scores under the same calibration rule are 2.7049 for Bayesian regression and 2.8951 for ridge. The continuous ranked probability score (CRPS) evaluates the full predictive distribution against observations. On SKIPP'D, without calibration, Bayesian regression has 13.55% lower CRPS and 12.00% lower interval score than ridge. Both differences are statistically significant; lower is better.

Thank you for the additional experiments and clarification. The revision strengthens the empirical evaluation, and I appreciate the effort to better establish the framework’s benefits. However, I share the concern that the methodological novelty remains limited. While the Bayesian formulation offers useful capabilities, the current evidence does not sufficiently establish it as a substantially new approach to visual world modeling, particularly given the lack of demonstrated benefit from visual inputs in the matched ablations. I therefore maintain my original score of 2

We thank the reviewer for acknowledging the strengthened empirical evaluation. We respectfully disagree, however, that the methodological novelty is limited. To the best of our knowledge, ViBR-WM is the first work to introduce a Bayesian regression framework for visual world modeling that integrates learned visual representations, physical histories, and known covariates within a unified probabilistic world model. The contribution is not simply applying Bayesian regression: ViBR-WM replaces the neural transition predictor with Bayesian regression of changes in a joint visual–physical state, performs Bayesian variable selection and posterior model averaging, supports both recursive joint-state and direct physical-target forecasting, and propagates parameter and disturbance uncertainty into predictive distributions.

We also directly address the reviewer’s concern regarding the benefit of visual inputs. The previous matched ablation asked whether vision provides additional information when informative numerical histories are already available. We have now added a complementary visual-only experiment, removing numerical histories, actions, and calendar predictors.

TaskViBR-WMConvLSTMPredRNNSimVPVisual-only TS
PointMaze0.9010 ± 0.01213.1206 ± 0.74491.1407 ± 0.00621.4355 ± 0.32755.3219 ± 1.7821
Wall0.9194 ± 0.02680.9781 ± 0.07310.9305 ± 0.04510.9538 ± 0.07911.3431 ± 0.5231
PhenoCam0.7649 ± 0.04330.9718 ± 0.02531.0281 ± 0.09270.8799 ± 0.11701.3513 ± 0.2912
SKIPP'D0.4275 ± 0.01430.5072 ± 0.04720.4290 ± 0.01950.4892 ± 0.02480.5476 ± 0.0768

NRMSE (lower is better), mean ± standard deviation over ten runs.

ViBR-WM achieves the lowest mean error on all four tasks using visual information alone, reducing error relative to the strongest comparator by 21.02%, 1.19%, 13.07%, and 0.35%, respectively. Thus, while vision may provide limited incremental improvement when strong numerical histories already encode the state, the new experiment directly demonstrates that ViBR-WM can effectively extract predictive information from visual inputs alone. We hope these results and the clarification of our methodological contribution address the reviewer’s concerns.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Thank You for Your Time, Careful Review, and Constructive Feedback

We sincerely thank all three reviewers and the Area Chair for their time, thoughtful feedback, and constructive suggestions. The rebuttal has helped us substantially strengthen both the positioning and empirical evaluation of ViBR-WM.

We would like to emphasize that our contribution is not that Bayesian regression, PCA, or visual representation learning is individually new. Rather, to the best of our knowledge, ViBR-WM is the first work to formulate visual world modeling within a Bayesian regression framework, using Bayesian inference as the predictive mechanism for joint visual–physical dynamics, with variable selection, posterior model averaging, interpretability, and uncertainty quantification. Methodological innovation often builds on established tools; the key contribution is how they are combined and used to create a new modeling paradigm.

The revised experiments further support this contribution. Across the standard benchmark tasks PointMaze, Wall, PhenoCam, and SKIPP'D, ViBR-WM achieves lower mean physical-target error than the main comparison methods. We also added SwinLSTM-B and visual-only experiments; with visual information alone, ViBR-WM again achieves the lowest mean error on all four tasks.

The main reviewer concerns were addressed as follows:

  • Reviewer Vo98: Added matched Bayesian-vs.-ridge and no-vision ablations, visual-only experiments, and improved uncertainty calibration and probabilistic evaluation.
  • Reviewer uy7a: Added component ablations, clarified the adapted-baseline comparisons, expanded cross-year/cross-site evaluation, and improved interval calibration.
  • Reviewer ZoL1: Added stronger and newer baselines, including SwinLSTM-B, additional ablations, broader real-world evaluation, and clearer discussion of the methodological novelty.

We have also uploaded reproducible code to facilitate independent verification and extension of the results.

We respect that reviewers may differ in their assessment of novelty, but we hope the final evaluation will consider the creativity of introducing a Bayesian paradigm into world modeling, the new capabilities this enables, and the consistent empirical performance across benchmark and real-world tasks.

Thank you again for your time and careful consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.