- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes Active Epistemic World Modeling (AEWM), which selects informative interactions to improve world model predictions. And its CDR module estimates counterfactual uncertainty reduction through ensemble reweighting.
Critical strengths
- Interesting research motivation: The paper tries to address a vital problem, selecting informative interactions to improve world model predictions rather than simply maximizing the local uncertainty.
- An attempt at efficient information acquisition: CDR provides an approximation to counterfactual uncertainty reduction without requiring repeated model retraining.
Critical weaknesses
- Poor writing, hard to read: The paper is difficult to follow due to its complicated organization and insufficient explanations. For example, the introduction section needs much improvement.
- Overly simplistic experimental setting and evaluation: The experiments are restricted to a single synthetic environment with low-dimensional states and actions. The primary evaluation focuses on one-step prediction accuracy in only 60 zero-delay worlds, without experiments on standard control benchmarks. More experiments are necessary.
- What is the specific advantage of the proposed framework for decision-relevant exploration when the current experiments rely on randomly sampled queries rather than planner-generated ones? I guess this should be clarified.
Questions for authors
Can the authors provide experiments on more challenging or established benchmarks, including evaluations of multi-step prediction and downstream control performance?
Reviewer aD3u: Prediction Through 16 Steps, Queries Generated by a Planner, and Downstream MPC Evaluation
We apologize for the delayed response. Thank you for the suggestions on prediction over multiple steps, downstream control, and queries generated by a planner. We have added these evaluations and clarified their relationship to the original experiment.
Prediction over multiple steps. We evaluated H=1, 4, 8, and 16 on 12 fixed worlds, with 16 action sequences per world and eight repeats with different simulator noise. Each rollout uses a fixed context. Predictions are compared with mean simulator trajectories. At B=64, state RMSEs are:
At H=16, global CDR reduces state RMSE by 30.2% relative to local. The paired difference is −0.007166 [−0.009293, −0.005040]. RMSE for predicted returns is 0.160701 versus 0.236797, a 32.1% reduction, with difference −0.076096 [−0.102355, −0.049837]. Random remains competitive, with lower mean errors and intervals for its differences from CDR that include zero. The appendix reports the complete budget and horizon results.
Queries generated by a planner. CEM generates 32 action sequences of length four from the learned models. Each member's discounted predicted return defines a query output. We test uniform weights and weights based on gaps in predicted returns. Query generation uses only the learned models, without hidden mechanisms or simulator outcomes. This directly tests the proposed use of a planner to supply the prediction targets, instead of relying only on randomly sampled actions. At B=64/H=16, planner CDR has state RMSE 0.016188, compared with 0.016557 for global CDR using random queries. The revised paper describes both query sources and reports their measured performance.
Downstream control. After B=0/32/64 identification interactions, we run CEM MPC with horizon eight for up to 32 control steps on the same 12 worlds with matched evaluation seeds. At B=64, mean returns are −23.276 for random, −23.080 for local, −23.121 for global CDR, and −23.358 for planner CDR. Success rates are 35.0%, 36.7%, 33.3%, and 31.7%, respectively. The interval for the return difference between global CDR and local is [−1.760, 1.676]. This provides a direct downstream evaluation, with no clear return advantage detected under the tested protocol. The paper distinguishes this result from the improvement in prediction accuracy.
Benchmark scope and presentation. These experiments extend the evaluation within the simulator with hidden physical parameters and no action delays. They do not replace an established second benchmark. We have clarified this scope, retained the original exposition where applicable, and updated the method and results to explain the query sources and downstream evaluation.
Reported intervals use five training seeds as the replication unit and are unadjusted for multiple comparisons.
Should you have any other concerns, we would be willingly to answer in further discussions.