- Rating
- 1
- Confidence
- 3(−1)
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes Offline Distributional World Model Value Iteration (ODWVI) for distributional value iteration in learned world models under offline partial observability. The method combines behavior-support constraints, a support-dependent reward shift, and OOD regularization to reduce the exploitation of model errors on weakly supported actions. The paper also provides a Wasserstein-based analysis of model-error propagation and convergence, and evaluates the method in three ViZDoom environments.
Critical strengths
The paper addresses the amplification of model errors through Bellman backups in offline world-model planning, which is a meaningful research problem.
Critical weaknesses
-
The paper contains numerous overtly defensive statements that appear intended to preempt potential reviewer criticisms. These statements substantially detract from the paper's scholarly tone and professionalism.
-
Key ablation studies are missing, making it difficult to assess the individual contributions of the distributional critic and the support-dependent left shift.
-
The main experiments provide insufficient baseline coverage, with some strong baselines reported in only one environment. Furthermore, the evaluation is limited to three ViZDoom tasks, which is too narrow to support broader conclusions.
Questions for authors
-
Could the authors clarify the meaning and purpose of the overtly defensive statements in lines 227–230, 243–247, 288–290, 422–423, 430–431, and 475–478?
-
Could the authors provide additional experiments addressing Weaknesses 2 and 3 to establish the method's broader applicability and isolate the contributions of the relevant components?
I would raise my score if the authors satisfactorily address these two key questions and resolve my concerns. Thank you for considering them.
5 comments
Thank you for your response and clarifications.
However, I believe the paper has substantial issues with its organization and presentation that require more than minor wording changes. Since ICLR allows manuscript revisions during rebuttal, I would have expected these concerns to be addressed directly in a revised manuscript rather than through promises of future changes.
As the manuscript remains unchanged, I will maintain my current rating. I encourage the authors to submit a revised version with the changes highlighted.
We have uploaded a substantially revised manuscript, and we appreciate the opportunity to address your concerns directly. In particular:
Organization and presentation: We revised the Introduction and reorganized the theoretical exposition, methodology, and results to establish a clearer scientific narrative. Section 6 now presents the algorithm through dedicated subsections and a new architectural overview (Figure 1). We also removed or rewrote the defensive passages identified in your review, presenting the relevant scientific qualifications more directly and consolidating the principal limitations in Section 8.
Component ablations: Section 7.3 and Figure 2 now report experiments examining the support-dependent left shift, OOD regularization, and categorical versus scalar critics. The complete paired statistical comparisons are provided in Appendix F. Removing OOD regularization significantly reduces performance across all four evaluated support sizes, while the categorical configuration significantly outperforms the scalar configuration at (k=1) after Holm correction.
Experimental scope: We have clarified the roles of the evaluated baselines and explicitly acknowledged the remaining limitations in environment and baseline coverage. Our conclusions remain restricted to the evaluated settings.
We have also provided a detailed summary of the revisions and their locations in our new top-level comment.
We are grateful for your feedback, which motivated substantial improvements to the manuscript's organization, clarity, and experimental validation. We hope the revised version addresses your principal concerns and welcome any further feedback.
Thank you for your careful review and specific feedback. We appreciate the opportunity to improve the paper’s presentation and provide clearer evidence about the contributions of its components.
We hope these additions address the component-isolation concern and make the remaining limitations clearer.