- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2(+1)
- Soundness
- 2
Read review
Paper summary
In my understanding, this paper introduces a test-time world-model learning and planning pipeline that resembles model-based RL for ARC-AGI-3. During interaction, the agent learns an executable transition model for next-state prediction, a reward model for goal evaluation, and an abstraction module that maps pixel observations into an object-oriented structured state. In parallel, it maintains a natural-language proto-world-model for unresolved hypotheses. Based on the learned world model, the agent either performs reward-directed planning or conducts epistemic exploration. Compared with WorldCoder, OPINE-World additionally introduces a Bayesian belief state to quantify uncertainty over object-action effects. High uncertainty is interpreted as evidence that the current ontology may be missing relevant state distinctions, which can trigger targeted exploration and state refinement, and may subsequently lead to revisions of the transition and reward models.
Critical strengths
I think the paper's major advantage is its framing in test-time ``RL'' for composite reasoning: The introduction of the Episodic Replay Buffer (memory) and Bayesian Belief State to quantify uncertainty seems to be a reasonable novelty to me, as I regard them as answering two key questions: what to learn from, and where to explore to collect data for continual world-model learning.
Audience in agentic reasoning/planning, harness design, and recursive improvement may find this work interesting. That's why I think this paper is worth a score of at least 2 for ICLR2027.
Critical weaknesses
-
The major weakness of this paper is its writing. Section 3 is particularly hard to follow due to the large number of notations and interacting representations introduced within a short space. For example, the method simultaneously maintains , , and , and later introduces additional quantities such as , , and . The presentation frequently switches between the natural-language model, Bayesian belief state, executable model, and planning loop, making it difficult to understand the overall workflow and the role of each component on a first read. I suggest that the authors reframe Sec. 3 with less but necessary notation, add a necessary illustration of how the pipeline works, with relatively complete examples in the appendix.
-
The proposed framework seems highly complex and computationally expensive (2B tokens for individual environments), and its advantages or weaknesses against baselines under compute normalization have not been discussed (e.g. matched token usage, wall-clock time, or inference cost).
Questions for authors
To the authors: I am confused about several details in the paper, although some of them may be due to my own misunderstanding. I would appreciate it if the authors could answer the following questions transparently. It is completely acceptable if some design choices not fully optimal, as long as their motivations and limitations are clearly explained.
To the AC and the authors: Since I cannot fully understand the paper before the discussion period, I may ask further questions during the discussion.
-
The authors demonstrate the effectiveness of their framework using Claude, while I am not sure whether the findings generalize to a broader range of models. Could the authors try models from different model families/providers and add some small-scale controlled experiments?
-
Do the authors claim that their method is optimal against other baselines (e.g., WorldCoder) for all problems due to its planning mechanism? If so, I would like to better understand why. Could the authors provide a brief explanation of where the advantage comes from? If not, I would still appreciate a discussion or qualitative comparison with related methods to clarify the inductive biases introduced by the proposed framework.
-
Continue from Q2, ARC-AGI-V3 is an interesting benchmark, but the current experiments are limited to it. I think the paper would be stronger if the authors could also evaluate on tasks used by WorldCoder, ideally with a comparably weak LLMs, to better illustrate the advantages and limitations of the proposed method relative to other baselines.
I think addressing these questions would improve the paper from an Occam's Razor perspective, by helping identify the minimal, controllable advantages and limitations introduced by the method's inductive biases.
5 comments
Response to Reviewer LTRb (2/3)
Question #1: Do the findings generalize beyond Claude?
The authors demonstrate the effectiveness of their framework using Claude, while I am not sure whether the findings generalize to a broader range of models. Could the authors try models from different model families/providers and add some small-scale controlled experiments?
We interpret this point as: the harness may be turned to work on Claude, and the full system should be tested on a different model family.
We agree that results on a second model family are needed to show the findings are not specific to Claude, so we ran the same system across GPT-5.5 high. The RHAE for this model was improved from 0.5% to 84.3%, demonstrating similar performance improvements across model families that OPINE-World was not built on. We have updated Table 1, which now additionally compares the system on GPT-5.5 high:
| System | Mean RHAE ± SE | Levels completed | Games completed |
|---|---|---|---|
| OPINE-World (GPT-5.5 high) | 84.3 ± 3.5 | 176/183 | 21/25 |
| OPINE-World (Opus 4.8 high) | 78.4 ± 6.3 | 160/183 | 20/25 |
| No (Opus 4.8 high) | 58.2 ± 8.3 | 126/183 | 12/25 |
| baseline1 | 63.7 ± 7.4 | 146/183 | 14/25 |
| DreamTeam | 38.1 ± 7.6 | 98/183 | 6/25 |
| NOOA | 50.2 | 118/183 | – |
| Continual | 22.2 ± 5.9 | 86/183 | 5/25 |
Per-game results are in Appendix F.1.
Response to Reviewer LTRb (3/3)
Question #2: Is the method claimed to be optimal over WorldCoder, and where does the advantage come from?
Do the authors claim that their method is optimal against other baselines (e.g., WorldCoder) for all problems due to its planning mechanism? If so, I would like to better understand why. Could the authors provide a brief explanation of where the advantage comes from?
We interpret this point as: whether OPINE-World is claimed to outperform WorldCoder on every problem, and if not, which of its design choices produce the gains or losses.
We agree that the paper should more specifically break down the benefits over prior work such as WorldCoder, therefore, we have run a WorldCoder comparison and added results to Section 5.1 (Table 2), and a discussion of where WorldCoder outperforms OPINE-World to Appendix E.
Solved episodes (x optimal)
| Environment | WorldCoder | OPINE-World |
|---|---|---|
| Empty-5x5 | 1.45 ± 0.02 | 1.52 ± 0.35 |
| DoorKey-5x5 | 1.27 ± 0.12 | 1.08 ± 0.04 |
| Unlock | 1.32 ± 0.05 | 1.10 ± 0.04 |
| Fetch-6x6-N2 | 1.59 ± 0.04 | 1.21 ± 0.14 |
| UnlockPickup | – | 1.31 ± 0.10 |
| BlockedUnlockPickup | – | 1.60 ± 0.19 |
| Sokoban, 2 boxes | 1.41 ± 0.20 | 1.23 ± 0.01 |
| Sokoban, 3 boxes | 1.04 ± 0.04 | 1.04 ± 0.02 |
Episodes to mastery
| Environment | WorldCoder | OPINE-World |
|---|---|---|
| Empty-5x5 | 11.0 | 11.0 |
| DoorKey-5x5 | 11.0 | 13.3 ± 2.3 |
| Unlock | 11.0 | 11.0 |
| Fetch-6x6-N2 | 11.0 | 11.0 |
| UnlockPickup | – | 11.0 |
| BlockedUnlockPickup | – | 15.0 ± 4.0 |
| Sokoban, 2 boxes | 12.0 ± 1.0 | 16.7 ± 3.0 |
| Sokoban, 3 boxes | – | 15.7 ± 4.2 |
In a direct comparison on WorldCoder-compatible domains, OPINE-World produces action sequences closer to the optimal solution despite both systems having access to a planner. WorldCoder's planner fails to find a reward-reaching plan in UnlockPickup due to the absence of synthesized heuristics and cannot progress to BlockedUnlockPickup, while OPINE-World eventually masters all six environments. On Sokoban, WorldCoder masters the 2-box environment in fewer episodes; however, even with a grid search across a 1000-12000 node depth limit for its MCTS planner, its most successful configuration at 6000 nodes exhausts its generous 24h search budget on the 3-box layouts and is unable to progress. This suggests that OPINE-World provides more efficient and computable solutions to simpler domains.
OPINE-World does not improve on domain specific approaches, such as WorldCoder, in every case. In these domains, the symbolic state, action names, and goal may be supplied, which removes the uncertainty that OPINE-World's object abstraction, goal inference, and exploration are designed for. Outside of these domains, WorldCoder is unable to progress as it lacks the systems to acquire this information. Within these domains, due to the rigid heuristic and sub-optimality of its MCTS planner, WorldCoder is unable to progress in certain levels due to planner intractability. These are the cases where OPINE-World demonstrates a higher capability.
Question #3: Can you evaluate on WorldCoder's tasks with a comparably weak LLM?
I think the paper would be stronger if the authors could also evaluate on tasks used by WorldCoder, ideally with a comparably weak LLMs, to better illustrate the advantages and limitations of the proposed method relative to other baselines.
We interpret this point as: an evaluation on WorldCoder's own tasks, with a weak model, would show the advantages and limitations of the method outside ARC-AGI-3.
We agree that a comparison on WorldCoder's tasks shows both the advantages and the limitations of the method, so we have added the WorldCoder comparison to Section 5.1 (Table 2) and Appendix E (refer to above table in Question #2). We run OPINE-World and WorldCoder on WorldCoder's MiniGrid and Sokoban domains, and give each agent information exposed in the original WorldCoder benchmark, with each run capped at 24 hours of planner CPU compute time on a Ryzen 7600X. We measure actions relative to the optimal plan and episodes to mastery. Both systems in the WorldCoder comparison use a weaker GPT-6 Luna so the domains don't saturate.
See our response to Question #2 for findings.
Response to Reviewer LTRb (1/3)
Thank you for your review and suggestions. We address each point below.
Critical Weakness 1
We interpret this point as: Section 3 introduces too much notation at once for the overall workflow, and the role of each component is not introduced in a sensible manner, to be clear on a first read.
We agree that section 3 of our paper is notationally dense and hard to follow on a first read, so we have restructured Section 3 to reduce the necessary notation to the components that minimally formalize the system shown in Figure 1, we have added an illustration of the system on an example domain in the new Figure 2, which Section 3 now references off during the introduction of components, and we have added concrete examples on each representation in Appendix A.
Critical Weakness 2
We interpret this point as: the cost of the method has not been compared against the baselines under matched tokens, wall-clock time, or inference costs. Specifically: In the case of a large cost disparity, the value and usability of our system is degraded, of which has not been discussed.
We agree that a compute-normalized comparison is needed to assess the method's advantages and weaknesses, therefore, we compare score under wall-clock budgets and inference cost in Table 18 (Appendix F.9). All systems use GPT-5.5 high.
RHAE within wall-clock budget
Tokens and cost
We were able to accrue enough compute to run our system across ARC-3 on GPT-5.5 high, and we report results that are accessible to us via the respective author’s publicly released results if they have a valid GPT 5.5-high run. For methods that have a blank entry, this result was not reported in their original paper or run artifacts.
On GPT-5.5 high, OPINE-World costs less than baseline1 and DreamTeam on the same model and scores higher at all wall-clock budgets. NOOA is reported by the NOOA team to score higher at that budget and at a lower cost, however, their final score is significantly lower. Specifically: The compute disparity comes from the single-agent nature of NOOA, combined with the lack of repeated tool calls and low total runtime. We do not claim that OPINE-World is superior under highly limited compute and time constraints as this is not the setting we aim to optimize, but it is a good direction for future work. Alongside baseline1 and DreamTeam approaches, we aim to demonstrate the capability of a system with tools and structures for pushing performance as high as is reasonable within open world exploration tasks.