- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper studies the performance of DreamerV3 on ARC-AGI-3 and attempts to distinguish representation learning failures from policy learning failures. The authors observe that the world model achieves low reconstruction losses and encodes certain task-related information, while the learned policy remains nearly uniform and achieves zero reward on most games. Based on these observations and several ablations, the paper concludes that the primary bottleneck lies in exploration and policy learning rather than world-model representation.
While the empirical observations are potentially useful, I find the central conclusion insufficiently supported. In particular, the paper does not establish that the learned world model is accurate enough to support meaningful policy optimization in the first place.
Critical strengths
- The paper provides a DreamerV3 baseline on ARC-AGI-3 and includes relatively extensive diagnostic experiments.
- The authors report negative results transparently, including unsuccessful interventions and limitations of the learned world models.
Critical weaknesses
- The world model itself appears inadequate. Low reconstruction loss does not imply accurate learned dynamics. In fact, the open-loop predictions consistently underperform a trivial copy-last-frame baseline. This raises serious doubts about whether the world model can provide meaningful imagined trajectories for policy optimization.
- The policy learning process is insufficiently analyzed. The authors mainly rely on policy entropy and zero rewards to diagnose policy failure, without adequately examining imagined rewards, value predictions, or actor/critic optimization. It remains unclear whether the failure comes from exploration, inaccurate world-model dynamics, or ineffective policy learning.
- The competence claim is overstated. Decoding level identity from latent representations provides limited evidence of task understanding, especially when the model cannot accurately predict environment transitions. The pretraining and imitation ablations also fail to isolate the underlying cause.
Questions for authors
- Given that the learned world model consistently underperforms copy-last-frame prediction, what evidence supports the claim that its dynamics are sufficiently accurate for meaningful imagined policy optimization?
- Can the authors provide direct diagnostics of imagined reward prediction, critic learning, and actor optimization to distinguish world-model failure from policy optimization failure?
- Would improving the world model's action-conditioned rollout accuracy, while keeping the policy architecture and exploration strategy fixed, improve downstream performance?