acceptodds
Under review as a conference paper at ICLR 2027

Competence Without Performance: Models Represent ARC-AGI-3 but Cannot Play It

Abstract

ARC-AGI-3 tests whether a model can enter an unfamiliar interactive world, infer its hidden rules through play, and act successfully on what it learns. We build the first Gymnasium- and DreamerV3-compatible environment wrapper for ARC-AGI-3 and use it to establish the first model-based reinforcement-learning baseline on the benchmark with a learned latent world model. Run on six public games, the agent scores zero on five of them under the benchmark's metric, Relative Human Action Efficiency (RHAE). To find out why, we ask whether these zeros mean that the agent has learned nothing about the games, or that it has learned useful structure but cannot turn it into play. Because DreamerV3 trains its world model on real frames and its policy only on imagined trajectories, what the model has learned can be measured separately from what the policy does. We find competence without performance. The world model fits every game at the single-step level, and linear probes on its frozen latent state decode which level is being played, above raw-pixel, elapsed-time and label-permutation controls, in many games on which the agent never scores. Yet the policy stays at its uniform initialization on five of six games. Ablations that pretrain the world model across games and that replace the reinforcement learner with a supervised imitation model confirm that the gap lies in exploration and policy learning rather than in representation. Learning a world model and learning to act from it therefore fail separately on ARC-AGI-3, and the benchmark score alone cannot tell the two failures apart.

Primary area:
interpretability and explainable AI
Keywords:
ARC-AGI-3, world models, model-based reinforcement learning, DreamerV3, competence versus performance, benchmark evaluation, exploration, abstract reasoning, representation probing

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_1RHMOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper studies the performance of DreamerV3 on ARC-AGI-3 and attempts to distinguish representation learning failures from policy learning failures. The authors observe that the world model achieves low reconstruction losses and encodes certain task-related information, while the learned policy remains nearly uniform and achieves zero reward on most games. Based on these observations and several ablations, the paper concludes that the primary bottleneck lies in exploration and policy learning rather than world-model representation.

While the empirical observations are potentially useful, I find the central conclusion insufficiently supported. In particular, the paper does not establish that the learned world model is accurate enough to support meaningful policy optimization in the first place.

Critical strengths
  • The paper provides a DreamerV3 baseline on ARC-AGI-3 and includes relatively extensive diagnostic experiments.
  • The authors report negative results transparently, including unsuccessful interventions and limitations of the learned world models.
Critical weaknesses
  • The world model itself appears inadequate. Low reconstruction loss does not imply accurate learned dynamics. In fact, the open-loop predictions consistently underperform a trivial copy-last-frame baseline. This raises serious doubts about whether the world model can provide meaningful imagined trajectories for policy optimization.
  • The policy learning process is insufficiently analyzed. The authors mainly rely on policy entropy and zero rewards to diagnose policy failure, without adequately examining imagined rewards, value predictions, or actor/critic optimization. It remains unclear whether the failure comes from exploration, inaccurate world-model dynamics, or ineffective policy learning.
  • The competence claim is overstated. Decoding level identity from latent representations provides limited evidence of task understanding, especially when the model cannot accurately predict environment transitions. The pretraining and imitation ablations also fail to isolate the underlying cause.
Questions for authors
  1. Given that the learned world model consistently underperforms copy-last-frame prediction, what evidence supports the claim that its dynamics are sufficiently accurate for meaningful imagined policy optimization?
  2. Can the authors provide direct diagnostics of imagined reward prediction, critic learning, and actor optimization to distinguish world-model failure from policy optimization failure?
  3. Would improving the world model's action-conditioned rollout accuracy, while keeping the policy architecture and exploration strategy fixed, improve downstream performance?
Reviewer_r9tyOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The work studies why DreamerV3 performs poorly on ARC-AGI-3. The main experiments evaluate six games, comparing training with and without world-model pretraining on human replays. The paper investigates reconstruction losses, information recoverable from the learned internal state, future-state predictions, and gameplay. Main conclusion is that agents can learn useful representations but fail to use them, due to the bottleneck in exploration and policy learning compared to representation learning.

Critical strengths
  1. The paper investigates beyond numbers, other metrics such as image reconstruction, information recoverable from the internal state, future-state predictions, and action-selection behavior.
  2. The work reports unsuccessful experiments and important limitations. Acknowledges poor multi-step predictions and unsuccessful experimental hypotheses.
Critical weaknesses
  1. The work doesn't show that the model representation is good enough to solve the problems. They evaluate reconstruction and the ability to identify the current level, which doesn't necessarily mean the representations are good enough for the model to solve the problem. Also, Table 4 uses the representation after the model sees the target state, which is not the same as predicting the next frame. The prediction test also shows that Dreamer’s eight-step predictions perform worse than copying last frame. Since the model learns from imagined trajectories, an inaccurate world model remains a potential contributor.

  2. The evaluation uses 4,102 possible actions and does not mask invalid actions. This makes exploration unnecessarily difficult. For example, in some games, only four actions are valid. The proposed variant, structured-action, also restricts unavailable action types, making it difficult to isolate the effect of action representation.

  3. The experiments do not establish that exploration/policy learning is the main bottleneck. The supervised TRM comparison changes both the model architecture and training method. The work does not show that a better policy can successfully use the same frozen DreamerV3 representation.

Questions for authors
  1. Can any other action-selection method successfully use the same frozen Dreamer representation? This would provide direct evidence that the information needed for successful gameplay is already present.

  2. Can the authors compare the original flat-action representation with the structured representation while applying the same available-action restrictions to both?

  3. Prior work already distinguishes learning useful representations (Pre-training) from learning a successful policy (RL), as the paper acknowledges. What new insight do the ARC-AGI-3 experiments provide apart from showing that reconstruction and level identification (good representation) do not guarantee successful gameplay?

Reviewer_5foxOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper claims that models like DreamerV3 can learn the world model properly, but its policy did not move beyond the initial state so score zero.

Methodology: Competence and performance are measured separately through DreamerV3 runs. Competence consists of predicting the next step of the game using DreamerV3’s world model, checking the task structure in the latent state (identifying the level and whether the level has been cleared just now), and predicting multi-step simulations. In the warm arm, the world model is initialized with a model pretrained on public human replays. The main results report the warm arm results.

Results: The model’s reconstruction loss is reduced significantly, but policy entropy mostly remains at the uniform random level. Checking the task structure in the latent state to identify the level works well in some games. However, multi-step prediction does not work properly. Performance is measured using RHAE, the actual scoring system on ARC-AGI-3; most games received zero points, with only vc33 receiving a 5% score. This demonstrated the difference between competence and performance, showing that the model performs single-step predictions correctly but fails to execute actions.

Critical strengths

Separating representation from performance: The paper combines behavioral metrics, controlled latent probes, and rollout tests to distinguish what an agent represents from what it can do. This provides a useful framework for diagnosing failures hidden by a zero benchmark score.

Critical weaknesses
  1. Unclear probe data splits: The probe training/evaluation splits and overlap with pretraining replays are insufficiently documented, making generalization difficult to assess. This warrants clarification, not a claim of confirmed leakage.
  2. Limited evidence of competence: Despite three controls, the key level-decoding result covers only two games at warm seed 0. Level identity does not establish rule understanding, and all six models underperform copy-last-frame in eight-step prediction.
  3. Pretraining does not rule out a world-model bottleneck: Lower initial reconstruction loss need not imply better dynamics or reward prediction. Zero scores in both conditions therefore cannot isolate exploration as the bottleneck.
  4. The TRM comparison is confounded: Architecture, objective, and supervision change together, so higher scores cannot be attributed solely to removing exploration. The authors acknowledge this; Appendix F adds controller-side evidence but does not fully isolate the causes.
Questions for authors

What does knowing the level have to do with understanding the game mechanics? Also, I don’t understand why the world model is good at single-step predictions but fails at multi-step predictions.

Appendix D appears empty on p.13, but its results are in Tables 6–7 on p.14. Clearer placement or cross-referencing would help.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.