- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies belief representation for learned code world models under partial observability. It introduces distributional particles, which retain distributions over state variables within each particle, and Abstract Belief Coder (ABC), which uses an LLM to propose state representations and model programs operating through a shared factor library. Candidate models are evaluated using filtering likelihoods. Experiments on six MiniGrid tasks and two customized Crafter tasks show improved planning performance over concrete-particle baselines. Additional experiments with hand-authored models, shared observation histories, and varying particle counts help characterize the benefits of the proposed belief representation.
Critical strengths
-
The paper addresses an important scalability bottleneck in learned probabilistic world models. Even an accurate learned model may be ineffective for decision-making if belief inference cannot handle uncertainty over multiple hidden variables. Retaining structured distributions within particles is a well-motivated way to address this issue, and integrating this representation with LLM-generated model programs is a useful contribution.
-
The empirical results provide complementary evidence for the approach. Improvements persist with hand-authored models, indicating that the gains are not solely due to differences in model generation. The belief-tracking comparison on identical observation histories and the particle-budget sweep further support the practical benefits of the representation in the evaluated settings.
Critical weaknesses
The main limitation is the restricted evaluation scope. All experiments use controlled grid-world environments with structured symbolic observations. Although the evaluation includes Crafter, these are customized tasks with small fixed maps and simplified objectives, rather than the broader environment. Consequently, the results establish effectiveness in these settings but provide limited evidence of scalability to larger, less structured environments with stronger dependencies among hidden variables.
Questions for authors
- How do the ground-truth and generated models handle transitions or observation likelihoods that couple factors (Appendix B.2), concretely in MultiMovingBalls and DoorMaze? Since d-particles are never split, is the joint posterior projected back onto a product form, and if so, how?
- Are there results in more complex practical domains?
We thank the reviewer for the constructive feedback.
Q1. How are coupled factors handled in MultiMovingBalls and DoorMaze? Is the joint posterior projected back onto a product form?
Yes. Within a d-particle, factors are kept independent. A coupling within one particle is therefore not represented exactly. T and O still partly account for it when they update each factor and compute the particle weight. For example, in MultiMovingBalls, O downweights a particle by the probability that two balls share a cell.
The benefit of this representation outweighs the error from the independence assumption. We added an experiment that measures this error directly in MultiMovingBalls with 3, 4, and 5 balls, using ground-truth models for both methods to exclude model-generation errors. We computed the exact joint posterior by enumerating all states while preserving dependencies between balls, and compared each method’s belief with it under the same action–observation histories using total variation (TV) distance (0: identical, 1: no overlap).
Table R8. TV distance from the exact joint posterior. Lower is better.
ABC-GT’s TV grows with the number of balls, which quantifies the error of the independence approximation. Pinductor-GT’s TV stays at 1.000 because K concrete particles cover only K joint states, while the posterior spreads over many more. Concrete particles can represent any coupling exactly, but this coverage gap outweighs that advantage. The gain also persists under much stronger coupling (Table R10: 89–95% vs. 35–52% success).
W1 / Q2. Restricted evaluation scope / Are there results in more complex practical domains?
We thank the reviewer for this suggestion. We add ALFWorld, a non-grid household domain with long-horizon goals (Table R9). We also add MultiMovingBalls variants with strongly coupled hidden variables (Table R10). ABC outperforms Pinductor by a large margin in both. Both experiments use ground-truth models, so the comparison reflects the belief representation rather than model-learning errors.
Our main evaluation follows the prior code world models for POMDPs, POMDP-Coder (Curtis et al., 2025) and Pinductor (Six et al., 2026), which both use MiniGrid. This setting isolates hidden-state tracking from perception and control errors, and it allows direct comparison with both methods. The paper already extends MiniGrid with moving hidden objects and includes Crafter.
ALFWorld. ALFWorld (Shridhar et al., 2021) is a partially observable household benchmark with long-horizon, multi-step goals. Each method receives a human-written model of the environment rules, written from the same rule specification in its own state representation.
Table R9. Success rate on ALFWorld with ground-truth models (100 episodes per task).
ABC-GT achieves higher success on both tasks, extending the evaluation beyond grid worlds.
Coupled hidden variables. The reviewer also raises stronger dependencies among hidden variables. We built three variants of MultiMovingBalls (9×9) in which the balls chase each other:
Without a joint factor, ABC-GT approximates this coupling, but its factors still reflect the pursuit: T updates the follower’s factor from the leader’s distribution. Pinductor’s bottleneck, in contrast, remains: each particle commits to one joint placement of all balls, so a fixed budget covers few plausible states.
Table R10. Success rate with ground-truth models (100 episodes per task).
ABC keeps its margin in all three variants, including deterministic coupling.
References.