acceptodds
Under review as a conference paper at ICLR 2027

Abstract Beliefs for Code World Models

Abstract

Code world models represent environment dynamics as executable programs. Under partial observability, existing methods maintain beliefs with concrete particles, each committing every hidden variable to a single value. As joint assignments grow combinatorially with the number of hidden variables, covering the plausible states exhausts a fixed particle budget. We introduce the distributional particle (d-particle), whose state variables can each hold a distribution over possible values, so one particle can represent many possible worlds. Building on d-particles, we propose Abstract Belief Coder (ABC): a shared Factor Library defines distribution families whose operations propagate and condition these distributions and return the likelihoods that weight particles, and an LLM proposes state representations and programs ranked by this filtering likelihood. On six partially observed MiniGrid tasks, ABC succeeds in 90.6% with ground-truth models versus 55.0% for a concrete-particle baseline, and in 75.6% versus 46.1% with LLM-generated models; it also leads on two Crafter tasks.

Primary area:
reinforcement learning
Keywords:
Code World Models, POMDPs, Belief Representation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_zx6zOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies belief representation for learned code world models under partial observability. It introduces distributional particles, which retain distributions over state variables within each particle, and Abstract Belief Coder (ABC), which uses an LLM to propose state representations and model programs operating through a shared factor library. Candidate models are evaluated using filtering likelihoods. Experiments on six MiniGrid tasks and two customized Crafter tasks show improved planning performance over concrete-particle baselines. Additional experiments with hand-authored models, shared observation histories, and varying particle counts help characterize the benefits of the proposed belief representation.

Critical strengths
  1. The paper addresses an important scalability bottleneck in learned probabilistic world models. Even an accurate learned model may be ineffective for decision-making if belief inference cannot handle uncertainty over multiple hidden variables. Retaining structured distributions within particles is a well-motivated way to address this issue, and integrating this representation with LLM-generated model programs is a useful contribution.

  2. The empirical results provide complementary evidence for the approach. Improvements persist with hand-authored models, indicating that the gains are not solely due to differences in model generation. The belief-tracking comparison on identical observation histories and the particle-budget sweep further support the practical benefits of the representation in the evaluated settings.

Critical weaknesses

The main limitation is the restricted evaluation scope. All experiments use controlled grid-world environments with structured symbolic observations. Although the evaluation includes Crafter, these are customized tasks with small fixed maps and simplified objectives, rather than the broader environment. Consequently, the results establish effectiveness in these settings but provide limited evidence of scalability to larger, less structured environments with stronger dependencies among hidden variables.

Questions for authors
  • How do the ground-truth and generated models handle transitions or observation likelihoods that couple factors (Appendix B.2), concretely in MultiMovingBalls and DoorMaze? Since d-particles are never split, is the joint posterior projected back onto a product form, and if so, how?
  • Are there results in more complex practical domains?
1 comment

We thank the reviewer for the constructive feedback.

Q1. How are coupled factors handled in MultiMovingBalls and DoorMaze? Is the joint posterior projected back onto a product form?

Yes. Within a d-particle, factors are kept independent. A coupling within one particle is therefore not represented exactly. TT and OO still partly account for it when they update each factor and compute the particle weight. For example, in MultiMovingBalls, OO downweights a particle by the probability that two balls share a cell.

The benefit of this representation outweighs the error from the independence assumption. We added an experiment that measures this error directly in MultiMovingBalls with 3, 4, and 5 balls, using ground-truth models for both methods to exclude model-generation errors. We computed the exact joint posterior by enumerating all states while preserving dependencies between balls, and compared each method’s belief with it under the same action–observation histories using total variation (TV) distance (0: identical, 1: no overlap).

Table R8. TV distance from the exact joint posterior. Lower is better.

Belief3 balls4 balls5 balls
ABC-GT0.1120.1700.234
Pinductor-GT1.0001.0001.000

ABC-GT’s TV grows with the number of balls, which quantifies the error of the independence approximation. Pinductor-GT’s TV stays at 1.000 because K concrete particles cover only K joint states, while the posterior spreads over many more. Concrete particles can represent any coupling exactly, but this coverage gap outweighs that advantage. The gain also persists under much stronger coupling (Table R10: 89–95% vs. 35–52% success).

W1 / Q2. Restricted evaluation scope / Are there results in more complex practical domains?

We thank the reviewer for this suggestion. We add ALFWorld, a non-grid household domain with long-horizon goals (Table R9). We also add MultiMovingBalls variants with strongly coupled hidden variables (Table R10). ABC outperforms Pinductor by a large margin in both. Both experiments use ground-truth models, so the comparison reflects the belief representation rather than model-learning errors.

Our main evaluation follows the prior code world models for POMDPs, POMDP-Coder (Curtis et al., 2025) and Pinductor (Six et al., 2026), which both use MiniGrid. This setting isolates hidden-state tracking from perception and control errors, and it allows direct comparison with both methods. The paper already extends MiniGrid with moving hidden objects and includes Crafter.

ALFWorld. ALFWorld (Shridhar et al., 2021) is a partially observable household benchmark with long-horizon, multi-step goals. Each method receives a human-written model of the environment rules, written from the same rule specification in its own state representation.

Table R9. Success rate on ALFWorld with ground-truth models (100 episodes per task).

TaskABC-GTPinductor-GT
Examine in Light82%61%
Pick Two & Place28%10%

ABC-GT achieves higher success on both tasks, extending the evaluation beyond grid worlds.

Coupled hidden variables. The reviewer also raises stronger dependencies among hidden variables. We built three variants of MultiMovingBalls (9×9) in which the balls chase each other:

  • One Follower: red steps toward yellow's updated cell with p=0.8p=0.8.
  • Two Followers: a pursuit chain; red follows yellow and blue follows red, each with p=0.8p=0.8.
  • One Strict-Follower: yellow and red form a hard-coupled length-2 snake; red deterministically enters yellow's previous cell.

Without a joint factor, ABC-GT approximates this coupling, but its factors still reflect the pursuit: TT updates the follower’s factor from the leader’s distribution. Pinductor’s bottleneck, in contrast, remains: each particle commits to one joint placement of all balls, so a fixed budget covers few plausible states.

Table R10. Success rate with ground-truth models (100 episodes per task).

TaskABC-GTPinductor-GT
One Follower90%44%
Two Followers89%35%
One Strict-Follower95%52%

ABC keeps its margin in all three variants, including deterministic coupling.

References.

  • Valentin Six et al. Learning POMDP World Models from Observations with Language-Model Priors. NeurIPS, 2026.
  • Aidan Curtis et al. LLM-Guided Probabilistic Program Induction for POMDP Model Estimation. CoRL, 2025.
  • Mohit Shridhar et al. ALFWorld: Aligning Text and Embodied Environments for Interactive Learning. ICLR, 2021.
Reviewer_iSRxOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces Abstract Belief Coder (ABC), a method for handling uncertainty in LLM-generated code world models. Instead of representing each possible world with a concrete particle, ABC uses distributional particles that can represent multiple possible states. An LLM proposes the state representation and executable world-model programs, while a shared Factor Library handles belief updates. The authors evaluate ABC on six MiniGrid and two Crafter tasks.

Critical strengths
  1. Useful approach: The paper addresses the difficulty of tracking multiple hidden variables with concrete particles. Combining distributional particles with LLM-generated world models is a useful contribution, even though the underlying Rao–Blackwellization technique is established.

  2. Strong experimental analysis: The authors compare ABC using both generated and ground-truth world models. They also compare belief quality on identical observation histories and vary the particle count. These experiments help isolate the benefit of the belief representation. The improvements are particularly convincing on tasks with multiple hidden objects.

Critical weaknesses
  1. Limited generalizability: ABC relies on a predefined Factor Library with five distribution families and restricted update operations. It is unclear how well the method would handle strongly correlated hidden variables or complex nonlinear dynamics. The current experiments do not directly test these settings.

  2. Planning and evaluation limitations: ABC uses open-loop planning without simulating future observations. This prevents it from explicitly considering the value of information-gathering actions. It also uses representative values for some uncertain interactions rather than evaluating the full distribution. Furthermore, the generated-model improvements are less consistent on harder tasks. For example, ABC achieves 43.3% success on DoorMaze compared with 40.0% for Pinductor. The evaluation uses only 30 episodes per task, which makes smaller differences difficult to assess reliably.

Questions for authors
  1. Could the authors evaluate ABC on a task with strongly correlated hidden variables to understand the limits of the factorized representation?

  2. How much does open-loop planning limit ABC's performance? Could the authors compare against a belief-aware planner that explicitly considers future observations and information-gathering actions?

  3. Could the authors evaluate the impact of the representative-value approximation used during planning?

  4. Could the authors report results across multiple independently generated world models? This would help establish how sensitive the results are to LLM model-generation variability, especially on the harder tasks.

5 comments

We thank the reviewer for the thoughtful questions; each led to a new experiment or analysis.

W1. With only five predefined distribution families, can ABC handle strongly correlated variables and complex nonlinear dynamics?

The five families are examples, and any family can be added as long as it is closed under its transition and observation updates. Even the current five cover every discrete variable, since Categorical is closed under any transition and observation over a finite domain, as well as continuous variables with linear-Gaussian dynamics through Gaussian.

Variables that are hard to represent with a single factor (e.g., a Gaussian mixture) can still be handled at extra computation. They can be represented with Dirac factors over more particles, or approximate updates (e.g., linearization or moment matching) can be added to the library, which we leave to future work.

Empirically, ABC-GT outperformed Pinductor-GT on all three strongly correlated tasks. In three pursuit variants where balls follow one another, including deterministic coupling, ABC-GT achieved 90%, 89%, and 95% success versus 44%, 35%, and 52% for Pinductor-GT over 100 episodes per task.

Q1. Could the authors evaluate ABC on a task with strongly correlated hidden variables to understand the limits of the factorized representation?

Thank you for raising this issue. With strongly correlated hidden variables, the factorized representation does introduce error from the independence assumption between factors, but the gain from representing many joint states with a few d-particles outweighs it. To show this, we conducted two evaluations. First, we measured ABC’s belief error against the exact joint posterior. Second, we evaluated how ABC performs under stronger correlations between hidden variables. In all settings, ABC-GT outperformed Pinductor-GT.

Approximation error evaluation. We measured ABC’s belief error in MultiMovingBalls with 3, 4, and 5 balls, using ground-truth models for both methods to exclude model-generation errors. We computed the exact joint posterior by enumerating all states while preserving dependencies between balls, and compared each method’s belief with it under the same action–observation histories using total variation (TV) distance (0: identical, 1: no overlap).

Table R3. TV distance from the exact joint posterior. Lower is better.

Belief3 balls4 balls5 balls
ABC-GT0.1120.1700.234
Pinductor-GT1.0001.0001.000

ABC-GT’s TV grows with the number of balls, which quantifies the error of the independence approximation. Pinductor-GT’s TV is near 1 because K concrete particles cover only K joint states, while the posterior spreads over many more. This gap is the gain from factorization, and it outweighs the approximation error.

Stronger correlations. We added three variants of MultiMovingBalls (9×9 grid) in which the balls are coupled by pursuit, not only by collisions. As above, we used ground-truth models to exclude model-generation errors. ABC-GT keeps each ball as a separate factor, with no joint factor, in all three variants.

  • One Follower: red steps toward yellow's updated cell with p=0.8p=0.8.
  • Two Followers: a pursuit chain; red follows yellow and blue follows red, each with p=0.8p=0.8.
  • One Strict-Follower: yellow and red form a hard-coupled length-2 snake; red deterministically enters yellow's previous cell.

Without a joint factor, ABC-GT approximates this coupling, but its factors still reflect the pursuit: TT updates the follower’s factor from the leader’s distribution. Pinductor’s bottleneck, in contrast, remains: each particle commits to one joint placement of all balls, so a fixed budget covers few plausible states.

Table R4. Success rate with ground-truth models (100 episodes per task).

TaskABC-GTPinductor-GT
One Follower90%44%
Two Followers89%35%
One Strict-Follower95%52%

ABC keeps its margin in all three variants, including deterministic coupling.

W3 / Q3. Could the impact of the representative-value approximation be evaluated?

To evaluate the impact of the representative-value approximation on planning performance, we compared it with MC sampling and found that MC sampling did not consistently improve success over representative values. Specifically, we replaced representative values with MC sampling of M∈{1,10,20}M \in \lbrace 1,10,20 \rbrace values per d-particle and evaluated ABC-GT over 100 episodes per task.

Table R5. Success rate (%) with representative values and MC sampling (100 episodes per task).

TaskRepresentative valueMC1MC10MC20
Unlock100100100100
MultiKeyUnlock93928988
MovingBall95949796
MultiMovingBalls93889294
DoorMaze99449697
TreasureHunt55615351
Crafter: Zombie Escape76443341
Crafter: Collect Iron43305053

Success rates are reported as percentages. MC performance generally improved with more samples; for example, success on DoorMaze increased from 44% with MC1 to 97% with MC20. MC20 and representative values achieved broadly similar success rates.

W5. Is 30 episodes per task sufficient to reliably assess performance differences?

All conclusions and performance gaps in Table 3 hold reliably when scaling up to 100 episodes per task. For generated models, these are evaluated across 10 independently generated models × 10 episodes.

Table R6. Success rate (%) on MiniGrid over 100 episodes per task.

MethodUnlockMultiKeyMovingBallMultiMovingDoorMazeTreasure
ABC-GT1009395939955
Pinductor-GT100655537501
ABC1009494815327
Pinductor99505441483
POMDP-Coder100347634118
Tabular50171204
Random00811017

Q4. Could results be reported across multiple independently generated world models?

Yes. Our results already aggregate them: following Pinductor's protocol, each task uses 10 independently generated models (paper: 10 models × 3 episodes; expanded evaluation in W5: 10 models × 10 episodes). Table R7 reports the spread across these models.

Table R7. Success rate (%), mean ± std across the 10 generated models (10 episodes each).

MethodUnlockMultiKeyMovingBallMultiMovingDoorMazeTreasure
ABC100.0 ± 0.094.0 ± 5.294.0 ± 9.781.0 ± 12.953.0 ± 49.027.0 ± 39.7
Pinductor99.0 ± 3.250.0 ± 11.554.0 ± 11.741.0 ± 19.148.0 ± 19.33.0 ± 4.8

W4. Less consistent gains on harder tasks (DoorMaze)

The smaller gain on DoorMaze stems from the LLM-generated world models, not from our belief representation. With ground-truth (GT) models, ABC-GT reaches 99% success versus 50% for Pinductor-GT (Table R6). This gap shrinks with LLM-generated models because the generated models often fall short of GT quality. Generated models that are correct already solve nearly every episode, so the remaining failures come from incorrect models. Since the belief representation, our main contribution, works well once the world model is accurate, improving LLM model generation and revision for complex tasks is an important next step.

W2 / Q2. Open-loop planning and information-gathering actions

We sincerely thank the reviewer for this suggestion. First, our main contribution is a new belief representation, d-particles, not a planner. To isolate the effect of the belief, we deliberately used a simple planner that searches over the whole belief and shared it across all methods.

Second, closed-loop planning with d-particles requires substantial additional machinery. It must simulate observations from each particle. A concrete particle renders a single observation, but a d-particle renders a distribution over observations, so it must be decoded into observations. Enumerating this distribution grows exponentially, and sampling one observation made plans unstable in our preliminary experiments.

Third, ABC is still not fully open-loop, since it replans after every actual observation. It executes only the first action, updates the ABS with the resulting observation, and replans (Section 2.5). Pinductor’s original planner also replans every step and differs only in additionally imagining observations within the search.

Fourth, our results show that the planner is not what limits ABC on our tasks. With ground-truth models, ABC fails in at most 2 of 30 episodes on five of six tasks, including none on DoorMaze, where door locks are revealed only by toggling them. On TreasureHunt, success rises from 6.7% to 76.7% with the number of particles alone (Section 4.3). A closed-loop planner would help most in tasks that require deliberate information gathering, as in TreasureHunt’s remaining failures, and we will state this as a limitation in Section 5.

Reviewer_41ADOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This submission proposes Abstract Belief Coder (ABC), which lets LLM-written code world models hold uncertainty inside each belief particle (a belief particle here is one weighted guess about the environment's hidden state).

In prior code world models, such as POMDP-Coder and Pinductor, each particle fixed every hidden variable to a single value (a "concrete" particle). ABC instead uses distributional particles (d-particles), in which each state variable holds a distribution (a factor) from a shared library that updates it exactly. With this, the authors claim that "one particle can represent many possible worlds". When these distributions stay independent, this is claimed to be a Rao-Blackwellized particle filter.

In the experiments, a lightweight frontier LLM (GPT-5.6 Luna) wrote candidate world models from offline trajectories without seeing hidden states, choosing both the programs and the distribution family of each state variable. ABC was evaluated on six MiniGrid tasks and two Crafter tasks, with both LLM-written and ground-truth (human-written) models. With LLM-written models, it succeeded more often than Pinductor and POMDP-Coder on MiniGrid (Figure 4), and in both settings more often than Pinductor on Crafter (Figure 9). With ground-truth models, a single d-particle matched or beat Pinductor with up to thirty particles (Figure 7).

Critical strengths
  • The paper automates a design choice that Rao-Blackwellized filters normally leave to a human: which hidden variables to track as exact distributions, and how to update them. The LLM makes this choice while writing the world model, and a shared library of distribution families supplies the exact updates. This makes the known benefit of Rao-Blackwellization (an accurate belief with far fewer particles when there are many hidden variables) available to LLM-written world models without problem-specific design.

  • The presented analysis separates the belief representation (how the hidden state is tracked) from planning and from model generation. With ground-truth models and identical histories, d-particles keep high probability on the true state, while Pinductor particles often lose it as hidden objects multiply, move or are revealed only indirectly (Figure 5). More proposals do not recover the true state (Figure 6), and more particles do not lift Pinductor's success above that of one d-particle (Figure 7). This points to the limitation of the belief representation in the Pinductor baseline.

  • The advantage holds with LLM-written models: ABC matches or beats both Pinductor and POMDP-Coder in every MiniGrid task (Figure 4), and it beats Pinductor on both Crafter tasks (Figure 9). This shows that ABC does not depend on human-written models or on knowing the true hidden state: an LLM that sees only observations can write models that use distributional particles well enough to outperform the baselines.

Critical weaknesses
  • Each d-particle treats its hidden variables as independent, but several tasks couple them, such as balls that block each other and door locks placed so that a route always exists (Appendix F). In such domains the filter is only an approximation, and its error is not measured. With just two small-scale domains with basic dynamics, it is hard to tell how large this error would be in larger real-world scenarios.

  • The paper's claim that "More concrete particles do not substitute for holding uncertainty inside particles" is tested against only one basic concrete filter. Pinductor's filter replaces rejected particles with fresh draws from the prior. So one wrong variable kills an otherwise good particle. Standard fixes address this failure, such as resample-move steps or factored beliefs like OO-POMCP, which the paper discusses but does not test in ground-truth experiments, where it would apply. ABC's large gains may therefore partly reflect a weak baseline.

Overall, the submission appears to be a careful combination of Rao-Blackwellized filtering with Pinductor-style LLM induction of code world models, though perhaps not a large conceptual advance. The main contribution seems to be in letting the LLM choose per-variable factor families inside a shared library. The two key open questions are whether the advantage holds against stronger concrete or factored filters, and in more varied and larger-scale domains where hidden variables are coupled.

Questions for authors

What is the justification for choosing MiniGrid and Crafter as the only domains tested? Their observations are noise-free, and they do not seem to mimic realistic dynamics well enough to draw strong conclusions. Both have been used in prior work, such as Pinductor on MiniGrid, but relying only on these domains seems quite limiting. These days, in other works I routinely see significant attempts to evaluate on at least one real-world domain, with either real data or a simulator tuned to resemble a real-world experiment. So with the current evaluation I am not yet convinced that the advance proposed in this work is significant. But I can be convinced otherwise if the authors show evaluations in more realistic scenarios.

5 comments

We thank the reviewer for the helpful and detailed review.

Q1. Why only MiniGrid and Crafter? Observations are noise-free; realistic scenarios are missing

Thanks for pointing this out. We adopt the benchmark used by prior code world models for POMDPs and extend it. The two existing methods, POMDP-Coder (Curtis et al., 2025) and Pinductor (Six et al., 2026), both use MiniGrid as their simulated benchmark. MiniGrid isolates hidden-state tracking from perception and control errors and allows direct comparison with both. We follow this protocol and extend it with Crafter, which adds richer dynamics.

To further test complex domains, we added ALFWorld (Shridhar et al., 2021), a partially observable household benchmark with long-horizon, multi-step goals. To isolate how belief tracking scales, we remove model-generation errors by giving each method a human-written model of the environment rules in its own state representation.

Table R1. Success rate on ALFWorld with ground-truth models (100 episodes per task).

TaskABC-GTPinductor-GT
Examine in Light82%61%
Pick Two & Place28%10%

ABC-GT achieves higher success on both tasks. These results extend the evaluation beyond grid worlds under ground-truth models, while broader real-world generalization remains open. We will add these ALFWorld results to the paper.

W1. Independence within a d-particle; its error is not measured

Thank you for raising this concern. We added an experiment that measures this error directly: in all settings, ABC-GT’s belief was much closer to the exact joint posterior than Pinductor-GT’s, even though Pinductor assumes no independence. Independence does ignore dependencies between variables. But each factor keeps its variable’s uncertainty, so a few d-particles cover many joint states, while each concrete particle commits to one. This coverage gain outweighs the error from the independence assumption.

We measured the error in MultiMovingBalls with 3, 4, and 5 balls, using ground-truth models for both methods to exclude model-generation errors. We computed the exact joint posterior by enumerating all states while preserving dependencies between balls, and compared each method’s belief with it under the same action–observation histories using total variation (TV) distance (0: identical, 1: no overlap).

Table R2. TV distance from the exact joint posterior. Lower is better.

Belief3 balls4 balls5 balls
ABC-GT0.1120.1700.234
Pinductor-GT1.0001.0001.000

ABC-GT’s error increased with the number of balls, but its beliefs remained closer to the exact posterior than Pinductor-GT’s in every setting. Reducing the approximation error introduced by this assumption could be a useful direction for future work.

In addition, factorized belief approximations are used across different areas, including probabilistic inference with mean-field methods (Saul et al., 1996) and robotic object search with OO-POMCP (Wandzel et al., 2019), to simplify computation.

W2. Pinductor’s filter may be a weak baseline

Thank you for your suggestions. Pinductor and POMDP-Coder are, to our knowledge, the only existing code world models that maintain a probabilistic belief over hidden states, so we compared against all prior work in this line. They also have a track record as baselines: Pinductor uses POMDP-Coder as its only code-world-model baseline, and we follow the same protocol.

Resample-move has never been applied to code world models, and applying it is a research problem beyond the scope of this paper. The existing filter samples directly from the model’s initial-state and transition programs, so their probability terms cancel in the importance weights. A Metropolis–Hastings move does not cancel them, so the LLM would have to write and validate programs for these probabilities. Developing such moves is a promising direction for future work.

OO-POMCP is not a code world model, and its belief representation is a special case of ABC’s. OO-POMCP plans with a hand-specified model, whereas ABC learns its model from data with an LLM. OO-POMCP’s belief is a product of per-object location distributions, which is a single d-particle with one factor per object. ABC extends this with mixtures of d-particles and with factor families that the LLM chooses for each variable.

W3. A careful combination of Rao–Blackwellized filtering and code world model, but perhaps not a large conceptual advance.

Thank you for this comment. We view ABC as a conceptual advance: it is the first to introduce distributional particles (d-particles), a new belief representation for code world models. Existing code world models for POMDPs (Curtis et al., 2025; Six et al., 2026) commit every hidden variable to one value per particle and lose the true state as hidden variables accumulate; a d-particle instead keeps each variable’s uncertainty within one particle.

Rao–Blackwellization was not our goal; it follows naturally from d-particles. Filtering d-particles exactly turns out to be an RBPF. Unlike standard RBPF, where which variables to marginalize is fixed by hand, ABC lets an LLM choose each variable’s distribution family together with the programs that update it.

This new representation brings large practical gains. With identical ground-truth models, ABC achieved 90.6% success versus 55.0% for Pinductor, and a single d-particle matched or exceeded 30 concrete Pinductor particles in every task (Sec. 4.3).

References.

  • Valentin Six et al. Learning POMDP World Models from Observations with Language-Model Priors. NeurIPS, 2026.
  • Aidan Curtis et al. LLM-Guided Probabilistic Program Induction for POMDP Model Estimation. CoRL, 2025.
  • Mohit Shridhar et al. ALFWorld: Aligning Text and Embodied Environments for Interactive Learning. ICLR, 2021.
  • Lawrence K. Saul et al. Mean Field Theory for Sigmoid Belief Networks. Journal of Artificial Intelligence Research, 4:61–76, 1996.
  • Arthur Wandzel et al. Multi-Object Search using Object-Oriented POMDPs. IEEE International Conference on Robotics and Automation, 2019.

Thank you for responding to each of the points raised in my review. This submission would be significantly stronger if benchmarks with observation noise were included. If a noise-free version of ALFWorld was used, then this additional evaluation was not the most informative option. The TV distance results would be useful, but they are difficult to interpret: Pinductor scored 1.000 in every setting, suggesting that something might have been misconfigured.

Furthermore, a comparison of ABC with stronger filters is still relevant and has not been addressed with a suitable experiment. It should have been feasible to identify a stronger baseline, either from my suggestions or elsewhere.

Hence, for now I would like to maintain my original rating, but I am open to a later discussion with the other reviewers regarding the evaluation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.