- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
World models let an agent predict how its environment will change and plan ahead. Most current ones are purely neural and represent the state as an uninterpretable vector, which makes them hard to inspect and hard to combine with what we already know about the world. The paper places itself in neurosymbolic learning, which combines neural networks with logical rules. It proposes Concept-Grounded World Models (CGWMs), in which the state is a set of named, human-readable facts such as "the agent is in this cell" or "this cell is a wall". A neural network learns to read these facts from images. How the world changes is not learned: it is written by hand as logical rules about how the facts evolve over time.
The network is trained with very little labelled data. It sees only a few yes/no labels per episode, such as "the agent eventually reached the goal", and must otherwise make its readings consistent with the hand-written rules. To make this trainable, the rules are scored with a soft, differentiable form of logic, and two training tricks help avoid degenerate solutions. Planning uses the same soft logic: a sequence of actions is adjusted by gradient ascent until the rules predict that the goal is reached. On two small grid-world puzzles, the method reads the facts almost perfectly, even with only 50 training episodes, and stays accurate over long imagined rollouts. It also plans much more successfully than a purely neural world model, including on a new layout and on a variant whose rules were rewritten by hand. A second finding is that different soft logics work best for learning and for planning.
Critical strengths
- Concepts are recognised almost perfectly from a few labels per episode, and the ablations show which training choices make this work. Trained with the product form of fuzzy logic, CGWM recognises the concepts with F1 of 0.98–1.00, even from 50 training trajectories (Tables 6–7). Both training tricks are needed. In Two Rooms with 3200 trajectories, F1 drops to 0.21 without blocking gradients through the "if" part of each rule, and to 0.51 without the entropy term (Table 6). Training with logic-based losses often gets stuck in solutions that satisfy the rules trivially, for example by predicting that the agent is in no cell. This is concrete guidance for anyone training models this way.
- The best fuzzy logic for learning differs from the best one for planning, and one model can use both (RQ4). With the min/max (Gödel) form of fuzzy logic the encoder fails to learn (F1 about 0.2–0.4, Tables 6–7), while the product form succeeds. For planning, the noisy min/max variant solves more instances than the product form, for example 88.4% versus 68.6% on Two Rooms (lava dense) (Tables 1, 8). The same trained encoder and the same rules serve both phases, so the logic can be chosen separately for each. The finding is clearly supported and useful to others.
Critical weaknesses
- CGWM's advantage is largely guaranteed by the setup. CGWM is given the environment's complete and exact transition rules (Tables 4–5). Once it reads the first frame correctly, its predictions must stay correct at every later step, as Sec. 5 itself notes, so the long-horizon accuracy result (RQ2) does not test the method. On the first frame alone, a linear probe on the neural baseline (LeWM) is as accurate as CGWM, even with 50 training trajectories (F1 = 1.00, Tables 6–7). The data-efficiency advantage claimed for RQ1 therefore comes entirely from CGWM not having to learn the dynamics. Likewise, the "zero-shot transfer" to the periodic-door variant (Table 1) consists of a person writing new rules (Table 5), while LeWM gets no new data or retraining. LeWM's planner was also replaced: its actions are optimised by gradient descent over the four discrete moves instead of by its original sampling-based planner (App. D.2). It then solves only 37.8% of Two Rooms (medium) instances, which suggests the replacement planner, not LeWM's representation, may be what limits it. The claims of "significantly more robust planning" and better data efficiency therefore mostly measure the knowledge handed to CGWM, not what it learns.
- There is no comparison with a standard search-based planner, which would solve every instance in this setting. The dynamics are known, deterministic and fully observable. Reading the first frame and then running breadth-first search over the rules would find an optimal plan for every instance; the authors themselves use breadth-first search to compute the shortest plans (App. D.3). CGWM's gradient-based planner still fails on 3.7–31.4% of instances in the harder Two Rooms variants (Table 1, Fig. 6). Its only justification is a future extension to continuous variables (Sec. 3.4, 7), which is not tested. Separately, the encoder classifies each grid cell from that cell's 8×8 pixels alone, and each object is a fixed, distinct sprite (App. D.1), so recognising concepts here is an easy problem. There is no experiment with varied visuals, objects spanning several cells, or incomplete rules, so the paper does not show that the approach works beyond these grid puzzles. This matters for the paper's overall argument, because the introduction motivates the work by robotics and AI in the physical world.
Questions for authors
- (Weakness 1: given versus learned dynamics) Can you add a variant of CGWM that keeps the same encoder, labels and constraints but learns its transition model from data instead of receiving the rules? Suppose this variant still recognises concepts over long rollouts and plans clearly better than the neural baseline. Then the advantage would come from the logic-based supervision rather than from the hand-written dynamics, and the first weakness would largely fall away.
- (Weakness 1: modified baseline planner) How does the neural baseline (LeWM) perform on Two Rooms with its original sampling-based planner instead of the gradient-based one used here? If its success rate rises well above the reported 22–38% on the medium and hard variants, much of the reported planning gap reflects the change to the baseline rather than a difference between the world models.
- (Weakness 2: no search-based planner) How does CGWM's gradient-based planner compare with reading the first frame and then running breadth-first search over the same rules? If search solves every instance, what does gradient-based planning offer in this paper's setting? A concrete case where search does not apply but the gradient planner works would answer this.
- (Weakness 2: dependence on complete hand-written knowledge) Does grounding still work when the knowledge is reduced or imperfect? Examples: only the transition rules and per-episode labels without the extra constraints (Tables 3–4), one transition rule removed or wrong, or cells whose appearance varies. If grounding collapses in these cases, the method needs a complete, hand-written simulator of the environment. That would undercut the "weak supervision" claim and the motivation from real-world settings.
9 comments
Answering question 2. The CEM Planner for LeWM usually performs worse. We had evaluated the LeWM baseline with the CEM optimization algorithm before submission. It attains low planning performance on most settings. We therefore evaluated the spatial LeWM variant with reward-based differentiable planning described in the paper.
The table below reports the planning performance of LeWM with CEM using both its original vector representation and the spatial representation used in our baseline. We include the spatial variant with gradient-based planning for comparison:
| Environment | Vector LeWM + CEM | Spatial LeWM + CEM | Spatial LeWM + gradient planning |
|---|---|---|---|
| Two Rooms (easy) | 81.3 (3.7) | 88.0 (2.5) | 95.8 (1.2) |
| Two Rooms (medium) | 0.0 (0.0) | 0.0 (0.0) | 37.8 (2.8) |
| Two Rooms (hard) | 1.1 (1.0) | 30.6 (6.4) | 22.0 (9.0) |
| Two Rooms (lava dense) | 0.0 (0.0) | 6.4 (3.2) | 9.6 (6.9) |
| Two Rooms (periodic door) | 0.5 (0.7) | 9.1 (5.2) | 7.2 (3.6) |
| Ice Slider (T=20) | 0.4 (0.5) | 50.4 (6.4) | 69.1 (12.6) |
| Ice Slider (T=T*) | 0.0 (0.0) | 0.2 (0.4) | 3.6 (2.9) |
Values are the mean percentage of solved instances over 10 seeds, with standard deviation in parentheses. Our planner outperforms the CEM planner in all but two rows, strengthening the baseline.
We will include the full CEM results in the camera ready.
To address the concern expressed in Weakness 1 about transfer to periodic-door, we also fine-tuned both LeWM variants on trajectories collected from that environment, starting from the model trained on Two Rooms (hard). We report the obtained planning results:
| Fine-tuned model | Planner | Solved instances (%) |
|---|---|---|
| Spatial LeWM | Gradient | 7.9 (3.6) |
| Spatial LeWM | CEM | 15.8 (4.3) |
| Vector LeWM | CEM | 0.5 (0.5) |
Values are the mean percentage of solved instances over 10 seeds, and standard deviation in parentheses. Fine-tuning improves planning performance for spatial LeWM, but both variants remain substantially below CGWM. We will include these results in the revised paper.
Answering question 3. BFS finds a shortest plan for every solvable instance. Given a correctly decoded initial state and complete, correct, deterministic transition rules, BFS returns a shortest plan for every solvable instance in these environments. Indeed, we already use BFS to compute optimal plan lengths (Appendix D.3). We now report CGWM results with BFS, extending Table 1.
The contribution of our differentiable planning procedure is to demonstrate that CGWM supports learning, prediction, and planning within the same differentiable reasoning framework. Although introduced to learn grounded world models, CGWM naturally supports action optimization through its own transition model: the same differentiable machinery used during learning can propagate goal-satisfaction gradients to action sequences during planning.
This reuse makes it valuable to investigate how the framework behaves across these phases. Our experiments show that the fuzzy semantics that performs best for learning the grounding differs from the one that performs best for planning, providing practical insight into the choice of semantics for differentiable planning. These findings can also inform the design of other differentiable neurosymbolic planners, including those operating over continuous action spaces (which standard BFS cannot directly enumerate).
We will add the BFS results and clarify both CGWM’s compatibility with classical planners and the contribution of demonstrating and studying planning within its differentiable framework in the revised paper.
| Two Rooms (easy) | Two Rooms (medium) | Two Rooms (hard) | Two Rooms (lava dense) | Two Rooms (periodic door) | Ice Slider (T=20) | Ice Slider (T=T)* | |
|---|---|---|---|---|---|---|---|
| CGWM (Godel Trick) | 100.0 (0.0) | 100.0 (0.0) | 96.3 (0.8) | 88.4 (2.8) | 94.9 (1.0) | 100.0 (0.0) | 100.0 (0.0) |
| CGWM (log-product) | 100.0 (0.0) | 100.0 (0.0) | 90.4 (1.9) | 68.6 (5.5) | 84.3 (3.3) | 99.8 (0.4) | 90.4 (1.7) |
| CGWM (BFS) | 100.0 (0.0) | 100.0 (0.0) | 100.0 (0.0) | 100.0 (0.0) | 100.0 (0.0) | 100.0 (0.0) | 100.0 (0.0) |
| LeWM | 95.8 (1.2) | 37.8 (2.8) | 22.0 (9.0) | 9.6 (6.9) | 7.2 (3.6) | 69.1 (12.6) | 3.6 (2.9) |
We thank reviewer 5qgC for the careful assessment and constructive suggestions. We have conducted additional experiments to test learning and planning when the explicitly provided knowledge is incomplete and incorrect. We address each question below.
Answering question 1.
Yes, CGWM can still ground and plan well under incomplete knowledge.
The reviewer suggests learning the transition dynamics from data. As a step in this direction, we introduced two variants of the Two Rooms (hard) setting, maintaining the concept vocabulary while introducing transitions that are absent from, or contradict, the explicitly provided transition knowledge:
In both environments we retain the original symbolic predictor, whose rules state that walls block the movement, lava is lethal, and button opens the door. These rules represent a reasonable initial model based on common-sense knowledge, but now make incorrect predictions about the transitions.
We add a residual predictor that corrects the symbolic predictor’s output and we train it on trajectories collected from each modified environment. We consider two settings: in the first, we freeze the encoder pretrained in the original Two Rooms (hard) environment and train only the residual predictor; in the second, we train the encoder and residual predictor jointly from scratch. The original symbolic rules remain unchanged in both settings. The target of the residual predictor is the encoding of the subsequent observation. The new transition knowledge and corrections to the incorrect rules are thus learned from data rather than provided explicitly. The corrected next state prediction is
z^t+1=Pred(zt,at)+δt, where Pred(zt,at) denotes the symbolic predictor’s output and δt the correction provided by the residual predictor. During planning, both the encoder and residual predictor are frozen, and the actions are optimized with the same differentiable procedure. We obtain the following results:
Values are the average percentage of solved instances over 10 seeds, with standard deviation in parentheses.
The high success rates obtained with the pretrained frozen encoder should also be interpreted in light of the changed tasks. Once the new transitions have been learned, jumping over the wall and safely stepping on lava can shorten the routes to the goal, and simplify planning compared to the original Two Rooms (hard) setting.
With the encoder jointly trained from scratch, planning success remains high on Lava-Door but is lower on Wall-Jump. The reason can be attributed to the grounding performance, which will be discussed in answering question 4. The Wall-Jump model grounds walls imperfectly, whereas the concept not grounded in Lava-Door is the button, which has no effect in the modified environment. These experiments show that CGWM can learn to correct incorrect transition knowledge and use the corrected predictions for planning without manually rewriting its symbolic rules.