- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes NS-WM, a neural-symbolic framework for world modeling under partial observability, with a focus on compositional generalization across sensor views and operating regimes. The main idea is to separate view-dependent observation mappings from shared state-action-regime dynamics, so that previously seen views and regimes can be recombined at test time.
The framework includes two prediction routes. One identifies sparse dynamics and sensor mappings in reference-sensor coordinates, and estimates the current state by fitting the observed history. The other uses a recurrent encoder to infer a latent state and learns symbolic dynamics in that space. Their predictions are combined using validation-selected weights. The paper also introduces additional forecasting experts for heterogeneous sequence datasets.
Experiments are conducted on six benchmark suites, including a controlled pendulum system and several time-series datasets. The results show improvements over neural world model and direct forecasting baselines on most evaluated settings. The authors also provide diagnostics on recovered dynamics, action effects, and state-estimation failures.
Critical strengths
Separating the dynamics from the observation model seems like a sensible way to approach compositional generalization, especially when the same physical system can be observed through different sensors. I also think the emphasis on state estimation under partial observability is important. A shared dynamics model is not very useful if it cannot be initialized reliably from the available observations, and the paper makes a reasonable effort to address this issue.
The controlled pendulum experiments are interesting, particularly the recovery of interpretable dynamics and the evaluation under alternative action plans. I also appreciate the analysis of failure cases. The state-aliasing example, where different state estimates explain the observed history almost equally well but lead to very different future predictions, is quite informative. It is good that the authors discuss this issue and acknowledge that the fusion strategy does not always help.
Critical weaknesses
My biggest concern is that the full NS-WM framework does not seem to offer a clear advantage over its identification component. In fact, the results in Appendix Table 4 are rather surprising. On the main controlled pendulum setting, the identification-only model achieves much lower MSE than the final mixture on IID, combination-OOD, and parameter-OOD evaluations. The difference is quite large, especially for combination-OOD, where the identification model obtains an MSE of 0.000430 compared to 0.07971 for the final model.
This makes me question how much the neural-symbolic encoder route and the fusion mechanism actually contribute. The paper does acknowledge that identification alone performs better on these test splits, but I think this deserves more attention given that the combination of these components is central to the proposed framework. Why should we prefer the full model over the simpler identification approach? Are there settings where the neural-symbolic route clearly helps, rather than introducing additional prediction errors? I would have liked to see a more systematic comparison of the individual components across the benchmarks.
Related to this, the identification route uses fairly strong structural priors. In the controlled pendulum experiments, the first two reference-sensor channels define the state coordinates, and the prescribed dynamics library already contains the relevant terms needed to represent the underlying system. The method also uses paired multi-view observations during training. These are reasonable assumptions for a structured system identification approach, but they make the problem considerably easier than general world modeling from partial observations. I am not fully convinced that the experiments demonstrate how well the approach works when the physical coordinates are not readily available, or when the true dynamics are not well represented by the predefined library. Since compositional generalization is the main motivation, I think it would be useful to understand how much of the improvement comes from these priors.
I also found the experimental comparisons somewhat difficult to interpret. Although the paper reports results across six benchmark suites, the actual prediction mechanisms differ across them. The controlled systems use the identified dynamics and history-based state estimation, while some heterogeneous sequence tasks rely on additional direct multi-horizon ridge forecasting experts. These are rather different mechanisms, and I am not sure that their results should be interpreted as evidence for the same neural-symbolic world modeling approach. It would help to separate the conclusions supported by the controlled experiments from those supported by the more general forecasting experiments.
The baseline comparison is another concern. The final NS-WM model involves multiple components, additional structural information, model selection, and in some settings an ensemble of symbolic predictors. In contrast, the reported neural and direct forecasting baselines use different architectures and computational budgets. The authors are transparent about this, but it is still difficult to tell which parts of the performance gain are due to the proposed modeling principles. In particular, a stronger comparison against a SINDYc-style model with history-based state estimation, or other factorized state-space models with comparable supervision, would make the results more convincing.
Finally, the statistical evidence seems a little limited given the strength of some of the generalization claims. The main baselines are averaged over three model seeds, while NS-WM is reported as a single selected composite. The additional pendulum population is also used during development, and each held-out pairing has its own trained and selected model. I understand that running the full pipeline repeatedly may be expensive, but some estimate of its variability would be helpful. I would be cautious about drawing strong conclusions from the overall 13/14 result without a more consistent evaluation protocol.
Questions for authors
-
The identification-only model performs substantially better than the final NS-WM mixture in the main controlled experiments (Appendix Table 4). Could the authors explain why the fusion strategy causes such a large degradation? More importantly, in which settings does the neural-symbolic encoder route provide a clear advantage over identification alone? A component-wise comparison across more benchmarks would be very helpful.
-
How sensitive is NS-WM to the choice of reference-state coordinates and the predefined dynamics library? The pendulum experiments seem to benefit from a representation that is already close to the true physical state and a library containing the correct functional terms. I would be interested in seeing what happens when these assumptions are relaxed, for example by using a less informative reference view or removing some of the relevant candidate functions.
-
Could the authors clarify which components of NS-WM are actually used in each benchmark suite? In particular, how much of the performance on the heterogeneous sequence datasets comes from the symbolic dynamics model, and how much comes from the additional direct forecasting experts? A clearer component-level comparison would help establish whether the same underlying mechanism is responsible for the improvements across tasks.
-
Would it be possible to include a more directly comparable structured baseline, such as sparse system identification combined with history-based state estimation, under similar supervision and computational budgets? I think this would be more informative than comparing only against the neural world model and direct forecaster, especially given how strong the identification component already is.
-
Could the authors provide some measure of variability for the complete NS-WM pipeline, rather than comparing one selected composite with the average of three baseline seeds? It would also help to clarify how much model selection and validation tuning contribute to the reported results.
22 comments
Tables for Response to W1 & Q1 (1/3) (Due to character limits, the reply will be split into multiple messages. Thank you for your understanding.)
Tables for Response to W1 & Q1 (1/3)
Table F.1 (Appendix F) — Component MSE in the three primary controlled conditions (extension of Table 4). Identification = identification route alone; Symbolic ensemble = K=3 encoder-route ensemble alone; Final mixture = the validation-selected S of Table 2. "Validation" is validation macro-domain MSE, the selection quantity. Skill = combination-OOD action-effect skill. Bold = best component in each row (values within 1% are all bold). W_enc lists the fitted encoder weights (rows: views 0–2; columns: channels 0–2). Forecast targets carry sensor noise with standard deviation 0.02 (§4.1), i.e., a noise variance of 4.0e−4.
| Condition | Split | Identification | Symbolic ensemble | Final mixture |
|---|---|---|---|---|
| (2,1.0) | Validation | 0.2200 | 1.114 | 0.1823 |
| IID | 0.000498 | 0.13833 | 0.04035 | |
| Comb-OOD | 0.000430 | 0.09302 | 0.07971 | |
| Param-OOD | 0.000419 | 0.18415 | 0.16849 | |
| Comb-OOD skill | 0.9514 | 0.9219 | 0.9432 | |
| W_enc | [[0, .0067, 0], [0, 0, 0], [1, .0069, 0]] | |||
| (0,0.7) | Validation | 0.4481 | 0.4063 | 0.1692 |
| IID | 0.000430 | 0.0825 | 0.0354 | |
| Comb-OOD | 0.000410 | 0.0218 | 0.000410 | |
| Param-OOD | 0.000417 | 0.0884 | 0.0331 | |
| Comb-OOD skill | 0.9514 | 0.8925 | 0.9514 | |
| W_enc | [[0, .0004, 0], [.0004, 0, 0], [1, 0, 0]] | |||
| (1,1.0) | Validation | 0.4397 | 2.631 | 0.2737 |
| IID | 0.2526 | 0.0966 | 0.0529 | |
| Comb-OOD | 0.000453 | 0.0270 | 0.000453 | |
| Param-OOD | 0.000419 | 0.0251 | 0.0036 | |
| Comb-OOD skill | 0.9622 | 0.9469 | 0.9622 | |
| W_enc | [[0, .0031, 0], [0, 0, 0], [1, .0016, 0]] |
Table F.2 (Appendix F) — Periodic state aliasing and the fitted weight on view 2 / channel 0. Aliased windows are windows whose history-based state estimate is offset by ±2π (squared error ≈ (2π)² ≈ 39.5), detected post hoc by checking whether a 2π shift of the estimate reduces the forecast error; this diagnostic uses future targets and plays no role in inference or selection. "Share" = fraction of the identification route's squared error from aliased windows. The last column refits the weight of Eq. (16) without the aliased validation windows (diagnostic only). In test splits that include view 2, view 2 / channel 0 accounts for 89.7–99.8% of the squared error of S (98.8–99.8% in the main condition).
| Condition | Split | Aliased windows (episodes) | Share of identification squared error | View-2/ch-0 MSE on aliased samples: identification → encoder | Fitted w(view 2, ch 0) | Refit without aliased windows |
|---|---|---|---|---|---|---|
| (2,1.0) | Validation | 24 (1) | 99.8% | 39.5 → 31.5 | 1.0 | 0.0023 |
| IID / Comb-OOD / Param-OOD | 0 / 0 / 0 | — | — | |||
| (0,0.7) | Validation | 34 (3) | 99.9% | 39.5 → 12.6 | 1.0 | 0.0008 |
| IID / Comb-OOD / Param-OOD | 0 / 0 / 0 | — | — | |||
| (1,1.0) | Validation | 40 (2) | 99.9% | 39.5 → 22.9 | 1.0 | 0.0025 |
| IID | 23 (2) | 99.8% | 39.5 → 7.97 | |||
| Comb-OOD / Param-OOD | 0 / 0 | — | — |
Table F.3 (Appendix F) — Trained models on the test episodes of Table F.24 (30 episodes per regime; IID spans two regimes). Nothing is refit or selected, and S reproduces Table F.24. The last column gives the same ratio on the primary test splits (Table F.1). Aliased windows appear in all seven splits that include view 2. S is below both K=3 ensembles in 9/9 cells; identification alone is below both in 8/9. Bold = lower value (both if within 1%).
| Condition | Split (episodes) | Aliased episodes | Identification | S | S / Identification | Same ratio on the paper's split |
|---|---|---|---|---|---|---|
| (2,1.0) | IID (60) | 1 | 0.0206 | 0.0449 | 2.2 | 81 |
| Comb-OOD (30) | 6 | 1.095 | 0.990 | 0.90 | 185 | |
| Param-OOD (30) | 1 | 0.0248 | 0.0935 | 3.8 | 402 | |
| (0,0.7) | IID (60) | 4 | 0.1211 | 0.1189 | 0.98 | 82 |
| Comb-OOD (30) | 0 | 4.12e−4 | 4.12e−4 | 1.00 | 1.00 | |
| Param-OOD (30) | 1 | 0.0248 | 0.0426 | 1.7 | 79 | |
| (1,1.0) | IID (60) | 5 | 0.1265 | 0.1055 | 0.83 | 0.21 |
| Comb-OOD (30) | 0 | 5.87e−4 | 5.87e−4 | 1.00 | 1.00 | |
| Param-OOD (30) | 1 | 0.0248 | 0.0145 | 0.58 | 8.6 |
Tables for Response to W1 & Q1 (2/3) (Due to character limits, the reply will be split into multiple messages. Thank you for your understanding.)
Tables for Response to W1 & Q1 (2/3)
Table F.4 (Appendix F) — New data seeds with larger validation and test sets. Three new data seeds × three conditions = 9 runs; all NS-WM components rebuilt with the paper's per-condition training settings (identification route refit, three encoder models retrained, fusion weights refit on validation); 14 training episodes per regime as in the paper, 20 validation and test episodes per regime; selection rule unchanged. Entries are seed means (validation macro-domain MSE in "Validation" rows, test MSE otherwise); "Change" is relative to identification alone. Over the 9 runs: validation contains aliased windows in 9/9, and the fitted w(view 2, ch. 0) ranges from 0.25 to 1.0 (all other weights ≤ 0.011); 17 of 27 test splits contain aliasing (IID 9/9), and S is lower on 14 of them, including all nine IID splits (16–76% per run). Conditions sharing a data seed reuse evaluation episodes where their cells overlap, so counts over runs are paired. N and D are not part of this experiment (see Table F.24). Absolute errors exceed those in Table 4 because aliased episodes dominate the squared error.
| Condition | Split | Aliased episodes (per seed) | Identification | S | Change |
|---|---|---|---|---|---|
| (2,1.0) | Validation | 2 / 3 / 2 of 40 | 0.0606 | 0.0478 | −21.2% |
| IID | 6 / 5 / 4 of 40 | 0.1741 | 0.1317 | −24.3% | |
| Comb-OOD | 3 / 0 / 2 of 20 | 0.3476 | 0.3038 | −12.6% | |
| Param-OOD | 1 / 1 / 0 of 20 | 0.02632 | 0.02645 | +0.5% | |
| (0,0.7) | Validation | 2 / 5 / 6 of 40 | 0.2242 | 0.1736 | −22.6% |
| IID | 5 / 4 / 8 of 40 | 0.2115 | 0.1391 | −34.2% | |
| Comb-OOD | 0 / 0 / 0 of 20 | 4.10e−4 | 4.19e−4 | +2.0% | |
| Param-OOD | 1 / 1 / 0 of 20 | 0.02631 | 0.02667 | +1.4% | |
| (1,1.0) | Validation | 6 / 4 / 5 of 40 | 0.1269 | 0.0963 | −24.1% |
| IID | 7 / 7 / 4 of 40 | 0.2097 | 0.1414 | −32.6% | |
| Comb-OOD | 0 / 0 / 0 of 20 | 6.17e−4 | 5.79e−4 | −6.3% | |
| Param-OOD | 1 / 1 / 0 of 20 | 0.02632 | 0.01371 | −47.9% |
Table F.5 (Appendix F) — Mean test MSE over the nine controlled test splits. Three conditions × {IID, combination-OOD, parameter-OOD}, equal weights. For the new data seeds, the nine cells are the seed means of Table F.4; the last column gives the change for each seed.
| Evaluation set | Test episodes per split | Identification | S | Change | Per seed |
|---|---|---|---|---|---|
| Primary test splits (Table F.1) | 5–10 | 0.0285 | 0.0460 | +61.8% | — |
| Test episodes of Table F.24 (Table F.3) | 30–60 | 0.160 | 0.157 | −1.9% | — |
| New data seeds (Table F.4) | 20–40 | 0.114 | 0.087 | −23.4% | −23.3% / −33.6% / −17.4% |
Table F.7 (Appendix F) — Route-level ablations over the 12 controlled endpoints. Worst-case ratio = maximum of MSE / min(N-ens, D-ens) over the nine MSE endpoints; a value below 1 means that no MSE endpoint is worse than the stronger K=3 baseline. Dense library refits the identification route without sequential thresholding. All variants are refit on training data; no test-based selection is used.
| Configuration | Endpoints beating both K=3 ensembles | Worst-case MSE ratio |
|---|---|---|
| Full NS-WM (validation-selected S) | 12/12 | 0.815 |
| Identification route only | 11/12 | 2.713 |
| Encoder route only | 4/12 | 1.234 |
| Identification route, dense library | 5/12 | 52.19 |
| Identification route, no history fitting | 3/12 | 68.08 |
Table F.20 (Appendix F) — Component decomposition on the heterogeneous suites (15 endpoints, inference only). Entries count endpoints with lower test MSE than both K=3 ensembles. The identification route does not apply to these suites (§3.5). The three endpoints where S does not win are those already reported in Table 2. On the IID endpoint, S is lower than the symbolic ensemble alone, the expert alone and both K=3 ensembles in all five suites (e.g., Dysts IID: S 0.719; symbolic ensemble alone 0.755; expert alone 4.28; N-ens / D-ens 1.08 / 1.07).
| Suite (endpoints) | Symbolic ensemble alone | Expert alone | S (combination) |
|---|---|---|---|
| Application (3) | 3 | 2 | 3 |
| Dysts (3) | 1 | 0 | 1 |
| TEP (3) | 2 | 2 | 3 |
| V2-all (3) | 0 | 2 | 3 |
| V2-synthetic (3) | 1 | 1 | 2 |
| Total | 7/15 | 7/15 | 12/15 |
Response to W1 & Q1 (Due to character limits, the reply will be split into multiple messages. Thank you for your understanding.)
We thank the reviewer for raising this central point and agree that it deserves more attention. The analysis is in Appendix F.1 (Tables F.1–F.5).
(1) Why the gap arises. The forecast targets carry sensor noise with variance 4.0e−4 (§4.1), and identification alone reaches 4.2–5.0e−4 on the main test splits, i.e., the noise level; S still beats both K=3 ensembles there (0.0797 vs. 0.258 and 0.103 on combination-OOD; Table F.23). View 2 measures (θ − ω, ω, cos θ) with the first channel hidden, and the visible channels and dynamics are 2π-periodic, so θ and θ + 2π give the same visible history (§4.4, Appendix D). A few aliased validation windows carry 99.8–99.9% of the identification route's validation error and set the static encoder weight on view 2 / channel 0 to 1, while the other eight weights stay at most 0.007 (Eq. 18; Tables F.1–F.2). On combination-OOD splits whose held-out view is not view 2, S equals identification alone.
(2) Settings where the encoder route helps.
(3) Systematic component comparison. The comparison covers Table F.1 (three controlled conditions: identification alone, symbolic encoder ensemble alone, S, validation, skill and fitted weights), Tables F.3–F.5 (larger evaluation sets), Table F.20 (heterogeneous suites) and Table F.7 (route-level ablations: S 12/12 against both ensembles with worst-case ratio 0.815, identification alone 11/12 with 2.713, encoder route alone 4/12 with 1.234). They show that on combination-OOD splits whose held-out view is not view 2, S coincides with identification; where aliasing occurs, S reduces the error on most splits; and without reference coordinates, the combination outperforms each component.
(4) Why prefer the full model. S is best on 24/27 primary endpoints against both the K=3 ensembles and a validation-selected baseline system (Table F.23), while identification alone applies to only 12 of them. Identification alone is a special case of S (w = 0 in Eq. 7), and the paper's selection criterion prefers S ("Validation selection reduces macro-domain MSE by approximately 17% relative to identification alone", §4.4; 21–24% lower validation MSE with new data seeds, Table F.4). S is the only configuration below both baselines on every controlled endpoint (Table F.7).
In short, the Table 4 gap comes from one validation-fitted fusion weight that responds to 2π aliasing in a few validation windows; where such aliasing occurs at test time, S is generally more accurate (14 of the 17 aliased splits with new data seeds, including all nine IID splits), and S is the only configuration that beats both K=3 ensembles on all 12 controlled endpoints (Table F.7).
Revisions in the paper (blue): §4.4 "Component tradeoffs" is rewritten along these lines; §3.4 states that w = 0 recovers the identification route; the Abstract and §5 state when each route is most accurate; the Table 4 caption is revised; new Appendix F.1 (Tables F.1–F.5) and Appendix F.5 (Table F.20).
This question also prompted the cross-benchmark component comparison of Appendix F.1, for which we are grateful.