- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper studies the decision problems when the dynamics structure is known. By learning a small amount of parameters within the true model structure, the proposed method can achieve better performance with less data compared to those baselines with no structural prior knowledge.
Critical strengths
- The paper is a relatively complete study, where the problem definition, method, experiment, theory, and the additional informations in the appendix are well-presented. These give me a whole picture of the method, and how it works in practice.
Critical weaknesses
- My main concern is the novelty. This approach (use a known dynamics structure, and only learn the corresponding parameters within) is called system/model identification in some literatures. I think this is kind of well-understood by the academia: With structural bias, the model learning will surely be more efficient, but when the assumed structure is not accurate, it will result in large model error. So for me this work delivers limited new insight.
- The applicability is another problem. This kind of approach is very useful when there is a simple and clear dynamics structure. However, when the system is complex, like multi-agent system, where we need to predict the behavior of other agents, there is hardly any useful structure to impose, and the bottleneck of learning will still be the non-structural part.
Questions for authors
- What new methodological or conceptual contribution does the proposed method offer, compared with standard system identification works?
4 comments
Response
Thank you for the response. I understand that the paper’s core contribution is a procedure for converting existing simulation models into world models suitable for planning. However, I still find the ideas and techniques straightforward and largely consistent with standard engineering practice. The procedure is described at a high level, while its implementation appears to rely heavily on manual, task-specific modeling choices. The rebuttal has therefore not resolved my concerns about the methodological contribution, and my score remains unchanged.
Follow-up Response to Reviewer KTPX
We thank Reviewer KTPX for taking the time to read our response and clarify the remaining concern.
Regarding manual, task-specific choices and applicability to multi-agent systems (W2, where we noted hierarchical structured world models as an ongoing direction), we share a brief preliminary follow-up check. Because multi-agent hierarchical coordination goes beyond the single-level scope of the current manuscript, we do not add new claims to the revised paper, but report the comparison here in case it is informative for the discussion.
Using the same construction principle from Section 4.1—preserving conservation/kinematic structure and delay/contact memory in the state interface while restricting learning to unresolved bottleneck closures—we instantiated a two-level (macro-micro) SD world model (SD-WM) across two distinct multi-agent testbeds:
- Discrete-event fleet coordination (10–50 warehouse robots with queueing and path-blocking, 6 regimes × 5 blind seeds);
- Continuous multi-joint contact control (MAMuJoCo
Hopper 3x1, 3 regimes × 5 blind seeds).
To eliminate task-specific planner confounding, all methods below share the exact same rule-free candidate grid, decision cadence, and model-free Greedy fallback (Candidate 0), and are trained on the exact same multi-regime offline datasets (Warehouse: 60 trajectories covering all 6 regimes—30 base and 6 per non-base regime including 50-AGV, sampled uniformly across regimes; Hopper: 48 trajectories covering all 3 regimes—24 base, 12 1.15g, and 12 goal_only). To test whether the learning bottleneck in multi-agent systems lies mainly in the unstructured part (W2), we compared SD-WM against an unstructured neural transition model (Pure-BB-WM), the implicit latent-value model TD-MPC2 [1], and a state-space adaptation of the recent end-to-end hierarchical latent world model H-JEPA [2] (retaining its two-level temporal predictors, SIGReg anti-collapse regularization, inverse dynamics, and top-down subgoal projection), all trained for 3,000 updates to convergence (3.75×–15× our update budget):
Method (shared planner & Greedy Cand. 0) | WH Params / Steps | WH 10-AGV (base) | WH 50-AGV (high_load) | WH 6-Case Mean | Hopper Params / Steps | Hopper 1.15g | Hopper 3-Case Mean |
|---|---|---|---|---|---|---|---|
Fixed + Greedy (no world model) | 0 / 0 | 229.7 | -69.0 | 189.6 | 0 / 0 | 630.8 | 668.1 |
Pure-BB-WM (converged) | 23.7k / 3000 | 228.2 | -250.2 | 162.2 | 24.7k / 3000 | 562.1 | 630.0 |
TD-MPC2 [1] (converged) | 50.4k / 3000 | 173.3 | 229.5 | 241.9 | 51.1k / 3000 | 583.6 | 644.0 |
H-JEPA [2] (converged) | 33.3k / 3000 | 209.7 | -394.2 | 36.7 | 28.1k / 3000 | 572.1 | 583.9 |
SD-WM (Ours) | 13.1k / 800 | 227.1 | 452.7 | 324.7 | 17.5k / 200 | 751.4 | 751.0 |
Two observations directly speak to W2 and the role of the structured interface:
- In low-coupling nominal regimes (e.g., 10-robot
base), converged black-box models matchSD-WM(228.2vs.227.1), confirming that the unstructured baselines fit passive in-distribution transitions well. - Under emergent multi-agent bottlenecks (50-robot congestion or
1.15gcontact overload), even with target-regime trajectories in the training set, converged unstructured and latent models misrank counterfactual macro actions not covered by healthy offline rollouts (triggering backlog penalties or fleet gridlock in50-AGV, often falling belowFixed + Greedy), whereas retaining stock/physical conservation and explicit delay/contact closure (SD-WM) yields consistent gains (+82.8to+135.1over the best baseline/heuristic across both domains) at a fraction of the update budget.
We appreciate the reviewer's candid feedback, which has helped sharpen both the boundary of the current paper and this multi-agent extension.
References
[1] Hansen, Su, and Wang. TD-MPC2: Scalable, Robust World Models for Continuous Control. ICLR, 2024.
[2] Zhang, Terver, Rabbat, LeCun, and Balestriero. H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planning. arXiv:2610.06805, Oct. 2026.
Thank you for providing these additional experiments. I agree that partial structural knowledge can be beneficial even in complex settings compared with having no structural prior.
To clarify my earlier comment, my motivation for mentioning multi-agent systems was to highlight settings where reliable structural knowledge is limited. For example, other agents may act independently, without a reliable model of their behavior being available. Learning their behavior from data may then remain the dominant bottleneck in the unstructured component. From the description provided, it remains unclear whether the additional tasks capture this situation, so I do not view the results as directly addressing this particular concern.
Overall, the additional experiments strengthen the discussion of applicability and help me better understand the paper. However, my main concern about the methodological novelty remains unresolved, and my rating remains unchanged.
Response to Reviewer KTPX
We thank the reviewer for recognizing the completeness of this study. The setups and full results of the supplementary experiments R1 and R2 referred to below are given in the General Response.
W1. Relative to system identification, what new insight does this paper provide?
We agree that "structural priors improve sample efficiency" is already known; once the structural model and the state–input interface are given, the parameter learning in our paper is also structured system identification.
A standard SD model usually embeds decision rules and keeps intertemporal effects in delay and pipeline variables. Our paper focuses on constructing an existing SD model into a world model for planning: Section 4.1 and Appendix A.2 give a six-step procedure for action externalization and dynamic-memory retention, together with a state-transition completeness criterion. Section 4.2 and Appendix B.6 discuss residual correction and error propagation under structural mismatch. The supplementary experiments test these construction requirements and the effect of residual correction:
Therefore, under this controlled mismatch, the structural prior still retains its planning benefit with limited data, and residual learning further reduces the loss that parameter identification alone does not remove.
W2. What if a complex system lacks a clear structure?
The applicability of our method depends on whether reliable structure covers the key mechanisms that affect planning, and on how hard the remaining unknown part is to learn. For systems that still have reliable local mechanisms, the current framework allows these mechanisms to be kept and systematic deviations not represented by the structural model to be corrected through residual learning. R2 provides evidence for this under a controlled mismatch.
We agree that, if the main prediction difficulty comes from the behavior of other agents for which no reliable model exists, the learning bottleneck may remain in the unstructured part, and the sample-efficiency advantage we observe cannot be directly generalized to such cases. Our paper targets partially known dynamical systems with mechanism structure that can be exploited; R2 is not sufficient to show effectiveness for general multi-agent systems. How to start from agent-level mechanisms and interactions to build hierarchical structured world models is a direction we hope to explore in future work. Section 7 of the revised paper adds this discussion of the scope of applicability.
Q1. Relative to standard system identification, what is the methodological and conceptual contribution?
Conceptually and methodologically, our paper explicitly separates the organization of existing structural knowledge, the identification of unknown dynamics, and planning, and connects them through a complete one-step transition interface. The construction procedure (Section 4.1, Appendix A.2) enables the model to accept actions specified by the planner and to support recursive prediction; after construction, the structural parameters can still be learned with standard system-identification methods, and the resulting model can be called by different planners (Sections 4.2–4.3).
Theoretically, our paper links the identification accuracy of the structural parameters to the planning-value error and the stability of action selection. Section 5 (Eqs. 6–8, Appendix B), using standard identification and Lipschitz error-propagation tools within a realizable structural model family and under the corresponding assumptions, gives a finite-sample identification error bound that depends on the effective parameter dimension. For the same candidate action sequences, it propagates the error through one-step transitions and multi-step prediction to the planning value, while accounting for the terminal-value approximation error.
If, at the current state, the true H-step planning value QHπ has a unique maximizing action over the finite candidate action set, and the value-estimation error on all candidate actions is smaller than half of the action-value gap, then exact maximization of the estimated values still selects the same action (Eq. 9).