acceptodds
Under review as a conference paper at ICLR 2027

From Sequence Prediction to Mechanism Execution: The Neural Mechanism Interpreter for World Modeling

Abstract

The prevailing paradigm of world modeling relies on Transformer or state-space model (SSM) backbones, which conceptualize physical dynamics as a dense sequence prediction problem. However, this global/recurrent sequence matching inextricably entangles entity state representations with transition rules, leading to catastrophic forgetting and a lack of combinatorial generalization when environmental dynamics non-stationarily mutate. In this paper, we propose a fundamental architectural departure: the Neural Mechanism Interpreter (NMI). Instead of context matching over monolithic sequences, NMI maintains a dynamic continuous state and executes a discrete, discoverable library of localized neural mechanisms. We introduce Dynamic Mechanism Routing (DMR) to sparsely assign state-transition programs to specific objects, and Targeted Mechanism Revision (TMR), a novel backpropagation protocol that isolates gradient updates exclusively to active mechanisms during prediction failures. This freezes unrelated mechanism parameters, reducing a direct source of catastrophic forgetting. Extensive experiments on complex, rule-mutating interactive physical environments and real-world pedestrian forecasting datasets demonstrate that NMI achieves zero-shot combinatorial generalization and cross-domain transfer significantly beyond Transformer and SSM baselines. Crucially, under sudden interventions of physical laws, NMI adapts and recovers prediction accuracy with minimal few-shot interactions, supporting its viability as a principled architectural foundation for causal-driven interactive models.

Primary area:
learning on time series and dynamical systems
Keywords:
World Modeling, Causal Mechanism Execution, Catastrophic Forgetting, Combinatorial Generalization, Dynamical Systems

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_cRkbOpenReview ↗
Rating
3(+2)
Confidence
2
Informativeness
2
Presentation
2(+1)
Soundness
2
Read review
Paper summary

The paper proposes a new world modeling structure called Neural Mechanism Interpreter. For each object in the system, the method aggregates information from nearby objects, then use the information as the input to a router. The router chooses a small neural network to use. The network computes the update to the state of this object. During the training phase, a trick is that only the network with the worst performance when the environment changes will be updated. The author hopes that this can regularize what each network learns.

Critical strengths
  1. It's an interesting idea to modify certain aspects of the environment in training.
Critical weaknesses
  1. The paper is extremely hard to read. The author invents a lot of names for different parts of their method, but does not clearly explain what these parts are doing. I'm confused about a lot of things in the paper. For example, what is the aggregation function of AGG? What is the training setting of the baselines (Do they observe the same environment change?)
  2. Inconsistency in the report. Appendix C says that all baselines use identical parameter counts, but immediately in Table 4 in Appendix D we see that the parameter count is far from identical.
Questions for authors

See weakness.

Reviewer_taHBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Neural Mechanism Interpreter (NMI), a pipeline for maintaining and training a set of transition functions (mechanisms) that apply over object states. At each prediction step each object aggregates local information from the neighboring objects and selects a mechanism to use to transition to the next state. The system is trained to match the observed trajectories via backpropagating the matching loss only through the selected mechanism. Additionally, the authors proposed to disentangle the effects of different mechanisms by optimizing a meta-objective on a set of observations yielded by synthetically generated rule-interventions. The evaluations demonstrate that NMI outperforms baseline methods in both zero-shot OOD accuracy and few-shot adaptation to rule-mutations.

Critical strengths
  1. The method is clear, generally intuitive and well-motivated.
  2. The method is compared to a broad range of baselines.
Critical weaknesses
  1. The approach currently assumes the object-centric state is available. While it is a valid assumption in some domains, it largely limits the application of NMI to a broad range of general environments the modern world models train on.
  2. The evaluation results are mixed (see Questions), which raises questions both regarding novelty and regarding what contributes most to the performance.
Questions for authors
  1. Most of the ablations in Table 3 (except for "No TMR" and "Soft") outperform the baselines in Table 2. This seems to indicate that the restricted parameter updates are crucial for better adaptation performance. Have the authors compared to baselines that also restrict parameter updates? E.g. a top-K MoE transformer that updates only the parameters of the experts at adaptation?
  2. Most of the ablations in Table 3 (except for "No local") yield significantly lower RMSE50 compared to the baselines in Table 1, which makes me question whether the most of the benefit here comes from the local aggregation, rather than TMR or other components. Baselines, like a Transformer with local attention could clarify this. Could the authors add such baselines? If not, how would they expect locality influence the performance of the baselines?
  3. The authors stated that omitting meta-objective remains competitive. Given that the rest of the components (independent experts, hard routing, local attention, restricted parameter updates) existed in the literature in some form, the meta-learning used to induce disentanglement in the mechanisms was significantly contributing to the novelty of the whole approach, and omitting it would hurt the novelty. Could the authors provide additional analysis or intuition on what else the meta-learning contributes to if not the performance on the target metrics? Does the inclusion of the meta-objective yield more interpretable mechanisms?
  4. Can NMI also be applied to a domain with no explicit state? Can it be instantiated inside a Transformer, as e.g. a hard top-K MoE with restricted backpropagation from Question 1?

At this stage I am leaning towards weak rejection, but I am willing to raise my score if the rebuttal addresses my concerns listed above.

Reviewer_ZYE6OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates how models can adapt to changes in environment while preserving performance on unchanged rules. It proposes the Neural Mechanism Interpreter (NMI), which aggregates local information through an object graph and uses hard routing to select a neural mechanism for each object’s state transition. During adaptation, the developed Targeted Mechanism Revision (TMR) updates only the selected mechanism while freezing the router and other modules. Experiments on a rule-mutating physics benchmark show improvements in combinatorial generalization and few-shot adaptation over the evaluated baselines, with an additional evaluation on ETH/UCY pedestrian forecasting.

Critical strengths

The paper addresses a meaningful problem, and the use of independent modules with selective updates provides a clear approach to reducing interference during adaptation. The reported results demonstrate improvements over the several baselines. The ablation studies also provide evidence that selective updates improve adaptation and retention. The paper provides theoretical analysis of gradient coupling, guarantees for parameter isolation under fixed routing, and a generalization bound for local adaptation.

Critical weaknesses

The technical contribution relative to closely related work is unclear. The paper does not adequately discuss its relationship to existing modular works. Specifically, [1] uses object-level hard rule selection, rule-specific MLPs, and residual state updates. Similarly, [2] keeps attention parameters fixed during the inner loop while updating the selected modules. These methods are close to the proposed DMR and TMR, respectively. The authors should clarify the differences from these methods and identify the main technical contributions of this work. The zero-shot transfer claim of this paper is unsupported. Appendix E trains NMI on pedestrian data, which does not demonstrate zero-shot transfer from physics. This experiment evaluates generalization across pedestrian scenes rather than zero-shot transfer from physics simulations to real-world trajectories. The authors should clarify which experimental results support the claimed zero-shot transfer. The theoretical results are limited in their support for the paper's main claims. Theorem 1 establishes nonzero samplewise gradient coupling under specific assumptions, but does not establish harmful interference at the population level or catastrophic forgetting. Therefore, the theoretical analysis does not fully explain the claimed advantages in adaptation and retention. The combinatorial generalization experiment changes two factors simultaneously. Both the number of objects and the rule combinations change between training and evaluation. Consequently, the results do not distinguish generalization to unseen rule combinations from generalization to larger scenes. These two factors should be evaluated separately. References

[1] Neural Production Systems. NeurIPS, 2021.

[2] Fast and Slow Learning of Recurrent Independent Mechanisms. arXiv, 2021.

Questions for authors

What are the main technical contributions of this paper compared with previous work [1, 2]? Is NMI initialized from a physics-trained checkpoint in the pedestrian forecasting experiments? Which results support the claimed zero-shot real-world transfer? When gravity and electromagnetism simultaneously affect the same object, a single selected mechanism may represent both effects. How can TMR revise the gravitational dynamics while preserving the electromagnetic response?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.