acceptodds
Under review as a conference paper at ICLR 2027

CausalWAM: Mechanism-first Causal World-Action Models

Abstract

Embodied agents often act in environments where the consequences of actions are governed by hidden mechanisms, such as modes, latches, timers, thresholds, rates, and delayed triggers. Existing imitation policies, world models, and emerging world-action models learn powerful action and prediction models, but typically leave these mechanisms implicit, making them difficult to track, intervene on, reuse, or revise. We introduce a mechanism-first causal world-action model that infers a mixed discrete-continuous belief over hidden mechanisms from observation-action history. The model plans by selecting mechanism-level target interventions, such as unlocking a mode or driving an accumulator past a threshold, before grounding those targets into low-level actions with a WAM-style action decoder. It further maintains an external mechanism library that stores reusable local transition fragments and retrieves them for belief inference, planning, compositional transfer, and online adaptation. To evaluate this capability, we introduce MechanismBench, a DSL-generated benchmark that provides hidden mechanism state, counterfactual interventions, compositional splits, nuisance shifts, and rule changes. On MechanismBench, our full model improves overall success from 75.2% for a WAM-style baseline to 88.9%, counterfactual accuracy from 71.2% to 88.3%, and held-out composition success from 46.3% to 76.2%. Under shifted mechanisms, the library-enabled model recovers to 85.4% success after ten interactions, compared to 58.4% without the library, and focused Franka experiments show the same pattern on physical hidden-mode, accumulator, and delayed-trigger fixtures.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Mechanisms, programs, memory

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_7eZHOpenReview ↗
Rating
3(+1)
Confidence
1(−2)
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces CausalWAM, a mechanism-first world-action model that represents hidden device dynamics with executable causal programs and uses belief inference, program composition, action grounding, and local rule repair for partially observed interaction tasks. It also introduces MechanismBench to evaluate hidden-state inference, alternate-action prediction, compositional generalization, and adaptation to mechanism changes, together with Franka robot experiments on three instrumented fixtures. CausalWAM outperforms several world-model, programmatic, and multimodal-agent baselines, particularly on held-out composition and post-change adaptation.

Critical strengths
  1. The paper provides an extensive empirical evaluation with multiple baselines, ablations, controlled program-repair studies, and physical robot experiments.

  2. The executable-program representation provides a clear engineering interface for composing local transition rules and selectively updating them when the environment follows the assumed mechanism structure.

Critical weaknesses
  1. The paper never clearly establishes what concrete problem setting it is solving.

The manuscript repeatedly refers to “devices,” “buttons,” “hidden modes,” “hidden mechanisms,” and “internal control logic,” but I still found it surprisingly difficult to determine what task is actually being formalized. It is unclear what class of systems the method is intended for, what exactly counts as a hidden mechanism rather than simply a hidden state, and whether the main challenge is partial observability, state tracking, transition-rule inference, or online mechanism adaptation. Even the later examples introduce more variables and mechanism types without resolving this ambiguity in a unified task definition. As a reader, I was repeatedly left trying to infer the intended problem from isolated examples rather than from a clear formulation, which makes both the motivation and the scope of the contribution difficult to assess.

  1. The methodological novelty is difficult to separate from the benchmark design, making the paper read more like an experimental report than a general method contribution.

MechanismBench is generated from the same mechanism-template vocabulary used by CausalWAM. The benchmark composes known template families while varying their bindings, parameters, and visual rendering, and the paper explicitly describes this as evaluation “within the specified mechanism abstraction.” As a result, the reported generalization is still largely confined to a mechanism space predefined by the method itself. Held-out composition mainly tests recombination of familiar building blocks, rather than whether the model can reason about genuinely unseen mechanism structures. This substantially limits the benchmark’s value as evidence for broader generalization or general hidden-mechanism reasoning.

  1. The causal interpretation is not sufficiently established.

The paper repeatedly describes its local programs as “causal” and uses intervention language such as action hooks and do(⋅)do(\cdot) (Figure 3), but it never clearly formalizes what the causal variables are, what exactly is being intervened on, or which structural relations are assumed to remain invariant under intervention. The hidden state mtm_t is introduced as a mixture of discrete and continuous variables, but it is not clear why these variables should be interpreted as causal states rather than simply latent state variables in a partially observed dynamical system. Likewise, changing an action, changing a hidden state, and changing a transition rule are conceptually different interventions, yet their causal roles are not cleanly separated. Without a clearer causal semantics, the proposed “causal programs” appear closer to structured transition rules with editable parameters than to a well-defined causal model, which weakens one of the paper’s central claims.

  1. The physical experiments provide limited evidence for the broad “device interaction” claims.

The physical evaluation is restricted to three custom-built fixtures corresponding to the same hidden-mode, accumulator-threshold, and delayed-trigger abstractions already used in the benchmark. These fixtures are themselves constructed around the predefined mechanism structure, so they do not meaningfully test whether the method transfers to naturally occurring devices with unknown or structurally different internal mechanisms. More broadly, the physical experiments extend the same controlled assumptions into a robot setting rather than challenging them. Together with the benchmark design, this makes the experimental section read more like an exploratory validation of a handcrafted mechanism framework than a rigorous evaluation of a broadly applicable methodological contribution.

Questions for authors
  1. What exactly is the target task setting of CausalWAM, and what is meant by a “hidden mechanism” in this paper? How is it distinguished from hidden state, unknown transition dynamics, or mechanism change?

  2. Since MechanismBench is generated from the same predefined mechanism vocabulary used by CausalWAM, how can the reported held-out composition results be interpreted as evidence of generalization beyond this predefined mechanism space?

  3. What is the formal causal semantics of the proposed “causal programs”? Specifically, what variables are causally intervened on, what remains invariant under intervention, and why should the hidden variables introduced in the model be interpreted as causal states rather than latent dynamical states?

  4. What evidence do the physical experiments provide for general device interaction when the three fixtures are themselves constructed to instantiate the same hidden-mode, accumulator-threshold, and delayed-trigger abstractions assumed by the benchmark and method?

Reviewer_rfjaOpenReview ↗
Rating
2
Confidence
3
Informativeness
3(+1)
Presentation
3
Soundness
2(+1)
Read review
Paper summary

This paper focuses on a robot’s operating tasks that target devices’ behavior depends on state the camera cannot see, such as a lamp that cycles through modes, or a lock that only opens after a long press. The proposed CausalWAM writes device logics as programs built from nine hand-crafted templates. From the history of observations and actions, it estimates which hidden state the device is likely in, and it plans by linking programs. A diffusion policy is trained together with a head that predicts future observations to turn the chosen step into actions. Training uses ground-truth labels provided by the benchmark that the authors build with the nine hand-crafted templates. The experiment is also conducted on this author’s own benchmark. On this benchmark, CausalWAM outperforms a baseline, a PoE-World adaptation, and an LLM agent.

Critical strengths
  1. The problem is real. Current visuomotor policies struggle when the right action depends on hidden device state.
  2. I found how the method tests rule updates convincing. When something unexpected happens, it checks whether the model can tell apart three cases: the device was simply in a different state, the observation was noisy, or the device’s rule really changed.
  3. Within the author's benchmark, the paper includes some well-chosen controls. For example, the Sup-WAM baseline is trained with the same labels as CausalWAM but does not run the programs. Comparing against it seems a fair way to check whether running the programs makes a difference.
Critical weaknesses
  1. My main concern is that the problem setting is quite narrow and seems to be defined around the method. This study only covers i) devices built from the nine templates, ii) one local rule change per session, and iii) edits that keep the same variable types. MechanismBench is generated from the same templates and provides the program labels the model is trained on, and the physical devices were also built to match them. For this reason, the experiments do not show that the method works for device interaction in general (or real world scenario).
  2. I am not convinced this is a world-action model. The planner decides what to do (which button to press and for how long), and the action decoder just carries it out. During training, the world part and the action part are only linked by an extra loss that predicts future observations. To me, this is closer to a symbolic planner with a learned skill decoder.
  3. Most of the individual ideas are not new. Separating rules, state, and belief is the standard POMDP setup. The templates, library, and validated refinement all come from existing work such as hybrid automata, PDDL+, DreamCoder, Voyager, and PoE-World.
Questions for authors
  1. What happens when a device’s behavior cannot be written with the nine templates? For example, if one template family is left out of training, or a device has something like a nested menu, can the model still handle it, or does the repair step just fail?
  2. For Real devices, the labels that MechanismBench provides for training would not be available. Can the model be trained without these labels, and how much does performance drop? It would also help to see results when only part of the training data is labeled.
  3. If the PoE-World baseline and the LLM agent were given the same nine templates, would CausalWAM still win? This would tell us whether the gains come from the method itself or mostly from having the templates.
  4. The paper calls Causal World-Action Model, but for me, it is not an WAM. The world part and the action part are only connected through an extra loss during training. Could you show that the world model actually shapes the actions with additional experiments?
  5. Could you show results on at least one environment you did not build yourselves? For example, an existing simulator with hidden modes or timers added, or commercial devices.
2 comments

Thanks for the second round. The paper is clearer now, but what it shows is a modest gain from a hand-designed prior built from known components. There is a small gain in the new experiments outside the author's own benchmark, but it is not large enough for me to see a clear contribution. I'm not convinced with "We can call it a world-action model".

Thank you again for your careful and prompt engagement throughout the discussion. We appreciate your acknowledgment that the manuscript is clearer and that the additional experiments were helpful. Your questions have made the comparisons more rigorous and helped us state the contribution more precisely.

We agree that the matched-template, zero-privileged-annotation setting is the appropriate primary comparison, and we have made it prominent in the revised manuscript. Under this setting, CausalWAM (w/o supervision) improves over PoE-MB by 3.8 percentage points on held-out composition and 4.1 points on adaptation AUC. We feel that for this contribution, the context of the capabilities isolated by the controlled studies and their transfer beyond the benchmark matters.

On independently implemented OmniGibson processes, with the same template access and no privileged annotations, CausalWAM achieves 79.5% success compared with 73.1% for PoE-MB and 72.8% for MM-agent. The gains are particularly pronounced in thermal and gated-machine processes, while other categories approach parity. These results characterize where mechanism-based reasoning is helpful.

The hand-designed vocabulary is an explicit inductive bias, but also an important part of the scientific instrumentation. It provides common transition semantics through which hidden-state ambiguity, dependency composition, and local rule changes can be defined and measured systematically. That the same templates improve PoE-MB and MM-agent (both baselines use LLMs), is itself evidence that this representation supplies useful structure beyond our particular method. Our matched comparisons then test the additional benefit of CausalWAM's computational use of that structure. Also, that our non-LLM mechanism-based approach remains competitive and sometimes superior to baselines (that makes online V/LLM calls) is scientifically meaningful.

Finally, our methodological contribution is the use of the same executable local mechanism hypotheses for belief-conditioned intervention planning, action grounding, and evidence-validated, binding-scoped revision. The controlled search, executable-rule intervention, and local-repair studies isolate the benefits of this organization. We use world-action to describe this coupling between predicted mechanism-state evolution and generated actions, with prediction performed in programmatic rather than video space. We respectfully note that the specific computational contribution does not depend on terminology alone, and that in the requested experiments, we have measured effect of future co-training, decoder co-adaptation,

We recognize the tradeoff of using bounded mechanism repertoire versus pretrained LLM priors, and have sought to make its limitations and impact explicit. Thank you again for the thoughtful discussion and for helping us sharpen both the evaluation and the scope of our claims -- this is a lot of work and we're thankful for it!

Reviewer_Gc9TOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
1
Soundness
2
Read review
Paper summary

The paper introduces a method and benchmark for learning causal laws that describe mechanisms in the world such as appliances, and which can be applied to simulated and real robotics. They compare against VLMs, program synthesis, and WAMs on a new benchmark they introduce, MechanismBench, and on a real robot. The method takes a fair amount of data to get started, but can learn across tasks, and adapt very quickly to new changes physical mechanisms (i.e. within 10 episodes).

Critical strengths
  1. The results are really exciting, and actually kind of shocking to me. They're learning causal world models, blending symbolic programs and neural perceptual inputs, and using it to reason about temporally-extended cause-effect chains on a real robot. It appears to work, and bests baselines I'm intimately familiar with.
  2. No LLMs in their system! Recent work like ExoPredicator can do similar stuff but only by wrapping frontier models. This make their work much more scientifically relevant -- no uncertainty about what's really going on or where the intelligence really comes from. (But the catch is what you'd expect, namely a DSL and having to train their own synthesizer)
  3. MechanismBench -- I can see this filling a niche within the world models community. I cannot think of any parallel.
Critical weaknesses
  1. The description of the method is very rough and hard to follow. Sec 3.1 needs examples of local causal programs, specify the action space (basically options? are they pretrained?). Sec 3.2 needs to clarify how much of the state is symbolic/discrete vs neural embeddings -- I thought it would be STRIPS-style, but the writing makes it sound more like neural embeddings (is it both?). 3.3 needs to illustrate the program repair with an example. Reading the methods gives whiplash -- as one example of many, on line 259 the reader suddenly learns that the planner has its own small transformer! (How is it trained? Remains a mystery) I know the authors claim to have used AI only for grammar, but the writing is very claude-y which doesn't help.
  2. The dataset is equally parts tantalizing and obscured. The reader wants to use your dataset! You've introduced MechanismBench -- show us the environments, tell us how many there are, give a rough inventory of what's in it.
Questions for authors

Can you rewrite the methods and explain the new benchmark, ideally in your own words and without any AI? I genuinely love the work but think the exposition is a show-stopper. I am happy to move from weak to strong accept if you can just explain it really clearly (and if my positive impression holds up upon receiving greater clarity about whats really going on).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.