acceptodds
Under review as a conference paper at ICLR 2027

TRACE-WM: Learning Physical Concurrency for Asynchronous World Models

Abstract

World models trained from asynchronous interaction logs can overfit arbitrary action serializations: physically interchangeable actions appear as different futures, while global order invariance can erase interactions for which order genuinely matters. We study whether a world model can instead infer where temporal order is physically irrelevant. We formulate state-dependent action swappability through a stochastic two-step diamond discrepancy and introduce Active Diamond Discovery (ADD), an active probing framework for uncertain likely-safe diamonds, together with TRACE-WM, which imposes invariance only on relation-admitted reorderings. In a fresh pre-specified 16-seed comparison, a local-uncertainty ADD acquisition discovers 96.19 safe cross-robot probes versus 54.19 for Random, a 77.5% gain with 16/16 paired wins, and exceeds the original variance-reduction instantiation by 28.0%. In a separate 16-seed matched held-out-state evaluation, TRACE-WM reduces balanced distributional prediction error by 34.0% relative to Ordered and dependent order-effect error by 57.2% relative to Global invariance. A size-matched, common-initialization control shows that ADD-selected relation elements are modestly more useful downstream than Random-selected elements, while extra imperfect coverage can offset this benefit. The discovery advantage also persists under a discontinuous hard-mutex interaction. Relation extrapolation remains imperfect, so the learned relation is treated as an empirical inference rather than a universal safety certificate.

Primary area:
reinforcement learning
Keywords:
World Models, Model-Based Reinforcement Learning, Action Commutativity, Active Learning, Uncertainty Quantification, Multi-Agent Systems, Robotics

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_QgBTOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper tries to solve the problem occurred in world models trained from serialized logs of asynchronous systems, where the same physical outcome can show up under different action orders. For a state s and two actions a and b, the authors compare the final-state distributions of executing a then b and b then a (the diamond discrepancy), and treat the pair as swappable at s if the difference is below a tolerance. Based on this, the paper proposes three parts. ADD fits a Gaussian process for each action pair over the state space and uses a fixed budget of paired probes to find states where swapping is safe. TRACE-WM then adds reversed-order samples to the world-model training data only where ADD accepts the swap. Graph-TRACE adds a residual model, calibrated with 40 probes, to transfer the two-robot relation to three or four robots. Experiments are on a synthetic intersection benchmark (two robots in the main experiments), with a small MLP that predicts the two-step endpoint as a Gaussian. ADD finds more safe swaps than random probing with the same budget. In the world-model experiments, TRACE-WM has lower prediction error (W2) than the Ordered baseline and a much lower order-effect error than Global invariance.

Critical strengths
  1. The problem is well motivated. Logs from multi-agent or distributed systems are serialized, and it is not obvious which order differences are physically meaningful. Learning this as a state-dependent relation, instead of assuming a fixed independence relation as in partial-order reduction, makes sense to me.

  2. I also liked that order-effect error is reported separately from average prediction error. Table 1 shows why this matters: Global invariance has lower balanced W2 than TRACE-WM but much worse order-effect error, which would be missed with W2 alone.

  3. The paper includes Oracle, Random-derived and FullDyn-derived relations as controls, which makes it possible to check where the gains come from.

Critical weaknesses
  1. My main concern is scope. Everything is evaluated on a single synthetic intersection benchmark (2D state and four actions in the main two-robot setting) with a two-step endpoint predictor. All data and the ground truth come from the authors’ own simulator, and the held-out states are taken from the same state pool, so the generalization results are close to interpolation. I also don’t see an easy way to scale this up. The method fits one GP per action pair over a low-dimensional state, and each probe needs to reset to exactly the same state and reuse the same noise. It is unclear how this would work with a latent world model or on a real system. The authors acknowledge some of this, but I think it limits the contribution more than the paper suggests.

  2. For the cross-robot pairs, which are the non-trivial part of the benchmark, order only matters when the two robots are close enough to interact. So in this setting, finding where actions commute is the same problem as finding where the robots do not interact. Learning this kind of local independence and using it for data augmentation has been done before (NRI; "Neural Relational Inference for Interacting Systems", or CoDA; "Counterfactual Data Augmentation using Locally Factored Dynamics"), and the paper does not discuss or compare with it. I also suspect part of the problem comes from modeling the logs as single-agent action sequences, since many multi-agent world models just take a joint action at each step.

  3. Also, the controls in the paper do not really show that ADD is needed. With a randomly selected relation, TRACE-WM performs about the same as with the ADD relation, and the size-matched control in App. G gives only a small difference. The main gains in the abstract are each against the weakest baseline for that metric. On balanced W2, TRACE-WM with ADD is still worse than Global invariance and the FullDyn relation.

Questions for authors

Q1. How were the training logs generated? For each state and action pair, is only one ordering recorded, or both? And how large is the dataset?

Q2. Could you add an environment where actions interact but still approximately commute, and compare with a CoDA-style relation? In the current benchmark the two notions coincide for the cross-robot pairs, so it is hard to see what the commutativity formulation adds.

Q3. Without resets and shared noise, how would the discrepancy be estimated, and how many samples would it take? I am asking because I could not think of a realistic case where the probes are both possible and expensive. With a simulator like the one in the paper, the discrepancy can be computed for every state (as done for the ground truth), and without one, the paired probes are not available.

Q4. If a learned dynamics model is good enough to read swappability from (as FullDyn does), why not use it directly as the world model? In what setting would TRACE-WM with ADD be the better choice?

Reviewer_PHyGOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

For a stochastic transition function P(s′∣s,a)P(s' \mid s, a), the problem studied by the paper is to identify the set of triples (s,a,b)(s, a, b) for which the distribution of the state reached by applying action aa followed by action bb from state ss is within distance ϵ\epsilon of the distribution of the state reached by taking action bb and then action aa. For a selected triplet, the authors sample end states after both action sequences with identical random seeds, compute the distance between each pair of end states, and finally fit a Gaussian process to these distances as a function of ss, separately for each action pair (a,b)(a, b). A triple is added to the estimated set if the posterior mean plus 2.5 posterior standard deviations is at most ϵ\epsilon, and the next triple to sample is chosen to maximize a score that increases with the posterior probability that the distance is below ϵ\epsilon and is nonzero only for triples not yet in the set. A model of the two-step transition distribution is then trained by maximum likelihood on the observed tuples (s,a,b,s′)(s, a, b, s'), together with the tuple (s,b,a,s′)(s, b, a, s') for each triple in the estimated set. On a synthetic task with a two-dimensional state, this model has lower prediction error than models trained with no added tuples or with added tuples for all triples, and slightly lower error than a model trained with added tuples for a randomly selected set of triples.

Critical strengths

None

Critical weaknesses
  1. The writing is not understandable. Core terms ("diamond," "quotient," "relation-admitted reorderings," "physical concurrency") are used without being defined, and the text is written like a log of revisions rather than a paper. It narrates things like which acquisition score was "original," "historical," "frozen," or "fresh," and which results were rerun under which protocol. It is very difficult for a reader to reconstruct the method or the experimental design from the main text.

  2. The experiments are too small to support generalizable claims. The only environment is a synthetic two-robot task with a 2D state, 200 initial states, and six action pairs, and the "world model" is a Gaussian predictor of the state after two steps.

  3. The motivation / ecological validity seems unclear. Estimating commutation as in the method seems to require resetting the system to arbitrary states and executing both action orders repeatedly, which assumes a simulator... but with a simulator, commutation can be queried directly or training data generated in every order, making the learned relation unnecessary. The order ambiguity also seems to arises from serializing simultaneous actions, which can be avoided at an implementation lvl by engineering. IMO the paper does not describe a realistic system in which commutation is unknown but also matters for prediction, and can be tested at acceptable cost.

Questions for authors

There are no answers to questions which could materially affect my assessment.

Reviewer_RJcnOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper aims to exploit the equivalence between action orderings to augment training data for world models. To this end, they first conduct a small set of experiments to measure the discrepancy between the outcomes of different action orderings. Then, they train a Gaussian-process model on these "supervised" samples and actively select additional experiments. They apply the discovered relation to augment training data by altering action orders with small predicted discrepancies. They carry out synthetic robot-interaction systems to validate their method.

Critical strengths
  1. The goal of learning structure in action orderings and leveraging it as an inductive bias is interesting and inspiring. This general idea may find broad applications in this space.

  2. I find this paper well-written. It reports negative findings faithfully and provides valuable insights into the proposed approach, especially its limitations. These may shed light on future work in this space.

Critical weaknesses
  1. Although interesting, this specific structure may be limited in practice. In general, real-world actions interact; therefore, their orders indeed make a difference. Identifying interchangeable actions may require substantial experimentation. The learned regression model doesn't guarantee generalization to unknown scenarios and thus may provide erroneous signals.

  2. The experiments are limited to synthetic settings. It remains unclear whether the approach would benefit realistic, complex environments. Moreover, the ADD advantage in balanced error is negligible over random selection, and the latter even has lower order-effect error.

Questions for authors

Please see the weakness points.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.