acceptodds
Withdrawn from ICLR 2027

ATLAS: Aligned Transport of Latent Structure for Reliable World Model Planning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Latent world models rely on representation geometry for planning, yet regularizing the latent marginal alone does not determine the state-to-state relationships used for action selection. We show that this can cause planning-relevant novelty structure to be weakened as representations are transformed into the final latent used by the planner. We introduce **Aligned Transport of Latent Structure (ATLAS)**, a training objective that explicitly preserves relational geometry while calibrating the global latent distribution. ATLAS transfers normalized pairwise structure from an informative encoder representation to the planning latent and uses Wasserstein embedding matching (WEMReg) to calibrate its marginal through one-dimensional Wasserstein-2 transport. Our analysis shows that relational preservation and marginal calibration impose non-redundant constraints, and connects finite-candidate planning stability to relational distortion, latent-scale mismatch, and prediction error. Instantiated in LeWM, ATLAS improves mean goal-reaching success across PushT, TwoRoom, and OGBench-Cube on both lower- and higher-novelty evaluation subsets, with the largest gain on higher-novelty TwoRoom episodes. Representation and rollout diagnostics further show stronger novelty-related structure in the planning latent, improved marginal calibration, and lower multi-step prediction error. Together, these results highlight preservation of planning-relevant latent geometry as an important ingredient for reliable world-model planning. Code is available at https://anonymous.4open.science/r/atlas-world-model-72C4/.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
World Model, representation learning, out-of-distribution robustness, optimal transport

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_XSugOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ALTALS, a method that consists of a regularization loss for anti-collapsing and a relational geometry calibration loss. The authors conducted experiments on PushT, TwoRoom, and OGBench-Cube with a goal offset of 50.

A few concerns about this paper:

  1. Renaming an existing method, SWD, to WEMReg should not be presented as a proposed method. Also, you should compare with [1] and [2] to show the real advantage. Due to the short timeline, these two papers have not been published yet, but neither have LeJEPA or LeWM, so I think it is fair to compare against these two methods.
  2. It is good to test on OOD data, but the OOD dataset selection should not be based on KNN. The stable-world-model repo provides an easy way to change the color, add noise, etc to create the true OOD test set. To make everything easier, you can just replay the original action. This comparison should be more convincing.
  3. Considering the scale of the experiments is relatively small, the authors should also add Reacher to provide a complete comparison among these methods.

[1] VISReg: Variance-Invariance-Sketching Regularization for JEPA training

[2] Rectified LpJEPA: Joint-Embedding Predictive Architectures with Sparse and Maximum-Entropy Representations

Critical strengths

I don't see an obvious strength of this paper. Although the performance of ATLAS is better than LeWM, there is a big misalignment between the motivation of the proposed method and the actual experimental settings. This makes the argument in the paper not convincing.

Critical weaknesses
  1. Renaming SWD to WEMReg should not be considered a proposed method.
  2. OOD should not be picked from the training set by KNN. Considering 50 steps as the offset, the result could be affected by the horizon rather than by OOD.
  3. VISReg and LpJEPA should be added to the related work. If possible, add the corresponding comparison.
  4. There are a lot of other works to compare in the ID setting. Please add them so the judgment is not biased.
Questions for authors

Why is the Reacher dataset not involved in this paper? Since this paper is mainly comparing with LeWorldModel, the comparison should be complete.

1 comment

W1: Renaming SWD to WEMReg should not be considered a proposed method.

Thank you for these pointers. WEMReg is not presented as a novel optimal transport distance. Its connection to sliced Wasserstein matching is explicitly acknowledged through citations to the original literature in both the introduction and related work. Importantly, using an established statistical discrepancy does not preclude a methodological contribution in representation regularization. SIGReg itself builds on established characteristic-function-based Gaussianity testing. The relevant question is therefore not whether the underlying statistical tool is new, but what role its formulation and analysis play in the proposed learning objective. Our contribution is to identify a limitation of finite-frequency matching, theoretically characterize this distinction in Theorem 1, and investigate projected Wasserstein calibration in conjunction with an independently motivated relational-preservation constraint. We will also incorporate VISReg and Rectified LpJEPA into the related-work discussion to clarify the positioning of ATLAS among existing JEPA regularization methods.

W2: OOD should not be picked from the training set by KNN. Considering 50 steps as the offset, the result could be affected by the horizon rather than by OOD.

We thank the reviewer for this suggestion. Our use of the term “OOD” follows the nearest-neighbor convention[1], where OOD refers to states lying in low density regions of the training distribution. Under this definition, the standard way to identify such states is by their distance to the training support, which is what the KNN split measures. ATLAS is designed to preserve the representational structure that separates states near versus far from the training support, while a perturbation test such as changing color or adding noise probes robustness to appearance or covariate shift, which is a valuable but different property. We will state our definition of OOD more explicitly in the revision to avoid this ambiguity.

[1] Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. Out-of-distribution detection with deep nearest neighbors. In International Conference on Machine Learning (ICML), 2022.

W3: The authors should also add Reacher to provide a complete comparison among these methods.

Thank you for raising this. Now we have added Reacher into our experiments and here are the success rates:

MethodReacher
LeWM (base)50.50±3.5150.50 \pm 3.51
ATLAS w/o WEMReg54.40±4.4754.40 \pm 4.47
ATLAS w/o OOD-recovery56.10±3.6156.10 \pm 3.61
ATLAS58.30±5.23\mathbf{58.30 \pm 5.23}
Reviewer_24R5OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ATLAS studies how a latent world model can preserve relationships between observations while keeping the overall representation distribution well-behaved. In LeWM, the paper observes that novelty scores computed from mean-pooled encoder patch features are more informative about planning failure than scores computed from the final representation used for planning. This motivates the OOD-recovery loss, which encourages normalized distances between planning representations to match those between the corresponding patch features. These features come from the same encoder and are treated as fixed targets for this loss within each update, while continuing to evolve during training.

ATLAS also replaces LeWM’s SIGReg with WEMReg, which matches one-dimensional projections of the planning representations to a standard Gaussian using squared Wasserstein-2 distance. Both terms are combined with the prediction loss, while retaining LeWM’s architecture and test-time planning procedure. The theoretical analysis distinguishes the constraints imposed by the two losses and bounds candidate-selection error relative to the patch representation under stated assumptions. Experiments on PushT, TwoRoom, and OGBench-Cube report higher mean goal-reaching success than LeWM, accompanied by analyses of representation geometry and prediction error.

Critical strengths

1. The controlled comparison with LeWM demonstrates a useful improvement from changing the training objective.

ATLAS improves mean planning success over LeWM on all three tasks, with the largest increase on TwoRoom. The four-way ablation varies the marginal regularizer and the relational loss, producing some useful observations. Although the gains vary across tasks, the results provide meaningful evidence that the proposed objectives can improve planning within the existing LeWM system.

2. The theoretical and empirical analyses distinguish properties that are often treated together.

Proposition 2 establishes that matching the overall representation distribution does not determine the relationships between particular observations, and vice versa. Theorem 3 then relates candidate-selection error to distortion of those relationships, differences in latent scale, and prediction error. The experiments examine corresponding representation properties and multi-step prediction accuracy rather than relying only on final success rates. Together, these analyses offer a useful framework for studying learned planning representations.

Critical weaknesses

1. The added planning benefit of combining both proposed changes is limited in the reported experiments.

Table 1 shows that most of the improvement over LeWM can already be obtained with one of the proposed changes. Relative to the stronger individual modification in each task, the full objective improves mean success by only 0.5% on PushT, 0.1% on TwoRoom, and 0.2% on OGBench-Cube. On PushT and OGBench-Cube, retaining SIGReg and adding OOD-recovery accounts for most of the gain. On TwoRoom, replacing SIGReg with WEMReg already reaches 73.9%, compared with 74.0% for the full method. Using one fixed combined recipe across tasks is a useful feature, and the paper acknowledges that the benefits can overlap. Nevertheless, the current results do not convincingly establish an additional planning advantage from the combination. This limits the demonstrated practical value of the full objective.

The broader comparisons also show that the benefit is task-dependent. ATLAS has the highest reported mean on both PushT subsets, but several methods perform similarly on OGBench-Cube. On TwoRoom, DINO-WM performs substantially better, and PLDM reports higher success on the higher-novelty subset, at 72.4% compared with ATLAS’s 61.0%. The comparison with DINO-WM involves a different pretraining regime, but this does not fully explain the broader pattern of results. In particular, PLDM is also competitive on TwoRoom, while several methods perform similarly on OGBench-Cube. The experiments establish an improvement over LeWM more clearly than they establish when ATLAS should be preferred over the other evaluated approaches. Hence, the practical value of the complete combination remains less established than the usefulness of its individual changes.

2. The evaluation does not establish generalization to excluded episodes or robustness across independent training runs.

Appendix A states that evaluation goals are sampled from the full set of valid episodes rather than from a held-out test split. The evaluation, therefore, does not ensure that these episodes were excluded when constructing the training data. The subsets called ID and OOD are then formed by dividing the evaluation episodes at the median of a novelty score based on distance to a sampled bank of training states. This measures relative familiarity within the evaluated sample, but the higher-novelty half is not necessarily unseen during training. An episode can be relatively far from the sampled reference bank while still supplying training clips. Hence, I believe that the reported results do not establish generalization to episodes excluded from training.

The reported uncertainty also leaves an important question unresolved. Table 2 lists one training seed, while the means and standard deviations in the main comparisons are computed over five evaluation seeds. These repetitions measure variation in evaluating the trained models, not variation arising when the models are trained independently. Results from additional training runs would help determine whether the small differences between the full objective and its simpler variants are consistent or depend on the particular checkpoints evaluated.

Minor concern

I would encourage the authors to increase the font size of the text in their figures, as it makes it easier to read them.

Questions for authors
  1. What proportion of the evaluation episodes also supplied training clips? Are any results already available using evaluation episodes excluded from world-model training?

  2. Are results from additional training seeds available for the four variants in Table 1? In particular, does the full objective consistently improve over the stronger individual modification?

3 comments

W1: The added planning benefit of combining both proposed changes is limited in the reported experiments.

We thank the reviewer for this careful reading. We do not claim the two objectives are additive, the Prop. 2 in paper states that they constrain different properties. What we claim is reliability under one fixed recipe, which the experiment results for each task support: 1. the best single component changes by task, so no single component is a safe a-priori choice; 2. a single component can regress, e.g. WEMReg alone underperforms LeWM on Cube while the full objective never does; 3. on PushT and TwoRoom the full objective matches the best single component mean with notably lower variance, i.e., it is more reliable, not merely equal in mean; 4. we added Reacher as a fourth task and the results show that the improvement is clearer.

MethodReacher
LeWM (base)50.50±3.5150.50 \pm 3.51
ATLAS w/o WEMReg54.40±4.4754.40 \pm 4.47
ATLAS w/o OOD-recovery56.10±3.6156.10 \pm 3.61
ATLAS58.30±5.23\mathbf{58.30 \pm 5.23}

For the comparison with other methods, we agree the ranking across baselines is task-dependent. But it is expected and not specific to ATLAS, even LeWM does not beat these baselines on every task. DINO’s advantage comes from a frozen DINOv2 encoder pretrained on 142M external images, and PLDM uses a different architecture and training regime. ATLAS trains from task data alone like LeWM, so it is a reliable objective level improvement for from-scratch JEPA world models.

W2: The evaluation does not establish generalization to excluded episodes or robustness across independent training runs.

Thank you for your feedback. Following the nearest-neighbor convention[1], we use “OOD” to mean states in low density regions of the training distribution, identified by KNN distance to the training support. Our aim is not to validate on a held-out OOD dataset, but to report performance across relative novelty levels. LeWM likewise samples goals from the full set of valid episodes with no held-out split, so we follow the same protocol for comparability.

On robustness across training runs, we retrained both LeWM and ATLAS at three training seeds on PushT, under the identical protocol and mirroring LeWM’s own training-seed study. The gain is consistent on every seed, so the improvement is not specific to the evaluated checkpoints in the paper.

Training seedLeWMATLAS
3072 (paper)45.1050.80
307343.0047.50
307444.4049.00
mean ± std44.17 ± 0.8749.10 ± 1.35

[1] Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. Out-of-distribution detection with deep nearest neighbors. In International Conference on Machine Learning (ICML), 2022.

I thank the authors for their rebuttal and additional experiments that they conducted. These experiments do add to the credibility of certain claims, and I am encouraged to see the performance persist across seeds as well. I do still feel like the additional benefit of combining both components is relatively modest. Overall, the new evidence does address the questions I had before, so I am raising my score in line with that.

Thank you so much for your update! We are grateful for your suggestions, which have helped improve our work.

Reviewer_ouPPOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes ATLAS, a training objective for latent world models that combines dynamics prediction with Wasserstein embedding matching (WEMReg) for marginal regularization and an OOD-recovery loss that preserves the normalized pairwise distance structure of the encoder's patch representations in the planning latent space. ATLAS achieves higher mean goal-reaching success rates than LeWM across three tasks—PushT, TwoRoom, and OGBench-Cube—with the largest improvement on TwoRoom.

Critical strengths

The paper uses the same LeWM architecture and planning setup, with a 2×2 ablation that isolates the effects of WEMReg and OOD-recovery. ATLAS improves planning success across all three environments, with a particularly substantial gain on TwoRoom (58.0% → 74.0%).

Critical weaknesses

1. Insufficient justification for the planning relevance of the patch representation geometry.

The fact that kNN novelty in the patch representation predicts planning failure does not establish that its pairwise Euclidean distances reflect state reachability or other planning-relevant relationships. The representation may simply distinguish common visual configurations from rare ones, with rare configurations also being harder to plan in. This could explain the observed correlation between novelty and planning failure without implying that patch distances capture meaningful reachability. Thus, the paper provides insufficient justification for preserving the patch representation's pairwise geometry as the target for the planning latent.

2. Insufficient justification for WEMReg's planning advantage over SIGReg.

(1). Inconsistent empirical gains. Without OOD-recovery, WEMReg substantially improves planning success on TwoRoom (58.0% → 73.9%), but yields only a modest gain on PushT (45.1% → 47.0%) and underperforms SIGReg on OGBench-Cube (48.1% → 46.9%). With OOD-recovery, its additional gains are marginal across all three tasks (+0.5, +2.2, and +0.2 percentage points, respectively).

(2). Limited theoretical justification. Theorem 1 shows that finite-frequency matching can miss non-Gaussian structure detectable by exact projected Wasserstein matching. However, this population-level result does not establish WEMReg's superiority under finite minibatches and sampled projection directions, let alone its advantage for planning. Both methods face finite-sampling limitations.

(3) Unclear motivation for replacing SIGReg. In lines 129–130, the paper argues that SIGReg's Gaussianization removes planning-relevant OOD structure. Yet WEMReg also encourages Gaussian matching and is presented as a more discriminating distributional constraint. Why should WEMReg avoid the same problem? The paper neither resolves this tension nor establishes that better marginal calibration leads to better planning.

Questions for authors

The paper defines OOD episodes as the higher-novelty half of the evaluation set based on a median split of kNN distances, rather than evaluating on a distinct OOD test distribution. Moreover, evaluation goals are sampled from the full set of valid episodes without a held-out test split. This evaluates performance across relative novelty levels but does not necessarily establish OOD generalization. Is there any controlled distribution shift in the planning evaluation? Do the improvements hold on genuinely held-out OOD states or configurations?

1 comment

W1: Insufficient justification for the planning relevance of the patch representation geometry.

We thank the reviewer for this point. Our motivation does not require patch-space Euclidean distances to approximate reachability or shortest-path distances. The reviewer’s concern therefore imposes a stronger requirement than our method assumes. Our justification is both theoretical and empirical. First, Theorem 3 explicitly identifies relational distortion as a contributor to finite-candidate planning instability, establishing why preserving the reference geometry matters under the stated assumptions. Second, Table 1 provides a controlled intervention: adding OOD-recovery while keeping the architecture and planning procedure fixed improves success. Third, Figure 8 shows that the baseline loses novelty-related structure in its planning latent, whereas ATLAS preserves it in closer agreement with the true-state oracle.

For the rarity concern, the possibility that rare configurations are intrinsically harder does not invalidate these findings. Our claim is not that novelty causes planning failure, but that preserving informative relational structure mitigates its degradation in the latent representation used for planning.

W2: Insufficient justification for WEMReg's planning advantage over SIGReg.

(1) Inconsistent empirical gains.

We thank the reviewer for this careful reading. The task-dependent gains are consistent with the distinct representation deficiencies targeted by the two components. WEMReg addresses marginal calibration, and Figure 5 shows that its largest calibration improvement occurs on TwoRoom, where its planning benefit is also strongest. Importantly, theoretical complementarity does not imply additive success-rate improvements. Proposition 2 establishes that marginal matching and relational preservation impose non-redundant constraints, not that their empirical gains must sum independently.

The full ATLAS objective provides a single fixed recipe across tasks where the strongest individual component varies. WEMReg alone can even regress on Cube, whereas the full objective improves over LeWM across all evaluated tasks. The newly added Reacher results further show a +7.8 pp gain over LeWM and a +2.2 pp gain over the strongest single-component variant.

MethodReacher
LeWM (base)50.50±3.5150.50 \pm 3.51
ATLAS w/o WEMReg54.40±4.4754.40 \pm 4.47
ATLAS w/o OOD-recovery56.10±3.6156.10 \pm 3.61
ATLAS58.30±5.23\mathbf{58.30 \pm 5.23}

(2) Limited theoretical justification.

Theorem 1 only establishes that finite-frequency matching can miss non-Gaussian structure that projected Wasserstein detects, it does not establish a finite-sample advantage or a planning advantage, and we do not intend it to. It motivates WEMReg as a strictly more discriminating calibration objective. The two claims it does not cover are supported separately. The finite-sample side is exactly what Figure 5 tests: those calibration diagnostics are computed under finite minibatches and randomly sampled projection directions, and WEMReg lowers covariance and projected-distribution error on several of them, most clearly on TwoRoom. The planning side is carried by the η term in Theorem 3, together with the ablation. We agree both estimators face finite-sampling, so we treat Theorem 1 as motivation and rest the finite-sample and planning claims on this evidence rather than on the theorem itself.

(3) Unclear motivation for replacing SIGReg.

Thank you for this point. As the introduction shows, a well-behaved marginal alone does not preserve state-to-state relationships whether enforced by SIGReg or WEMReg, and Prop. 2 shows the two terms are independent constraints. So we replace SIGReg not to preserve structure but because WEMReg is a more faithful calibration objective.

Answers to Questions:

Thank you for raising this. Our use of the term “OOD” follows the nearest-neighbor convention[1], where OOD refers to states lying in low density regions of the training distribution. Our evaluation protocol follows LeWM, in which goals are sampled from the valid-episode set and success is measured without a held-out test split. LeWM reports a single aggregate success rate per task and uses no distinct OOD test distribution, so we match the established setup for a fair comparison.

[1] Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. Out-of-distribution detection with deep nearest neighbors. In International Conference on Machine Learning (ICML), 2022.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.