acceptodds
Under review as a conference paper at ICLR 2027

Action-Consequence Alignment for Reliable Planning and Self-Improving in Latent World Models

Abstract

Latent world models learn to predict observed transitions, yet low prediction error alone does not guarantee reliable planning. Inspired by self-tickling experiments in neuroscience showing that disrupting motor–sensory correspondence increases prediction mismatch, we examine whether learned world models preserve an analogous action–consequence correspondence. The results show nearby alternatives can receive lower prediction errors despite producing physical outcomes farther from the recorded target. With that future treated as a goal, this reveals a concrete prediction–planning mismatch: the model assigns a lower cost to an action that achieves the target less accurately. To mitigate this gap, we introduce Action–Consequence Alignment (ACA), a training objective that complements forward prediction by penalizing the prediction-error advantage of locally searched alternatives over factual actions without additional model components or environment interactions during training. The same principle can also guide additional data collection for self-improvement. We demonstrate that across diverse environments and evaluation settings, ACA improves planning performance and reduces real goal error, while ACA-guided data collection outperforms random local sampling. These results support action–consequence alignment as a practical principle for bridging predictive learning and reliable planning.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
latent world model, action consequence, generalization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_KwtBOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper identifies a prediction–planning gap in latent world models where low prediction error on recorded transitions does not necessarily imply that the model will favor actions that achieve desired physical outcomes. The authors demonstrate that locally perturbed counterfactual actions can receive lower prediction errors than the factual actions while producing outcomes farther from the observed targets.

main contributations:

  1. Identify and demonstrate the prediction–planning gap.This establishes that accurate forward prediction alone does not guarantee reliable action selection.

  2. It augments the standard forward-prediction objective with a local factual–counterfactual comparison.

The authors evaluate ACA using LeWorldModel (LeWM) and Sensorimotor World Models (SMWM) across four simulated control environments: TwoRoom, Reacher, Push-T, and OGBench-Cube, including stricter evaluation protocols and limited-data settings.

Critical strengths
  1. A simple and principled approach to mitigating the prediction–planning gap. The paper identifies a meaningful failure mode in latent world models, where counterfactual actions can receive lower prediction errors despite producing worse physical outcomes.

  2. Consistent empirical improvements across diverse planning settings. ACA improves planning success across both LeWM and SMWM on four simulated environments, with particularly notable gains under strict evaluation, single-plan, and long-horizon settings.

Critical weaknesses

The main limitation is that the validation is conducted in simulated environments. The results establish promising evidence for improved model-based planning, but do not yet demonstrate transfer to physical robots or real-world deployment.

Questions for authors
  1. What is the overhead computation cost introduced during training,
  2. How sensitive are the improvements to the perturbation radius and ACA loss weight across different environments and action spaces?
Reviewer_jw2bOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

This paper's contribution is based on the observation of what they call the “prediction-planning gap”, where the predicted action to achieve a transition doesn’t correspond to the actual action that would create the transition in the physical world. Based on this observation they introduce ACA, an alignment loss to make the predictive world model better align to the actual world dynamics.

Critical strengths
  1. The paper executes a reasonable empirical evaluation of their alignment loss showing that effectively aligning the model with the actual action dynamics in fact benefits planning and that the benefits are independent of the predictive loss (the forward loss).
  2. The contribution, albeit with limited novelty, might be useful for the JEPA community.
Critical weaknesses

Similar to previously published JEPA papers, the authors fail to position their contributions in the wider literature of world models, planning, RL and sequential decision-making. The insight that is provided in this paper: pure prediction (the forward loss in latent space) is not enough for planning has been known for a while now:

  • Gelada C, Kumar S, Buckman J, Nachum O, Bellemare MG. Deepmdp: Learning continuous latent space models for representation learning. InInternational conference on machine learning 2019 May 24 (pp. 2170-2179). PMLR.
  • Misra, D., Henaff, M., Krishnamurthy, A., and Langford, J. Kinematic state abstraction and provably efficient rich-observation reinforcement learning. In Proceedings of the 37th International Conference on Machine Learning, volume 119, pp. 6961–6971, 2020.
  • Allen C, Parikh N, Gottesman O, Konidaris G. Learning markov state abstractions for deep reinforcement learning. Advances in Neural Information Processing Systems. 2021 Dec 6;34:8229-41.
  • Lamb A, Islam R, Efroni Y, Didolkar A, Misra D, Foster D, Molu L, Chari R, Krishnamurthy A, Langford J. Guaranteed discovery of control-endogenous latent states with multi-step inverse models. arXiv preprint arXiv:2207.08229. 2022 Jul 17.
  • Rafael Rodriguez-Sanchez, Cameron Allen, and George Konidaris. "From Pixels to Factors: Learning Independently Controllable State Variables for Reinforcement Learning." Reinforcement Learning Journal, vol. 7, 2026 (This is only a short list of works that use similar insights). Although the authors do recognize that in previous JEPA papers inverse models have been introduced to solve the same issue, and they use it as baselines, they fail to recognize that these ideas have already been explored in similar applications.
    I would highly encourage the authors to study the model-based RL, and representation learning in RL literature and position their ideas correctly within the JEPA family of work.
Questions for authors

IMHO, this paper introduces an insight that might be lacking out there in the JEPA field, but with limited novelty in the wider literature. In order to consider it for acceptance, I would like to see their contributions framed correctly and that would start by expanding their literature review, recognize the previous works that have put forwards these ideas before, and change the presentation to take this into account.

Reviewer_KhHjOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper talks about the gap between prediction accuracy and planning performance in latent world models, which means a model may predict observed transitions well but still prefer actions that lead to worse real-world outcomes. Then the authors propose Action–Consequence Alignment (ACA), which encourages the model to assign lower prediction error to the actual action than to nearby alternative actions. It improves planning success across several environments and reduces real goal errors. The authors also use ACA to guide the collection of additional training data, enabling better self-improvement than random action sampling.

Critical strengths

The paper identifies a meaningful gap between prediction accuracy and planning reliability in latent world models. And ACA introduces a straightforward training objective based on comparing factual and counterfactual actions, without requiring additional model components.

ACA improves planning success and reduces real goal errors, and the authors extend it to guide targeted data collection, achieving better planning performance than random local action sampling.

Critical weaknesses

My major concern is about the assumtions. ACA assumes that the factual action should locally minimize prediction error for the observed future. However, this assumption may not hold in stochastic environments or when multiple actions can produce similar outcomes.

Also, ACA enforces relative prediction-error ordering rather than directly improving counterfactual dynamics accuracy. The objective may be minimized by increasing errors for alternative actions without learning their true consequences.

Although planning performance improves, the experiments do not directly evaluate whether ACA produces more accurate counterfactual predictions. This makes it difficult to determine what drives the improvements. The reported results use multiple planning seeds but only a single trained checkpoint, leaving the robustness of improvements across independent training runs unclear.

Questions for authors

See weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.