acceptodds
Under review as a conference paper at ICLR 2027

ConTent: Contact-Force Prediction for Latent World Models Enhances Dynamic Manipulation

Abstract

Latent world models are powerful encoders of both observations and environment dynamics in robot learning. This study applies a world model to a dynamic manipulation task in which a robot uses a stick to push a ball rolling down a tilted board up into a goal zone. In this task, it is crucial that the observed task-relevant world state, such as the ball state, be accurately encoded in the latent space. Yet, although the regularisers of world models prevent collapse of the latent space, the representation can still degenerate and discard such information, thereby degrading the performance of model predictive control. We show that the discarded information can be recovered by predicting contact forces, which anchor the object representation in the latent space. We exploit this recovery to propose ConTent, contact-force prediction for latent world models, which outperforms the LeWorldModel baseline on our manipulation task. Our method predicts contact forces at training time and raises the success rate from 2% to 52% when planning with the cross-entropy method. Fine-tuning the predictor on on-policy data further improves it to 77%.

Primary area:
applications to robotics, autonomy, planning
Keywords:
world models, joint-embedding predictive architecture, JEPA, contact force, auxiliary prediction, dynamic manipulation, non-prehensile manipulation, model predictive control, representation learning, robot learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_gFWtOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

Through theoretical analysis, this paper show that latent world models can discard task-relevant object information despite avoiding representation collapse and achieving low prediction loss. They further investigate how contact-force prediction can help preserve such information in the latent representations. Based on this analysis, they propose ConTent, which adds a training-time head to predict high-frequency contact indicators and forces from intermediate predictor features, aiming to improve object information retention and dynamics prediction. Experiments on a simulated dynamic ball-pushing task show that ConTent substantially improves planning success over LeWM.

Critical strengths
  1. The paper provides a theoretical analysis showing that avoiding representation collapse does not guarantee that a latent world model retains the object information needed for control.

  2. ConTent introduces a lightweight auxiliary contact-force prediction head during training to encourage the model to retain task-relevant object information. The method achieves a substantial improvement in planning success over the LeWM baseline.

Critical weaknesses
  1. Limited novelty. Prior work, such as PhyLatent, has already discussed how non-collapsed representations can still fail to capture control-relevant physical information and proposed training-time physical state supervision to address this issue. The theoretical analysis and contact-force supervision in this paper do not seem to offer a clear conceptual or technical advance beyond this existing line of work.

  2. Insufficient justification for contact-force supervision. It is unclear why contact force is chosen as the auxiliary prediction target. Other physical signals, such as object pose trajectories, velocities, or contact points, could also help the model retain task-relevant information. The paper does not adequately explain why contact-force supervision would be preferable, nor does it provide experimental comparisons to establish its specific benefits.

  3. Limited experimental evaluation. The experiments only demonstrate improvements over LeWM in a single simulated ball-pushing environment. It remains unclear whether the same representation failure occurs in other latent world models or whether the proposed approach generalizes to different models and tasks.

Questions for authors
  1. What specific advantages does contact-force supervision offer over other physical auxiliary targets, such as object pose trajectories, velocities, or contact points? Can the authors provide experimental evidence to support this choice?

  2. How does ConTent differ from prior work such as PhyLatent that also studies the retention of control-relevant information through physical-state supervision? Please clarify the distinct theoretical insight and methodological contribution, and identify which results specifically support that distinction.

  3. Because ConTent adds an auxiliary prediction head, what evidence shows that the gains come specifically from contact-force information rather than from adding an auxiliary prediction objective? Could the authors report a capacity-matched control with a generic or non-contact target, or clarify which ablations address this question?

  4. The evaluation uses one simulated ball-pushing task and LeWM. Have the authors tested ConTent with other world-model architectures or manipulation tasks? If not, what evidence supports its applicability beyond this setting, and how should the scope of the claims be bounded?

Reviewer_1aN9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ConTent, a contact-force prediction framework designed to improve task-relevant representations in latent world models for dynamic robotic manipulation. The authors identify a limitation of joint-embedding predictive architectures: although regularization prevents representation collapse, the learned latent representations may still discard information essential for downstream control. To address this issue, ConTent introduces an auxiliary contact-force prediction objective during world-model training. The force prediction head predicts contact occurrences and planar contact forces at a higher temporal resolution than the control frequency. Importantly, contact forces are used only as training targets and are not required during inference or planning. The paper also presents a theoretical analysis of information retention in latent representations, examining conditions under which auxiliary prediction targets can preserve otherwise discarded state information. The proposed approach is evaluated on a simulated dynamic manipulation task in which a robot uses a stick to push a ball toward a target region on an inclined surface. Compared with LeWorldModel, ConTent improves the success rate from 2% to 52% on the challenging task. Additional on-policy fine-tuning with a ranking objective further increases the success rate to 77%.

Critical strengths
  1. A simple and effective approach to improving task-relevant latent representations. The main strength of this work is the use of contact-force prediction as an auxiliary training objective to encourage the world model to preserve information relevant to physical interactions. Unlike approaches that require tactile observations during inference, ConTent uses contact information only during training. This is an appealing design choice because it potentially improves representations without introducing additional sensing requirements at deployment.
  2. A meaningful theoretical motivation supported by representation-level analysis. The paper goes beyond demonstrating empirical improvements by investigating why conventional latent world models may discard important object information despite avoiding representation collapse. The distinction between representation collapse and the loss of task-relevant information is particularly insightful. The theoretical analysis, together with the probing experiments and visualization of predicted trajectories, provides a useful perspective on how auxiliary physical supervision can influence the quality of learned representations.
Critical weaknesses
  1. Limited experimental validation and insufficient ablations to isolate the contribution of contact-force prediction. The experimental evaluation is restricted to a single simulated ball-pushing task, with LeWorldModel serving as the primary baseline. While the performance improvement is substantial, it is difficult to determine whether the benefits arise specifically from contact-force prediction or more generally from introducing additional task-relevant supervision. In particular, the paper does not compare against alternative auxiliary targets such as contact occurrence prediction alone, object-state prediction, or other dynamics-related signals. Furthermore, the absence of experiments involving different manipulation tasks, physical parameters, or real-world environments limits the evidence for generalization.
  2. Limited practical applicability due to reliance on ground-truth contact-force supervision. The proposed method relies on ground-truth contact-force labels obtained directly from the MuJoCo contact solver during training. While this is straightforward in simulation, obtaining equivalent supervision in real-world manipulation is considerably more challenging. This raises concerns about the scalability of ConTent to real-world robotic datasets, particularly those involving multiple contacts or human demonstrations. Evaluating robustness to noisy or partially observed force supervision would significantly strengthen the practical relevance of the proposed approach.
  3. A substantial gap between the theoretical assumptions and the practical experimental setting. The theoretical results rely on restrictive assumptions, including deterministic dynamics, open-loop actions, and, for the strongest retention guarantees, vanishing prediction errors and sufficient observability through auxiliary targets. However, the majority of the training data consists of expert demonstrations generated by a state-dependent policy, which violates the open-loop action assumption. Additionally, the implemented world model uses observation histories, whereas the principal theoretical results concern information retention in individual latent representations. Although the authors acknowledge these limitations, the analysis does not directly establish that contact-force prediction guarantees retention of task-relevant information under the actual experimental conditions.
Questions for authors

Q1. Is the improvement specifically attributable to force prediction, rather than auxiliary supervision in general? Could the authors provide an ablation comparing (i) contact occurrence prediction only, (ii) contact-force prediction as proposed, and (iii) auxiliary ball-state prediction?

Q2. How robust is ConTent to changes in physical dynamics and task configurations? The current experiments focus on a single simulated manipulation environment. Could the authors evaluate the learned representations and planning performance under changes in ball mass, friction, or other relevant physical parameters?

Q3: How sensitive is ConTent to the quality and availability of contact-force supervision? Since the proposed method uses contact-force labels obtained from a simulator, it is unclear whether similar improvements can be achieved with real-world force measurements, which are often noisy, incomplete, or aggregated across multiple contacts.

Reviewer_1MpeOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces ConTent, a contact-force prediction approach for improving latent world models in dynamic manipulation. The authors highlight that preventing representation collapse does not necessarily ensure the preservation of task-relevant object information, and provide theoretical analysis of how auxiliary supervision can encourage its retention. Building on LeWorldModel, ConTent adds a lightweight prediction head that learns contact indicators and contact-force trajectories during training, without requiring force measurements at planning time. The method is evaluated on a simulated tilted-table ball-pushing task using model predictive control with the cross-entropy method. In the more challenging bottom-init setting, ConTent improves the success rate from 2% to 52%. Further fine-tuning using simulator-generated candidate rollouts and a state-based ranking objective raises the success rate to 77%. Representation probes, decoded rollouts, and planning-horizon evaluations provide additional analysis of the relationship between object information, dynamics prediction, and control performance.

Critical strengths
  1. It clearly distinguishes between preventing representation collapse and preserving task-relevant information. Satisfying distributional regularization constraints does not necessarily ensure that object position or dynamics information is retained. The analysis in Section 3 and the appendix provides a principled motivation beyond simply introducing an auxiliary loss.

  2. ConTent predicts contact indicators and contact-force trajectories only during training, using an auxiliary head with approximately 0.38M parameters. Planning continues to rely on visual observations and actions, without requiring force measurements. This design makes the approach practically appealing.

  3. Under the same planning configuration, ConTent increases the bottom-init success rate from 2% for LeWM to 52%. This provides compelling evidence that the proposed supervision benefits control performance in the evaluated task, rather than merely improving an offline prediction metric.

  4. Position probes, image reconstructions, multi-step predictions, planning-horizon evaluations, and fine-tuning objective comparisons provide insight into the relationships among representation quality, dynamics prediction, and control performance. The differing failure modes observed with the small and full datasets are also informative.

Critical weaknesses
  1. The main comparison is against LeWM without the auxiliary prediction head. While this demonstrates the benefit of the proposed supervision, it does not establish which component drives the improvement: contact detection, force magnitude and direction, or task-relevant auxiliary supervision more generally. Comparisons with simpler auxiliary targets would help clarify whether contact-force prediction provides a distinct advantage.

  2. All experiments are conducted in the same simulated ball-pushing environment. The centre-init and bottom-init settings vary the initial position rather than the manipulation task itself. Consequently, the results do not establish whether the benefits extend to different objects, dynamics parameters, or contact conditions. Broader evaluation would strengthen the evidence for the method’s general applicability.

  3. Figure 2 shows that LeWM’s representation still retains decodable information about the ball when trained on the full dataset. Figure 4 also shows that both models achieve relatively low position-probe errors toward the end of training, despite a substantial gap in planning success. These observations leave open whether the improvement primarily stems from better preservation of object information, more accurate dynamics predictions, a latent-space distance better aligned with planning, or information such as velocity that the current probes do not assess. Additional analysis separating these factors would make the proposed mechanism more convincing.

Questions for authors

Please refer to the "Critical Weaknesses" for specific issues. If these can be resolved, I will raise the score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.