acceptodds
Under review as a conference paper at ICLR 2027

Supervise What Decides Success: Criterion-Aligned Auxiliary Losses for Latent World-Model Planning

Abstract

Latent world models plan by scoring candidate action sequences with distances in latent space. However, task success is judged by physical quantities, hereafter the success-criterion quantities. In all four latent world models we examine, the end-effector position is encoded in the latent state with an error larger than the success criterion allows. Such a latent state cannot separate successful candidates from failing ones. We propose an auxiliary loss that uses success-criterion quantities as training targets, whereas existing latent world models take them only as inputs. During training, a linear head on the encoder and predictor outputs regresses the success-criterion quantities, and the regression error is added to the training loss. The head is discarded after training, so the model, its cost, and its inputs at inference are unchanged. This loss alone improves the success rate on PushT and cube by 3.5% and 3.4% (absolute), and both improvements are statistically significant. A success criterion thus specifies what a world model must retain in its latent state, and we show that it can serve directly as a training target.

Primary area:
applications to robotics, autonomy, planning
Keywords:
world model, robotic manipulation, planning, representation learning, jepa

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_wG9DOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper argues that latent world models use distances in latent space for scoring candidates during planning. This latent-space distance may not capture the physical quantities required to determine success precisely enough to separate successful candidates from the failing ones. To rectify this, the paper adds an auxiliary term to the loss function during training only, using the success-criterion quantities as the training target. As a result, the success rate increases by 3-5% for the manipulation tasks considered, and a per-stratum analysis shows that supervising some quantities can help some trials while hurting others.

Critical strengths
  1. The paper clearly states the proposed contributions and tests their approach on different manipulation tasks. The idea is simple and plausibly useful, and it doesn't add any inference cost.
  2. Figure 1 clearly explains the entire setup.
  3. The paper also clearly points out its limitations and describes the future work adequately.
Critical weaknesses
  1. The whole premise of goal-conditioned world models like LeWM and DINO-WM is to develop a model trained without any task-specific knowledge. Still, with the proposed approach, it became task-centric again. A different task will require separate training from scratch, which defeats the entire purpose. The frozen V-JEPA 2 suggests that the bottleneck lies elsewhere, as the readout improved while the success rate fell.
  2. The paper mentions that "the deviation does not vanish with representation size or pretraining," however, Section 2.2 mentions that median hand error decreased from 3.10 cm to 2.10 cm by moving from DINOv2-S to DINOv2-L, which suggests that increasing the representation size does help. Appendix B reports a V-JEPA 2 predicted latent hand error of 1.18 cm, which is already within the tolerance limit.
  3. The paper states that the improvements are statistically significant using the McNemar test, but it only uses a single seed. McNemar tests capture trial-to-trial variance, not training variance. It doesn't indicate which method is better. In fact, Appendix D shows the seed variance is as large as the effects being claimed. The paper does cite Agarwal et al., 2021, but doesn't follow it properly.
  4. The main suggestion is to supervise every quantity, but that's not the case with the "coffee" task.
Questions for authors
  1. What is the difference between readout error and coarse readout? Adding a formal definition to the paper can help clarify it.
  2. In the fitted ridge regression from the latent vector, what's the probe's training set size and regularisation strength? Are the frozen encoder---the CLS tokens or the pooled patch features for probing the representation purposes?
  3. Why are the OGBench cube-(double/triple/quadruple)-v0 tasks not considered? These tasks align better with the proposed approaches and, in fact, involve different downstream operations, such as placing in order and stacking. How will these different tasks for the same environment be handled with the current setup?
  4. Why are navigation tasks not considered? The success criterion for the navigation tasks is identical in terms of goal tolerance, as the method handles. The reachability problem will affect the baseline equally, so the comparison will stay paired. Running them will help show that the gain survives even when the cost's geometry is also wrong, or reveal otherwise. this can potentially make the claims stronger.

Minor Suggestions:

  1. Table 9 is not formatted properly.
  2. The reacher exclusion seems avoidable. We can encode the joint angles as sines and cosines, which would remove the wrapping issue.
Reviewer_12haOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes using an auxiliary loss to guide policy learning, where the auxiliary loss is computed based on the distance between a predefined success-criterion quantity and the corresponding quantity inferred from the latent representation. Overall, this design is reasonable but too ad hoc and requires task-specific design.

Critical strengths
  1. This paper proposes using a task-relevant auxiliary loss to improve policy performance, which is a reasonable approach.

  2. The experimental results also demonstrate the effectiveness of the proposed method.

Critical weaknesses
  1. The design of the success-criterion quantity is too ad hoc and requires customization for each task.

  2. This work is incremental, and I think similar ideas of using auxiliary tasks have already been extensively explored in different machine learning domains.

Questions for authors

N.A.

Reviewer_73xGOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The authors study which quantities a latent world model should retain in its latent state for successful planning. They note a mismatch between how task success is decided and how planning actions are computed (task success is judged by physical quantities, which they call success-criterion quantities, while planning ranks action candidates only by latent distance). They propose an auxiliary loss that uses the success-criterion quantities as training targets, which they add to the training objective. They evaluate on LeWM with PushT, cube, Reacher, stack and coffee, and on a DINO-WM-type model with a frozen DINOv2 encoder on stack. The improvements in success rate are 3–5%.

Critical strengths
  • The method is simple and adds no cost at test time.
Critical weaknesses
  1. The writing is poor and imprecise, and notions are mentioned well before being introduced. A few examples:
  • “Given a goal image of the scene the robot should reach” (l32) is imprecise. What does it mean for a robot to reach a scene?
  • “The planner selects the action sequence whose predicted latent state is closest to the latent state of the goal image” (l33). What’s a predicted latent state?
  • “A success criterion asks whether the positions of the hand and the objects lie within a tolerance of their values at the goal” (l37). No hand has been mentioned so far.
  • “How much a difference in the success-criterion quantities between two candidates shows up as a difference in their latent distances enters the loss nowhere” (l76) is hard to read.
  • “A corrected readout therefore need not raise the planning success rate” (l78). What’s a corrected readout? “unless both outputs hold the success-criterion quantities in a form the same linear head can read” (l152). What’s a form the linear head can read?
  1. The evaluation protocol is unclear. In Sec. 3.1, it is not clear what a “trial pair” is. Is it a pair of start and goal (l194), or the “pair of outcomes of the two models on the same trial” (l202)? As a result, “LeWM uses identical pairs of 2000 trials; the DINO-WM type identical pairs of 1000 trials” (Table 2) is hard to understand.

  2. There are a few inaccurate claims:

  • The paper says it adds “only one linear head per quantity and one term to the training loss” (l132). From Eq. (1), it seems that each quantity adds two terms, one for the encoder output and one for the predictor output.
  • If the authors use the same data as prior work, then the training data are not successful demonstrations. There is no notion of success in the data, since the goal is only defined at test time as 25 steps away from the sampled start.
  1. Including the success criterion during training defeats the purpose of a world model, since it means we need one training per task. It would have been interesting to have it as a light fine-tuning.

  2. The improvements are marginal.

Questions for authors
  • In the frozen-encoder case, the encoder does not receive gradients. Which parameters does the encoder term of the auxiliary loss train?
  • “Tasks in which reachability decides success, such as mazes or tasks with walls, are outside our scope” (l197). What is different in those tasks?
  • Table 2: why not use the same number of trial pairs for LeWM and DINO-WM?
  • Why does supervision sometimes decrease the success rate?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.