acceptodds
Under review as a conference paper at ICLR 2027

MoMo: Conditioned Contrastive Representation Learning for Preference-Modulated Planning

Abstract

Temporally contrastive representation learning induces a latent structure that reduces long-horizon planning to inference in a low-dimensional linear system. However, existing contrastive planning work learns a single latent geometry that cannot distinguish among multiple valid behaviors, thereby preventing it from trading task efficiency against risk exposure for the same start-goal query. We introduce MoMo, a preference-conditioned contrastive planner allowing a scalar user preference to continuously modulate plan conservativeness at inference time, without retraining. MoMo learns a joint conditioning of the representation geometry and latent prediction operator via Feature-Wise Linear Modulation and low-rank neural modulation, respectively. We show that our formulation preserves the probability density ratio encoded in the representation space, which is required for inference-driven contrastive planning, while retaining its inference-time efficiency. Across six environments, MoMo smoothly adapts plan safety to user preferences, yielding improved temporal and preferential consistency compared to state-augmentation baselines.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Representation Learning, Contrastive Learning, Safe Planning, Conditioning

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_QKn7OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose a MoMo method to alleviate the problem of long-horizon planning reduction via latent structures. The idea constitutes preference conditioning with feature-wise linear modulation (FiLM) and low-rank neural modulation, thereby preserving probabilities ration in latent space without significant inference overhead. The method is tested on six environments with demonstration of empirical improvement of temporal consistency for longer horizons.

Critical strengths
  • the geometry during decoding is preserved as to representations of candidates and anchors that justifies the theoretical propositions and intuition for integrating the modulation direction, the result is consistent across environments in the fact MoMo's waypoint reconstruction on the plain level of order is far better than state-augmentation baselines
  • the main empirical is simple and consistently stable across reliable tasks with multi-seed evaluation with additional checking of spearman correlation of positive mean curve (Appendix B.3)
Critical weaknesses
  • Equation (11), formulas from Appendix A do look like separate instances, though it seems they try to tell about one thing which is about ratio of unconditioned equation similar to (4). Maybe they tell about the same thing but from this paper's part the soundness is not so clear to fully understand the connection: how ratio relates to waypoints and linear gaussian transition clearer than it is described at the moment
Questions for authors
  • which density ratio is the one of the loss that was used for training - equation (11), equation (17) in the appendix, or something else? Does the second term of (10) use Δi\Delta_i or Δj\Delta_j?
  • how often is the cost monotonic in Δ\Delta? Is one of the discrete collection levels held out during training?
  • related to a more general sense of evaluation: is the perfect correlation meant as a success of the model or as a check that the axes are the same quantity?
5 comments

Thank you for your time in reviewing our paper and for your precise feedback. We appreciate the opportunity to address the questions raised and respond to each point below.

Density ratio encoding

The relation between density-ratio encoding, the loss function, and Gaussian transitions Equations (4) and (11), and the derivation in Appendix A.1 describe the density-ratio property that underpins inference-driven planning. In the unconditioned case, this property states that learned logits encode the ratio of the probability of seeing a certain future state given an anchor state, over the probability of seeing that future state at random when drawing from the marginal state distribution. Equation (4), S⋆(s,s+)∝pγ(s+∣s)p(s+),S^\star(s,s^+)\propto \frac{p_\gamma(s^+\mid s)}{p(s^+)}, describes this first density-ratio for unconditioned learning. In the conditioned case, this ratio changes to the conditional probability of seeing a future state given both an anchor state and preference context. This is described in Equation (11), SΔ⋆(s,s+)∝pγ(s+∣s,Δ)p(s+),S_\Delta^\star(s,s^+)\propto \frac{p_\gamma(s^+\mid s,\Delta)}{p(s^+)}, such that Equations (4) and (11) are the unconditioned and conditioned analogs of one another, respectively.

Appendix A.1 derives these results via a logit function optimization problem. This starts with the unconditioned case [1], which derives the optimal logit function g∗(s,s+)g^* (s,s^+) in Equation (16) as equal to relation given in Equation (4). The conditioned case extends this derivation under the conditioned loss function of Equation (10) to yield the optimal conditioned logit function (g^)∗(s,s+;Δ)(\hat g)^*(s,s^+;\Delta) in Equation (17), equal to the relation from Equation (11). In this way, Equations (16) and (17) are the results of the derivations proving the analogous statements of Equations (4) and (11), respectively.

Under the Gaussian marginal assumption discussed in Appendix A.2, the density-ratio property yields the Gauss-Markov latent transitions used for inference in Section 3.2 and Appendix A.3. In the conditioned case, for a given Δ\Delta, let z=ψΔ(s)z=\psi_\Delta(s) and z′=ψΔ(s+)z'=\psi_\Delta(s^+). Let pΔ(z′)p_\Delta(z') denote the distribution obtained by encoding marginally sampled future states under this preference. Using the conditioned logit from Equation (6), the density-ratio relation gives

p(z′∣z,Δ)∝exp⁡(−12∥AΔz−z′∥22)pΔ(z′).p(z'\mid z,\Delta) \propto \exp\left( -\frac{1}{2} \lVert A_\Delta z-z'\rVert_2^2 \right) p_\Delta(z').

Under the Gaussian marginal assumption pΔ(z′)=N(0,cI)p_\Delta(z')=\mathcal N(0,cI), this becomes

\exp\left( -\frac{1}{2}\lVert A_\Delta z-z'\rVert_2^2 -\frac{1}{2c}\lVert z'\rVert_2^2 \right).$$ Completing the square gives the conditioned version of Equation (5): $$\mathcal N\left( \frac{c}{c+1}A_\Delta z, \frac{c}{c+1}I \right).$$ Thus, the density-ratio and Gaussian marginal together produce a Gaussian transition with a mean that is linear in the current representation. This transition reflects the discounted future distribution used to sample training pairs. Under the Markov-chain model, conditioning these transitions on the encoded start and goal gives $$p(z_{1:n}\mid z_0,z_g,\Delta) \propto p(z_g\mid z_n,\Delta) \prod_{k=1}^{n} p(z_k\mid z_{k-1},\Delta),$$ such that the posterior is Gaussian. Its block tridiagonal matrix and endpoint terms are given by Equations (21) and (22), with $A_\Delta$ replacing $A$. Solving $\Lambda_\Delta\mu=\eta_\Delta$ produces the latent waypoint means. Nearest-neighbor retrieval under $\psi_\Delta$ then converts them into state-space waypoints. This derivation therefore depends directly on the optimal density-ratio encoding. In summary, and in response to the reviewer's first question, Equations (11) and (17) are equivalent. Equation (17) derives the optimal conditioned density-ratio relation, while Equation (11) states this result for the conditioned similarity scores. Equation (6) defines the exponential similarity function used to approximate this relation during training. Regarding the loss function of Equation (10), the second term uses $\Delta_i$ in the numerator and $\Delta_j$ in the denominator so that the logits are always conditioned on the preference context of the anchor state. The numerators of the log terms are diagonal terms corresponding to positive pairs, so the preference of the anchor and future are equal, as they come from the same trajectory. In the first denominator, we sum over futures indexed $j$ for a given anchor, so $\Delta_i$ of the anchor is used for conditioning. In the second denominator, we sum over anchors indexed $j$ for a given future, so $\Delta_j$ of the anchors is used for conditioning. This allows for the loss function to remain symmetrized such that Equation (17) holds for both terms in the loss function. [1] Eysenbach, B., Myers, V., Salakhutdinov, R., and Levine, S. (2025). Inference via Interpolation: Contrastive Representations Provably Enable Planning and Inference. https://arxiv.org/abs/2403.04082
AuthorsOpenReview ↗

Cost monotonicity and preference-level holdout

We evaluate preference modulation using 11 uniformly spaced values of Δ\Delta, with the same start-goal queries at each value. Across 8 training seeds and 6 environments, MoMo achieves a positive endpoint cost change and positive Spearman correlation in all 48 seed-environment evaluations. In particular, the correlation is at least 0.870.87 for every seed on Point Four Obstacles, Point Contour, Point Habitat, and UR5. We provide these results under “More training seeds” and “Seeds and recovering cost across preference levels” comments in our response to Reviewer qGmb.

To directly assess monotonicity, we additionally examine all 10 consecutive preference intervals in each seed-specific mean cost curve. MoMo's cost is non-decreasing across 79%79\% of these intervals, compared with 61%61\% for IvI-SA, 55%55\% for PCA-SA, and 53%53\% for VIP-SA. The entire cost curve is non-decreasing across all 11 preference values in 8 of MoMo's 48 evaluations, including 6 of 8 UR5 seeds, compared with only 1 of the 144 baseline curves. Thus, while the response is not strictly monotonic in every setting, MoMo produces substantially more consistent preference ordering than the baselines. This is also reflected in its lowest mean normalized curve roughness in five of six environments (Table 6).

At the individual-query level, MoMo's endpoint cost increases with Δ\Delta for 98%98\%, 92%92\%, and 84%84\% of queries on Point Four Obstacles, Point Contour, and Point Habitat, respectively, and 80%80\% and 63%63\% on Drone and Ant Habitat. We further verify preference ordering during closed-loop execution on UR5, where the crossing height decreases monotonically with Δ\Delta for 151 of 160 query-seed pairs, and every pair achieves a Spearman correlation of at most −0.92-0.92. These results demonstrate that MoMo's preference response is consistently directed across environments and remains strongly ordered during execution.

To address the reviewer's question about generalization to unseen preferences, we introduce an additional UR5 experiment where all trajectories with Δ∈[0.40,0.60]\Delta\in[0.40,0.60] are removed from both the training and decoding datasets. Despite the absence of these preference levels, MoMo achieves a mean crossing-height error of only 2.02.0 cm against validation trajectories, places 100%100\% of withheld preferences between the observed boundary preferences, and maintains 99.7%99.7\% closed-loop success. In comparison, IvI-SA incurs a 12.712.7 cm error and achieves 83.0%83.0\% success. We provide the full experimental protocol and results under "Generalization to unseen preferences" in our response to Reviewer TqsC.

AuthorsOpenReview ↗

Correlation representing model success

The correlation between the queried preference level and the realized waypoint cost measures whether the planner produces behavior ordered correctly by the requested preference. The horizontal axis is supplied before planning. For a fixed start and goal, we query different values of Δ\Delta. We then compute the cumulative cost of the inferred waypoint sequences, which is represented by the vertical axis. We repeat this evaluation across held-out start-goal pairs to assess whether the preference response generalizes across different planning queries. In this way, the two are coupled only through the planner's ability to modulate plans effectively as a function of the queried preference.

The conditioning variable Δ\Delta, which is a label during training and part of the query at inference, and the evaluation cost thus share an intended notion of risk, but their relation must be learned. This is the fundamental problem of preference-conditioning, whereby a plan must reflect a user-specified preference. The alignment of the plan with the preference is measured via the cumulative cost.

Accordingly, a perfect correlation indicates an exact rank ordering of the mean waypoint cost across the queried preference values. This is indicative of successful preference modulation. We also measure waypoint reconstruction error against feasible held-out trajectories to assess whether the inferred waypoints preserve the temporal relationships required for planning. This is done by querying the planner with a held-out trajectory's start, goal, and preference label, and observing whether it recovers its intermediate states. Therefore, both the preference modulation curve and the waypoint reconstruction plots must be viewed together to evaluate the success of MoMo and baselines. For instance, IvI-SA returns a larger difference in endpoint costs, ΔC\Delta C, than MoMo in the Ant Habitat environment; however, it has a 38.6×38.6\times reconstruction error over the IvI baseline. Together with its substantially higher cost-curve roughness (Table 6), this shows that IvI-SA's larger cost range does not translate into equally reliable preference-conditioned waypoint predictions. In conclusion, and in response to the reviewer's third question, recovering the requested behavioral trend is therefore an intended model outcome.

Reviewer_TqsCOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2(+1)
Soundness
2
Read review
Paper summary

The paper extends contrastive planning to preference-conditioned planning. Each training trajectory carries a scalar preference label between zero and one, computed from the riskiest stretch of its per-step costs, rescaled so the most risk-averse trajectory maps to zero and the most risk-tolerant to one. MoMo feeds this label into the state encoder through feature-wise linear modulation, and into the transition operator as a small preference-dependent correction to a shared base matrix. The appendix shows the conditioning preserves what the Gauss-Markov planner relies on: similarity scores that encode how likely one state follows another under a preference, relative to the data distribution alone (A.1), and measures how far the representation distribution drifts from the assumed Gaussian shape (A.2). At inference, a start-goal query with a preference reduces to one sparse linear system, decoded by nearest neighbors. Six environments and four embodiments are tested: three 2D Point tasks, a Drone, the 29-dimensional Ant in AI Habitat, and a UR5 pick-and-place. The main result is a near-monotonic relation between queried preference and realized waypoint cost, with planning error close to the preference-agnostic IvI baseline, while state-augmentation baselines flatten or behave erratically. On one environment they also close the loop with a goal-conditioned SAC tracker: 96.4% success versus 90.3% (IvI-SA) and 79% (MODULI), with the largest cost swing across the preference range.

Critical strengths
  1. This is a good attempt to add inference-time preference control to contrastive planning while actually checking that the density-ratio encoding survives (A.1) and quantifying the Gaussianity drift (A.2).
Critical weaknesses
  1. Because the label and the metric come from the same cost statistic, near-monotone curves could be achieved by conditioned retrieval. Nothing in the evaluation directly tests generalization, for instance start-goal pairs that require composing states never visited consecutively in training, or preferences outside the labeled range. This matters because the paper sells planning under preferences, not conditioned imitation, and those are different claims.
  2. Scalability rests on one task. Ant Habitat is the only high-dimensional, longer-horizon setting, and it is where both the correlation and the Gaussianity check degrade. I cannot tell from the paper whether things degrade gently or sharply at, say, 50+ dimensions or 100+ step horizons.
Questions for authors
  1. How are evaluation start-goal pairs built? Are any of them not on a single training trajectory, i.e., do they need composition of intermediate states that never appear consecutively in the data? If yes, how does modulation quality look on those pairs?
  2. How sensitive are results to the CVaR quantile used for labeling, and to label noise in the preference values? Any experiment with held-out preferences, e.g., train on the lower four fifths of the preference range and query the top fifth?
  3. On Ant Habitat the drift grows monotonically with the preference. Does the planner's posterior stay calibrated there? Would renormalizing the predicted covariance restore the assumption, or is the drift benign as is? Even a small ablation would help.
  4. Could you add closed-loop results on one more environment, say UR5 or Ant? Outside Point Four Obstacles, the modulation is verified on decoded waypoints, not on executed trajectories.
9 comments

We thank the reviewer for their time and effort in providing feedback on our paper. Please find below the responses to the questions raised.

Compositional generalization to unseen start-goal queries

To evaluate compositional generalization, we construct start-goal queries by pairing the initial state of 1 trajectory with the final state of a different trajectory. We verify that none of the 40 sampled start-goal pairs in each environment occurs together in any single dataset trajectory. We evaluate these queries across 8 training seeds on Point Contour, Point Habitat, Drone, and Ant Habitat.

To examine how MoMo constructs these plans, we trace its 12 inferred waypoints back to their source trajectories. Across the four environments, MoMo draws its waypoints from an average of 8.2-10.4 distinct dataset trajectories per plan. Moreover, 72-91% of consecutive waypoint pairs do not appear together in the same order in any dataset trajectory. At most 0.1% of MoMo's waypoint sequences follow a single demonstration, compared with 2.6-17.7% for IvI-SA, whose waypoints come from only 1.9-3.6 trajectories on average.

We additionally evaluate preference modulation on these new start-goal pairs using the metrics from Table 6 and the same 8 training seeds reported in our response to Reviewer qGmb. The following table compares original validation pairs with new start-goal pairs, reporting mean values across seeds.

EnvironmentΔC\Delta C (original / new)ρ\rho (original / new)RnormR_{\text{norm}} (original / new)
Point Contour2.70/3.022.70 / 3.020.99/0.990.99 / 0.990.98/1.070.98 / 1.07
Point Habitat8.35/8.028.35 / 8.020.93/0.930.93 / 0.931.30/1.621.30 / 1.62
Drone2.86/2.572.86 / 2.570.49/0.420.49 / 0.424.52/5.864.52 / 5.86
Ant Habitat3.06/2.793.06 / 2.790.73/0.630.73 / 0.632.34/2.812.34 / 2.81

MoMo maintains a positive endpoint cost change in all 32 environment-seed evaluations and a positive preference-cost correlation in 31 of 32. The preference response remains similar to the original queries on Point Contour and Point Habitat, while Drone and Ant Habitat show some reduction in correlation and increased curve roughness. These results show that MoMo can generate waypoint sequences for start-goal pairs absent from the dataset by combining states from multiple trajectories, while maintaining preference modulation.

Scalability to longer horizons and higher dimensions

Our evaluation already includes planning queries spanning considerably longer horizons than the 100-step range mentioned by the reviewer. Except for Point Four Obstacles, each query connects the first and last states of a held-out validation trajectory, so the planning horizon reflects the trajectory's length. These range from 390-589 steps on Ant Habitat (median 400), 300 steps on Drone, 154-400 steps on Point Habitat, and 62-400 steps on Point Contour. In particular, Ant Habitat evaluates approximately 400-step queries in a 29-dim state space, while UR5 further tests planning with 57-dim observations.

To examine whether preference modulation deteriorates with increasing horizon, we additionally analyze the relationship between trajectory length and preference-cost correlation across 8 training seeds and 40 queries per seed. On Point Contour, the mean per-query Spearman correlation increases from 0.580.58 for the shorter half of the queries to 0.750.75 for the longer half, while Point Habitat shows a similar trend from 0.510.51 to 0.610.61. The Spearman correlation between query horizon and per-query preference-cost correlation is positive in all 3 environments with varying horizons: +0.30+0.30 on Point Contour, +0.20+0.20 on Point Habitat, and +0.24+0.24 on Ant Habitat. These results provide no indication that preference modulation deteriorates over the evaluated horizons.

On the higher-dimensional Ant Habitat task, MoMo also maintains consistent preference modulation across training seeds. With 8 seeds, the endpoint cost change is positive for every seed (ΔC\Delta C ranges from 1.551.55 to 3.883.88), with a mean preference-cost correlation of ρ=0.73±0.20\rho=0.73\pm0.20. MoMo also maintains the lowest mean normalized curve roughness RnormR_{\text{norm}} among the preference-conditioned methods, indicating smoother preference responses despite the increased task complexity.

Finally, the hierarchical planning formulation is designed to handle long trajectories without increasing the number of inferred waypoints proportionally to the underlying execution horizon. As described in Section 3.2, intermediate waypoints are obtained through a sparse linear solve in the learned latent space, with complexity O(ndz3)O(nd_z^3) for nn waypoints and latent dimension dzd_z. With the waypoint count fixed, the inference computation therefore does not grow with the number of low-level execution steps. Together, these results support MoMo's ability to retain preference modulation over longer-horizon and higher-dimensional tasks while preserving efficient latent waypoint inference.

AuthorsOpenReview ↗

Sensitivity to the CVaR quantile and label noise (UR5)

In our experiments, preference labels are computed deterministically from logged trajectories (Section 4.1), and the choice of risk statistic is fixed for each task. Nevertheless, we evaluate sensitivity to label noise and alternative risk statistics as additional stress tests. We independently retrain MoMo on UR5 (3 seeds per setting), including an original-label control. Each seed uses its own task-level train/test split, with 40 held-out start-goal pairs evaluated across 11 preference values. The 6-waypoint configuration and evaluation metrics match Table 6, while the evaluation queries differ from the fixed validation set used in the main eight-seed results. For label noise, we perturb each training trajectory's preference label with Gaussian noise of σ∈{0.1,0.2}\sigma \in \{0.1, 0.2\}, clipped to [0,1][0,1]. For quantile sensitivity, we replace UR5's original maximum end-effector height label with the CVaR of the per-step trajectory costs (Eq. 7), using α∈{0.5,0.75,0.9}\alpha \in \{0.5, 0.75, 0.9\}. We report planned cost modulation (ΔC\Delta C, ρ\rho), normalized curve roughness (RnormR_{\text{norm}}), and closed-loop performance in PyBullet using a goal-conditioned SAC policy. For execution, ΔCrollout\Delta C_{\text{rollout}} measures the change in mean per-step cost of successful rollouts between the preference extremes.

Training labelsΔC\Delta C ↑\uparrowρ\rho ↑\uparrowRnormR_{\text{norm}} ↓\downarrowSuccess ↑\uparrowΔCrollout\Delta C_{\text{rollout}} ↑\uparrowρrollout\rho_{\text{rollout}} ↑\uparrow
Original (retrained control)0.92±0.100.92 \pm 0.101.00±0.001.00 \pm 0.000.44±0.030.44 \pm 0.03100%0.360.361.001.00
Noise, σ=0.1\sigma = 0.10.83±0.070.83 \pm 0.070.98±0.010.98 \pm 0.010.41±0.070.41 \pm 0.07100%0.340.341.001.00
Noise, σ=0.2\sigma = 0.20.61±0.160.61 \pm 0.160.85±0.110.85 \pm 0.110.87±0.460.87 \pm 0.46100%0.330.331.001.00
CVaR0.5\mathrm{CVaR}_{0.5}0.61±0.070.61 \pm 0.070.41±0.080.41 \pm 0.081.85±0.191.85 \pm 0.19100%0.270.270.990.99
CVaR0.75\mathrm{CVaR}_{0.75}0.74±0.330.74 \pm 0.330.45±0.390.45 \pm 0.392.70±0.742.70 \pm 0.74100%0.320.320.990.99
CVaR0.9\mathrm{CVaR}_{0.9}0.65±0.310.65 \pm 0.310.60±0.330.60 \pm 0.333.74±1.763.74 \pm 1.76100%0.270.270.940.94

MoMo remains robust to label noise. At σ=0.1\sigma=0.1, the planned preference response remains comparable to the original setting, with ρ=0.98±0.01\rho=0.98\pm0.01 versus 1.00±0.001.00\pm0.00 and Rnorm=0.41±0.07R_{\text{norm}}=0.41\pm0.07 versus 0.44±0.030.44\pm0.03. Even at σ=0.2\sigma=0.2, which changes preference labels by 0.14 on average, MoMo maintains a positive planned cost response (ΔC=0.61±0.16\Delta C=0.61\pm0.16, ρ=0.85±0.11\rho=0.85\pm0.11). More importantly, closed-loop execution maintains 100% goal-reaching success and perfect correlation between the queried preference and mean executed cost (ρrollout=1.00\rho_{\text{rollout}}=1.00). The executed cost range also remains comparable to the original setting (0.330.33 versus 0.360.36), demonstrating that preference modulation persists despite substantial perturbations to the training labels.

MoMo also maintains strong closed-loop performance across the CVaR quantiles. For every α\alpha, all queries execute successfully, the realized cost remains strongly correlated with Δ\Delta (ρrollout≥0.94\rho_{\text{rollout}}\geq0.94), and the executed cost range reaches 75–89% of that obtained using the original labels. Although the planned cost curves exhibit lower correlations and greater roughness under CVaR-based labeling, the executed preference responses remain strongly ordered. The increase in planned roughness at higher quantiles is consistent with reduced resolution in the resulting preference labels. At α=0.9\alpha=0.9, the riskier half of the dataset has a mean normalized label of at least 0.950.95, while the mean label difference between neighboring risky behaviors (0.0050.005) is smaller than the spread within a single behavior (0.0140.014). This makes distinct risky behaviors difficult to distinguish through their labels and compresses their representation along the preference axis. Lower quantiles retain greater separation between these behaviors, consistent with smoother planned responses. Overall, these experiments show that MoMo maintains reliable preference-conditioned execution under both label perturbations and alternative risk statistics, while the smoothness of the planned response depends on how effectively the chosen labels distinguish the underlying behaviors.

AuthorsOpenReview ↗

Generalization to unseen preferences

To evaluate generalization to unseen preferences, we retrain MoMo and IvI-SA (3 seeds each) after removing all trajectories with Δ∈[0.40,0.60]\Delta \in [0.40, 0.60] from both the training data and the decoding set. We conduct this experiment on UR5, where Δ\Delta is derived from maximum end-effector height, which serves as a clearance proxy over the divider. Consequently, no remaining trajectory crosses the divider at the withheld clearance heights. We evaluate 40 start-goal pairs per seed at 21 uniformly spaced preference values, five of which fall within the withheld interval. Similar to the waypoint reconstruction evaluation in Figure 4, we compare the inferred plans against validation trajectories at the corresponding preferences, here using the end-effector height when crossing the divider. We additionally report preference-cost correlation ρ\rho, curve roughness RnormR_{\text{norm}} (Table 6), and closed-loop success using a goal-conditioned SAC policy in PyBullet (Appendix B.1).

MetricMoMoIvI-SA
Crossing-height error at withheld Δ\Delta (cm) ↓\downarrow2.02.012.712.7
Withheld Δ\Delta placed between Δ=0.35\Delta=0.35 and Δ=0.65\Delta=0.65 ↑\uparrow100%100\%7%7\%
ρ\rho over all 21 values of Δ\Delta ↑\uparrow0.89±0.170.89 \pm 0.170.59±0.180.59 \pm 0.18
RnormR_{\text{norm}} ↓\downarrow1.74±0.831.74 \pm 0.8315.29±7.2215.29 \pm 7.22
Closed-loop success at withheld Δ\Delta ↑\uparrow99.7%99.7\%83.0%83.0\%

MoMo successfully generalizes to the withheld preference levels. Its mean crossing-height error is 2.0 cm relative to the validation trajectories, close to the 1.8 cm obtained when trained on the full dataset. Moreover, 100% of the withheld preference evaluations produce crossing heights between those at Δ=0.35\Delta=0.35 and Δ=0.65\Delta=0.65. Despite the absence of corresponding trajectories in the held-out set, MoMo generates intermediate behaviors by inferring waypoints from the remaining states under its preference-conditioned representations and dynamics. Across the full preference range, MoMo maintains strong preference-cost correlation (ρ=0.89±0.17\rho=0.89\pm0.17) and low curve roughness (Rnorm=1.74±0.83R_{\text{norm}}=1.74\pm0.83), compared with ρ=1.00\rho=1.00 and Rnorm=0.44R_{\text{norm}}=0.44 when trained on the full dataset. These plans also translate to reliable execution, achieving 99.7% closed-loop success at the withheld preferences.

In contrast, IvI-SA struggles to recover the withheld preferences. As discussed in Appendix B.4, state augmentation makes waypoint decoding sensitive to the availability of states with matching preference contexts in the held-out set. Without these candidates, IvI-SA produces plans that cross the divider 7-15 cm below the validation trajectories, corresponding to riskier behavior than requested. Only 7% of its withheld preference evaluations fall between the boundary preferences, and its closed-loop success drops to 83.0%. Even augmenting all decoding candidates with the queried Δ\Delta does not resolve this issue, yielding a 5.5 cm crossing-height error and only 13% correctly placed withheld preferences.

Reviewer_2jfhOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper learns a planner from offline trajectories whose plans can be made safer or more aggressive at query time, by setting a single preference Δ in [0, 1], without retraining. Between the same start and goal, a robot can often take a safe detour or a faster route that passes close to obstacles, and Δ picks a point on that range. The paper builds on contrastive planning by inference (IvI), where states are embedded so that a waypoint plan between start and goal falls out of a closed-form solve in latent space. IvI learns one latent geometry, so each start-goal pair gets one plan.

Core idea: make the whole geometry depend on Δ while keeping the property that makes the closed-form solve valid. The encoder and the latent transition are both conditioned on Δ, and every contrastive comparison is scored under the anchor trajectory's Δ. The paper shows that the learned similarity is then a density ratio conditioned on Δ, so the same closed-form planner applies for each Δ, and changing Δ changes the plan. Simply appending Δ to the state, the obvious alternative, learns a different ratio and breaks this.

How it works: each training trajectory is labelled with how much cost it incurs in its worst steps (a tail average of per-step cost), rescaled to [0, 1]. FiLM layers condition the encoder on Δ, and the latent transition is a shared matrix plus a small Δ-dependent correction. At query time, the start and goal are encoded under Δ, the latent waypoints are solved for, and each is mapped back to the nearest real state.

Experiments: point-mass navigation (three maps), a 29-D Ant robot in an apartment, a drone circling task, and UR5 arm pick-and-place, against IvI and versions of it that append Δ to the state. Plan cost rises with Δ in every environment and seed, waypoint error stays close to IvI's in most environments, and plans change more smoothly with Δ than the baselines'. In closed loop on one point-mass map, tracking the waypoints with a goal-conditioned policy reaches 96.4% success, against 79.0% for MODULI, a preference-conditioned diffusion planner.

Critical strengths

1. The paper turns a preference that already exists in the data into a planner that can be steered at query time.

  • Labels from the data: each trajectory is labelled with its own tail cost, so no preference annotation is needed, and the same state, reached by a cautious and by a risky trajectory, gets a different representation in each (Section 4.1).
  • Planner kept intact: every contrastive pair is scored under its trajectory's label, so the learned similarity stays a density ratio for each Δ, IvI's closed-form waypoint solve still applies, and one trained model serves every preference (Section 4.4, Appendix A.1).

2. Preference-conditioned plans that keep their temporal structure.

  • Conditioning the planner on Δ changes plan cost in the intended direction in every environment and training seed, while waypoint fit stays near the unconditioned planner's in most environments (Figure 4, Table 6, Appendix B.3).
Critical weaknesses

1. MoMo is compared with other preference-conditioned methods only on the simplest task, and its efficiency claim is not measured.

  • The paper's stated gap is preference control that scales to "high-dimensional or long-horizon settings" [p.1, Section 1], but MODULI, CDT and CCAC are run only on Point Four Obstacles, a 4-D point-mass map [p.18, Appendix B.1]. On Ant Habitat and the longer point-mass maps, MoMo is compared only with IvI and its state-augmented variants.
  • On that map, MoMo succeeds more often than MODULI (96.4% against 79.0%), but MODULI's successful rollouts cost less at almost every Δ, for example 9.41 against 25.50 at Δ = 0 [p.19-20, Tables 2 and 4].
  • The paper claims to retain "the inference-time planning efficiency" [p.2, Section 1], yet it reports no planning time, path length or time to goal.
  • Fix: run MODULI on Ant Habitat, and report planning time and path length per Δ for MoMo, IvI and MODULI.
Questions for authors

None

4 comments

Thank you for the detailed comments and proposed fix for this paper. We really appreciate your guidance on how to improve our work!

Scaling preference control in closed-loop (A)

The gap we address is the intersection of planning under unknown dynamics from offline data while enabling preference control at runtime. Through modification of the CRLP algorithm, we claim that under preference-conditioning, we retain the scaling to higher dimensions and longer horizons via the hierarchical approach of inferring intermediate latent waypoints. This can guide low-level policies that are able to navigate safely over shorter horizons. Thus, the primary focus of this paper is placed more on conditioned representation learning for the waypoint inference step rather than closed-loop control. The additional comparisons with CCAC, CDT, and MODULI in the Point Four Obstacles environment are included to demonstrate MoMo's ability to guide action policies for end-to-end preference-conditioned planning in a sample case.

We agree with the reviewer that this claim needs to be reinforced and thus have adopted your proposed fix, comparing the closed-loop approaches for the Ant Habitat environment. Waypoint tracking is performed with a goal-conditioned RL policy trained using SAC. Here, CC is each rollout's mean cost per step. Cost statistics use successful rollouts, while success rates and rollout-step counts use all queries. The reported cost and step uncertainties are standard errors across queries.

Closed-loop comparison against offline learning baselines:

Δ\DeltaMoMo+GCRLMODULICCACCDT
Success ↑\uparrowCost ↓\downarrowSuccess ↑\uparrowCost ↓\downarrowSuccess ↑\uparrowCost ↓\downarrowSuccess ↑\uparrowCost ↓\downarrow
0.058.5%0.228 ±\pm 0.00915.1%0.018 ±\pm 0.00412.1%0.072 ±\pm 0.01428.3%0.185 ±\pm 0.013
0.156.3%0.207 ±\pm 0.00915.5%0.022 ±\pm 0.00512.1%0.097 ±\pm 0.01827.5%0.182 ±\pm 0.013
0.257.9%0.237 ±\pm 0.00915.5%0.020 ±\pm 0.00412.2%0.093 ±\pm 0.01727.1%0.179 ±\pm 0.012
0.360.8%0.264 ±\pm 0.01015.4%0.019 ±\pm 0.00512.0%0.109 ±\pm 0.01826.0%0.179 ±\pm 0.013
0.466.3%0.294 ±\pm 0.00915.5%0.023 ±\pm 0.00611.1%0.077 ±\pm 0.01625.1%0.171 ±\pm 0.013
0.561.9%0.260 ±\pm 0.00915.5%0.019 ±\pm 0.00510.5%0.084 ±\pm 0.01725.2%0.175 ±\pm 0.014
0.665.0%0.282 ±\pm 0.00914.7%0.023 ±\pm 0.00610.7%0.107 ±\pm 0.01924.9%0.174 ±\pm 0.013
0.768.3%0.308 ±\pm 0.01015.3%0.020 ±\pm 0.00510.4%0.101 ±\pm 0.01824.7%0.174 ±\pm 0.014
0.869.1%0.330 ±\pm 0.01015.5%0.028 ±\pm 0.0078.3%0.078 ±\pm 0.01925.1%0.188 ±\pm 0.014
0.969.2%0.329 ±\pm 0.01115.7%0.019 ±\pm 0.0048.7%0.113 ±\pm 0.02224.9%0.174 ±\pm 0.013
1.070.4%0.358 ±\pm 0.01115.3%0.022 ±\pm 0.0057.9%0.109 ±\pm 0.02324.3%0.163 ±\pm 0.013
Mean64.0%--15.4%--10.5%--25.7%--

Closed-loop comparison against state augmentation baselines:

Δ\DeltaIvI-SA+GCRLVIP-SA+GCRLPCA-SA+GCRL
Success ↑\uparrowCost ↓\downarrowSuccess ↑\uparrowCost ↓\downarrowSuccess ↑\uparrowCost ↓\downarrow
0.049.2%0.393 ±\pm 0.00914.4%0.176 ±\pm 0.01667.4%0.337 ±\pm 0.012
0.162.3%0.359 ±\pm 0.00612.9%0.234 ±\pm 0.02067.3%0.340 ±\pm 0.012
0.245.3%0.240 ±\pm 0.00813.4%0.238 ±\pm 0.01968.1%0.344 ±\pm 0.012
0.360.6%0.332 ±\pm 0.00611.7%0.219 ±\pm 0.01968.5%0.347 ±\pm 0.012
0.496.4%0.287 ±\pm 0.00611.7%0.237 ±\pm 0.02068.2%0.345 ±\pm 0.012
0.589.1%0.354 ±\pm 0.00513.4%0.269 ±\pm 0.02268.0%0.342 ±\pm 0.012
0.649.7%0.383 ±\pm 0.00814.1%0.323 ±\pm 0.02268.1%0.345 ±\pm 0.012
0.776.7%0.439 ±\pm 0.00814.2%0.366 ±\pm 0.02168.6%0.351 ±\pm 0.012
0.855.5%0.280 ±\pm 0.00913.2%0.338 ±\pm 0.02169.1%0.355 ±\pm 0.012
0.972.6%0.435 ±\pm 0.00913.5%0.404 ±\pm 0.02167.8%0.343 ±\pm 0.012
1.035.0%0.116 ±\pm 0.00713.7%0.366 ±\pm 0.02467.3%0.338 ±\pm 0.012
Mean62.9%--13.3%--68.0%--
AuthorsOpenReview ↗

Scaling preference control in closed-loop (B)

Trajectory steps across baselines:

Δ\DeltaMoMo+GCRLMODULICCACCDTIvI-SA+GCRLVIP-SA+GCRLPCA-SA+GCRL
0.0106.9 ±\pm 4.2350.5 ±\pm 3.8357.2 ±\pm 3.5288.4 ±\pm 4.6119.4 ±\pm 2.7107.9 ±\pm 4.061.4 ±\pm 2.2
0.1110.7 ±\pm 4.3350.1 ±\pm 3.8357.3 ±\pm 3.5295.3 ±\pm 4.5105.9 ±\pm 2.4118.9 ±\pm 4.060.6 ±\pm 2.1
0.2102.3 ±\pm 4.0348.6 ±\pm 3.9357.9 ±\pm 3.4299.6 ±\pm 4.5106.1 ±\pm 3.8130.9 ±\pm 4.260.9 ±\pm 2.1
0.3102.5 ±\pm 3.8349.2 ±\pm 3.8359.2 ±\pm 3.4303.5 ±\pm 4.489.3 ±\pm 2.8124.6 ±\pm 4.261.2 ±\pm 2.1
0.495.3 ±\pm 3.3348.7 ±\pm 3.8361.8 ±\pm 3.3306.5 ±\pm 4.480.0 ±\pm 2.1127.4 ±\pm 4.061.5 ±\pm 2.2
0.5108.8 ±\pm 4.0349.4 ±\pm 3.8364.6 ±\pm 3.2305.9 ±\pm 4.481.8 ±\pm 2.0132.6 ±\pm 3.961.5 ±\pm 2.2
0.688.6 ±\pm 3.3351.7 ±\pm 3.8364.8 ±\pm 3.2305.4 ±\pm 4.4137.3 ±\pm 4.6139.9 ±\pm 4.161.8 ±\pm 2.1
0.787.9 ±\pm 3.2348.6 ±\pm 3.9365.5 ±\pm 3.2309.7 ±\pm 4.475.4 ±\pm 2.2147.1 ±\pm 4.562.7 ±\pm 2.2
0.8110.2 ±\pm 3.9349.8 ±\pm 3.8373.2 ±\pm 2.8309.2 ±\pm 4.3117.2 ±\pm 3.5153.7 ±\pm 4.463.7 ±\pm 2.2
0.981.5 ±\pm 3.0348.9 ±\pm 3.8371.5 ±\pm 2.9308.1 ±\pm 4.391.8 ±\pm 2.3154.9 ±\pm 4.562.0 ±\pm 2.2
1.069.3 ±\pm 2.3349.7 ±\pm 3.8374.9 ±\pm 2.7309.3 ±\pm 4.3160.7 ±\pm 5.2161.1 ±\pm 4.661.2 ±\pm 2.1

Across the closed-loop evaluation, MoMo achieves a mean success rate of 64.0%, compared with 15.4% for MODULI, 10.5% for CCAC, and 25.7% for CDT. Although MODULI achieves lower costs on successful rollouts, these costs are computed over a much smaller subset of completed queries (15.4% versus 64.0% for MoMo). Therefore, the cost comparison must be considered alongside goal-reaching success and preference modulation. MoMo’s successful-rollout mean cost per step increases from 0.228±0.0090.228\pm0.009 at Δ=0\Delta=0 to 0.358±0.0110.358\pm0.011 at Δ=1\Delta=1, with a strong positive aggregate preference-cost correlation of 0.940.94. This demonstrates that MoMo's preference control translates into executed behavior in a higher-dimensional environment with complex locomotion dynamics. Its endpoint cost change of 0.1300.130 also exceeds the corresponding changes of 0.00380.0038 for MODULI, 0.03790.0379 for CCAC, and −0.0226-0.0226 for CDT. Together, these results show that MoMo completes substantially more queries than these baselines while producing a stronger positive response to the requested preference.

The state-augmentation results further show why task completion and preference control must be assessed together, with no baseline being able to simultaneously achieve the higher success rate, recovered cost range, and preference-modulation correlation of MoMo. IvI-SA achieves 62.9% mean success but with an unstable cost response and a negative endpoint change of −0.2767-0.2767 and correlation of −0.08-0.08. PCA-SA's endpoint cost change is only 0.00070.0007, indicating little response to the requested preference. VIP-SA achieves a strong preference-cost correlation of 0.940.94, but succeeds on only 13.3% of rollouts. MoMo combines a 64.0% success rate with substantial positive cost modulation and a correlation of 0.940.94. Since these waypoint methods use the same GCRL controller, the comparison supports the benefit of MoMo's conditioned representation approach for waypoint generation.

MoMo further shows the associated improvement in trajectory efficiency as risk tolerance increases. MoMo's mean rollout length decreases from 106.9±4.2106.9\pm4.2 steps at Δ=0\Delta=0 to 69.3±2.369.3\pm2.3 at Δ=1\Delta=1. Across preference levels, MODULI, CCAC, and CDT reach much higher trajectory step counts due to timeouts associated with their substantially lower success rates. This indicates the difficulty of jointly learning to complete long-horizon variable-risk navigation under complex locomotion dynamics. These results are consistent with the benefit of MoMo's hierarchical approach, whereby the planner can provide preference-conditioned intermediate waypoints via learned representations to guide a low-level locomotion policy. This leads to successful completion of more queries with fewer environment steps than direct policy-conditioned baselines, while retaining control over the realized cost.

AuthorsOpenReview ↗

Planning time and Path Length

Planning Time

We benchmark MoMo and IvI across all six environments using 40 start-goal queries and 11 preference values per environment. Following the nearest-neighbor decoding procedure of Eysenbach et al. [1], both methods infer latent waypoints and retrieve corresponding states from a held-out decoding set Dheld\mathcal{D}_{\text{held}} containing 250k candidate states.

EnvironmentInference (ms), MoMo / IvIDecoding (ms), MoMo / IvITotal per query after preparation (ms), MoMo / IvI
Point Four Obstacles0.08/0.060.08 / 0.060.07/0.060.07 / 0.060.15/0.120.15 / 0.12
Point Contour0.08/0.060.08 / 0.060.07/0.070.07 / 0.070.15/0.130.15 / 0.13
Point Habitat0.08/0.060.08 / 0.060.08/0.080.08 / 0.080.16/0.140.16 / 0.14
Drone0.10/0.070.10 / 0.070.14/0.110.14 / 0.110.23/0.180.23 / 0.18
Ant Habitat0.53/0.480.53 / 0.480.26/2.390.26 / 2.390.80/2.880.80 / 2.88
UR50.06/0.040.06 / 0.040.04/0.040.04 / 0.040.10/0.080.10 / 0.08

MoMo retains efficient latent waypoint inference and decoding, requiring only 0.100.10–0.800.80 ms per query after preference-specific candidate preparation, with times comparable to IvI across all six environments. Since the representation depends on Δ\Delta, preparing the decoding set for a new preference takes 0.440.44–0.810.81 s, a one-time cost that can be shared across queries at that preference. As discussed in Section 3.2, latent waypoint inference requires a single sparse linear solve rather than repeated optimization over the full execution horizon.

Path length

On UR5, where the riskier path through the divider is also shorter, MoMo's executed path length decreases consistently with Δ\Delta, from 0.8890.889 m to 0.5140.514 m. CDT and MODULI also produce shorter paths at higher preferences, while IvI remains fixed at 0.6220.622 m. On Point Four Obstacles, however, Δ\Delta controls obstacle proximity rather than path length directly, so a monotonic decrease in distance is not expected.

Importantly, MoMo maintains substantially higher goal-reaching success across preferences than MODULI (Appendix B.1). While MODULI produces shorter executed paths, its success decreases from 100%100\% at Δ=0\Delta=0 to 55.8%55.8\% at Δ=1\Delta=1. MoMo maintains at least 85.8%85.8\% success at every preference value, demonstrating that its preference-dependent plans retain reliable execution across the risk spectrum.

[1] Eysenbach, B., Myers, V., Salakhutdinov, R., and Levine, S. (2025). Inference via Interpolation: Contrastive Representations Provably Enable Planning and Inference. https://arxiv.org/abs/2403.04082

We would like to thank the reviewer again for your time and constructive feedback. We hope our responses answer your questions, as they address important points raised in your review. We are happy to provide clarification on anything further during this discussion period.

Reviewer_qGmbOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper tackles the problem of learning temporal contrastive representations for planning. The paper builds on the framework of Contrastive Representation Learning for Planning (CRLP) introduced in prior work, and incorporates the ability of preference-conditioned planning, such as planning conditioned on acceptable risk/safety levels. To do this, the paper introduces MoMo, which represents the preference as a scalar between 0 and 1, and modifies the CRLP training architecture by modulating this scalar with the latent encoder and also modifying the latent transition matrix with this scalar. The paper then empirically shows on a suite of robotics planning tasks that MoMo generates temporally consistent plans that differ appropriately according to the user-specified conditioning.

Critical strengths
  • The core idea of this paper to condition the latent planning on a user-preference is interesting and important. This can help a user to generate diverse plans and select the most appropriate one by simply varying a scalar parameter.
  • Momo seems to generate diverse plans on a variety of different robotics tasks and different agent dynamics.
Critical weaknesses
  • Choice of baselines: The paper only considers state-augmented baselines. Given that the choice to modulate the representation encoder and the transition dynamics is a choice of the paper, the experimental baselines should have included methods that vary other aspects of planning such as conditioning the cost or policy, and hopefully, show that the papers design choice of changing the encoder+transition dynamics generates more preference-sensitive plans than changing the costs. Some potential baselines that the authors might consider for this are (i) IvI repeatedly used for costs corresponding to different preference levels, and (ii) CVaR based planners.
  • With due respect 4 seeds is too less for one to be reasonably confident of the claims. Especially as in Figure 4, for more complex tasks such as drone/ant/UR5, MoMO is barely/not recovering the cost achieved by IvI at any preference level.
Questions for authors
  • Do the authors have any explanation about why in figure 4 for the more complex tasks of drone/ant/UR5, MoMo barely/does not match the IVI waypoint cost at any Δ\Delta?
  • In section 5, for answering Q1, are the Waypoint MSE bars of figure 4 on the right averaged over all tasks/seeds? What value of Δ\Delta was chosen for MoMo for that?

Some presentation suggestions that will strengthen the manuscript in my opinion:

  • Figure 3 is difficult to understand. Please consider highlighting the trajectory of the object/manipulator endpoint so that the cube's relative journey to the gate divider is better visible.
  • Section 3.2 is a bit tough to read with several constants introduced only to be deferred to the appendix, which is confusing to the reader. Please consider rewriting this section to make it less dense and more self-sufficient.
6 comments

We thank the reviewer for their time and effort in providing feedback on our paper. Please find below the responses to the questions raised.

Choice of baselines

Our work addresses preference-conditioned waypoint planning from offline trajectories without requiring a known dynamics model or retraining for different preferences. Specifically, we investigate how preferences can be incorporated into CRLP while retaining the temporal structure that enables efficient inference over intermediate waypoints. While the primary focus of this paper is on the waypoint inference step and not closed-loop control, we present the additional comparisons with CCAC, CDT, and MODULI in the Point Four Obstacles environment (Section 5 and Appendix B.1) to demonstrate MoMo's ability to guide low-level action-generating policies for end-to-end preference-conditioned planning. We are not aware of directly comparable baselines that exist in this gap of waypoint generation through learning from offline data to take user preferences as an input at inference, including in the family of CVaR-based planners.

For this reason, we selected state-augmentation variants of the planning approaches compared by Eysenbach et al. [1] as controlled baselines. Each operates on offline data and produces intermediate waypoints, while state augmentation provides the most direct generic mechanism for adding the preference condition without otherwise changing the underlying planning method. IvI-SA tests this within the same contrastive-planning backbone as MoMo, while PCA-SA and VIP-SA test preference conditioning across different latent planning structures based on principal-component interpolation and temporal-distance representations, respectively. These baselines evaluate the joint effect of MoMo's conditioned objective and architectural modulation, as discussed in Section 3.3 and Appendix B.4.

Following the reviewer's suggestion, we provide an additional baseline using multiple IvI models, which trains separate vanilla IvI models for different preference levels. For each environment, we partition the training trajectories into 11 buckets centered at Δ∈{0,0.1,…,1}\Delta \in \{0,0.1,\ldots,1\}, using each trajectory's preference label. Interior buckets have width 0.10.1, while the endpoint buckets have width 0.050.05. We train one unconditioned IvI model on each resulting subdataset. At inference, a query (s,g,Δ)(s,g,\Delta) selects the model whose bucket center is nearest to Δ\Delta. The selected model receives (s,g)(s,g) and generates the waypoint sequence. For example, Δ=0.12\Delta=0.12 selects the model trained on the bucket centered at 0.10.1.

[1] Eysenbach, B., Myers, V., Salakhutdinov, R., and Levine, S. (2025). Inference via Interpolation: Contrastive Representations Provably Enable Planning and Inference. https://arxiv.org/abs/2403.04082

AuthorsOpenReview ↗

IvI for different preference levels

We evaluate both MoMo and Discrete IvI across 8 training seeds in all six environments, using the metrics defined in Table 6. Decoding is performed by taking the same held out set as MoMo, encoding it using the selected IvI model, and performing nearest neighbor decoding.

EnvironmentMetricMoMoDiscrete IvI
Point Four ObstaclesΔC (↑)\Delta C\,(\uparrow)7.68±1.567.68 \pm 1.56−0.85±0.47-0.85 \pm 0.47
ρ (↑)\rho\,(\uparrow)0.97±0.030.97 \pm 0.030.39±0.130.39 \pm 0.13
Rnorm (↓)R_{\text{norm}}\,(\downarrow)1.54±0.441.54 \pm 0.4415.94±11.9315.94 \pm 11.93
Point ContourΔC (↑)\Delta C\,(\uparrow)2.70±0.352.70 \pm 0.351.23±0.321.23 \pm 0.32
ρ (↑)\rho\,(\uparrow)0.99±0.010.99 \pm 0.010.66±0.150.66 \pm 0.15
Rnorm (↓)R_{\text{norm}}\,(\downarrow)0.98±0.360.98 \pm 0.362.75±1.152.75 \pm 1.15
Point HabitatΔC (↑)\Delta C\,(\uparrow)8.35±1.408.35 \pm 1.402.36±3.102.36 \pm 3.10
ρ (↑)\rho\,(\uparrow)0.93±0.030.93 \pm 0.030.41±0.090.41 \pm 0.09
Rnorm (↓)R_{\text{norm}}\,(\downarrow)1.30±0.461.30 \pm 0.4615.07±16.8015.07 \pm 16.80
DroneΔC (↑)\Delta C\,(\uparrow)2.86±0.952.86 \pm 0.95−0.92±1.08-0.92 \pm 1.08
ρ (↑)\rho\,(\uparrow)0.49±0.240.49 \pm 0.24−0.16±0.16-0.16 \pm 0.16
Rnorm (↓)R_{\text{norm}}\,(\downarrow)4.52±1.824.52 \pm 1.8228.94±26.1928.94 \pm 26.19
Ant HabitatΔC (↑)\Delta C\,(\uparrow)3.06±0.773.06 \pm 0.770.51±1.420.51 \pm 1.42
ρ (↑)\rho\,(\uparrow)0.73±0.200.73 \pm 0.200.12±0.320.12 \pm 0.32
Rnorm (↓)R_{\text{norm}}\,(\downarrow)2.34±0.642.34 \pm 0.6441.62±53.8541.62 \pm 53.85
UR5ΔC (↑)\Delta C\,(\uparrow)1.26±0.161.26 \pm 0.160.80±0.110.80 \pm 0.11
ρ (↑)\rho\,(\uparrow)1.00±0.001.00 \pm 0.000.86±0.100.86 \pm 0.10
Rnorm (↓)R_{\text{norm}}\,(\downarrow)0.40±0.180.40 \pm 0.183.21±0.463.21 \pm 0.46

MoMo achieves a larger mean endpoint cost change, higher preference-cost correlation, and lower normalized roughness across all six environments. For instance, on Ant Habitat, Discrete IvI exhibits weak correlation, 0.12±0.320.12\pm0.32, with a limited cost range, 0.51±1.420.51\pm1.42, while MoMo achieves 0.73±0.200.73\pm0.20 and 3.06±0.773.06\pm0.77, respectively. Discrete IvI also exhibits considerable variation across seeds, consistent with the reduced temporal coverage available when each model is trained on a subset of trajectories. Discrete IvI is more effective on UR5, reaching a correlation of 0.86±0.100.86\pm0.10, although MoMo achieves 1.00±0.001.00\pm0.00 and lower normalized roughness, 0.40±0.180.40\pm0.18 versus 3.21±0.463.21\pm0.46. The improvement is therefore consistent across the reported metrics, but the extent varies by environment.

These differences are likely attributable to the reduced state and transition coverage available to each IvI model after partitioning the dataset based on preference labels. Each model must independently learn representations from a subset of trajectories, which may provide insufficient temporal information. MoMo trains jointly on the full dataset, allowing its shared representation to learn temporal relationships across all observed preference contexts. MoMo conditions its encoder and latent transition operator on the queried preference, allowing it to adapt the planning representation while sharing temporal information across the full training dataset. These results are consistent with the benefit of sharing temporal information across preferences while adapting the representation used for each query.

AuthorsOpenReview ↗

Seeds and recovering cost across preference levels

To strengthen confidence in the paper's claims, we have doubled the number of random seeds from 4 to 8 and updated the reported results accordingly.

Regarding the reviewer's first question on Figure 4, IvI's waypoint cost is the output of a preference-agnostic planner trained on the mixture of observed behaviors in the dataset. Since preference labels describe dataset-relative behavior, they do not prescribe a cumulative waypoint cost. IvI's constant cost for a given query therefore serves as a reference rather than a target cost for MoMo at any particular Δ\Delta, since MoMo does not associate IvI’s unconditioned solution with a specific Δ\Delta during training or planning. Importantly, MoMo maintains a consistent preference-dependent cost response across the full range of Δ\Delta, which is the intended objective rather than recovering IvI's unconditioned cost.

In comparison to IvI-SA, Figure 4 must be considered across its two components. First, the waypoints must retain the temporal relationships needed to reflect a feasible trajectory from start to goal, measured through reconstruction MSE adopted from Eysenbach et al. [1]. Second, waypoints must respond to the preference provided by the user. In response to the reviewer's second question, the MSE bars of Figure 4 are provided for a single representative seed, and Table 6 reports the variations in preference-modulation metrics across training seeds. To compute the reconstruction MSE, we take the start, goal, and preference label as a tuple from each trajectory in a held-out set. We then use each tuple as a query to the planner and compare the inferred intermediate waypoints to the held-out trajectory to measure the deviation. Successful preference-conditioned planning is therefore assessed across these two aforementioned axes. For instance, a model can produce a large cost range without satisfying the first temporal requirement.

In state augmentation, representations may distinguish positive and negative pairs simply via preference agreement without adequately encoding which states follow each other, as discussed in Appendix B.4. Such a representation may group states by preference and produce a cost trend, while reconstructing intermediate trajectories poorly and thus not holding meaningful information for planning.

The Ant result illustrates this effect. IvI-SA produces a larger cost range than MoMo, 14.77±13.3314.77\pm13.33 versus 3.06±0.773.06\pm0.77, but its reconstruction MSE in Figure 4 is 38.638.6 times IvI’s. Similarly, VIP-SA achieves a higher preference-cost correlation, 0.77±0.340.77\pm0.34 versus 0.73±0.200.73\pm0.20, but its reconstruction MSE is 239.9239.9 times IvI’s. MoMo maintains IvI-comparable waypoint reconstruction MSE while producing a smoother preference response as shown in Figure 4 and Table 6. Thus, a larger cost range or higher correlation alone does not establish better planning performance, particularly when accompanied by substantially higher waypoint reconstruction error.

[1] Eysenbach, B., Myers, V., Salakhutdinov, R., and Levine, S. (2025). Inference via Interpolation: Contrastive Representations Provably Enable Planning and Inference. https://arxiv.org/abs/2403.04082

Comments on the paper

1 comment

We would like to thank the reviewers for their careful assessment and constructive feedback. As the review discussion period is concluding, we would like to summarize the additional work we have provided to address the reviews.

  1. We have expanded the closed-loop evaluation to Ant Habitat and UR5. While the central focus of this paper is on conditioned waypoint generation, we acknowledge the importance of closed-loop performance. To our knowledge, this is the first approach to enable runtime preference-conditioned waypoint generation for a given start-goal query by directly conditioning learned representations, without retraining. To demonstrate the usefulness of our approach for downstream robotics applications, we additionally evaluate the generated waypoints with GCRL policies in Ant Habitat and UR5. Across these environments, MoMo achieves higher goal-reaching success rates than the end-to-end methods MODULI, CCAC, and CDT. It achieves mean goal-reaching success rates of 64% on Ant Habitat and 100% on UR5, exceeding all of the end-to-end methods in both environments, with Spearman correlations of 0.94 and 1.00, respectively.
  2. We have added a Discrete IvI baseline, where we train 11 separate vanilla IvI models per environment, each on a different preference bucket. MoMo recovers a larger mean endpoint cost change and higher mean preference cost correlation across all experiments, highlighting the advantage of learning preference-dependent deformations of the latent manifold to perform conditioned planning.
  3. We have doubled the number of training seeds from four to eight in our preference-modulation evaluation. This expanded evaluation still confirms the original findings on MoMo’s preference-modulation capabilities.
  4. We have added a study on generalization to new start-goal combinations. This study considered the composition of trajectories by directly selecting start-goal pairs whose endpoints did not occur together in any dataset trajectory. The inferred plans combine waypoints from an average of 8-10 source trajectories, supporting compositional waypoint generation beyond conditioned retrieval from a single demonstration.
  5. We have added a study on withheld preference ranges, where we use a dataset of UR5 trajectories that excludes preference labels between 0.4 and 0.6 from both the training and held-out set. We then tested MoMo’s ability to support preference queries in the excluded range, where near-perfect closed-loop success was achieved. This highlights the generalizability of conditioned latent manifolds to support planning on held-out preferences.
  6. We have added a study on the sensitivity of MoMo to preference labels, where we add noise at different standard deviations and use different CVaR quantiles to construct preference labels. Every setting maintains 100% closed-loop success and cost correlation of at least 0.94.
  7. We tested recalibration of the latent waypoint posterior to address the covariance drift from the isotropic Gaussian assumption on the marginal distribution of representations, finding that the recalibration does not improve waypoint reconstruction or preference modulation over MoMo.
  8. We added planning time measurements to support the claim of fast planning at inference and path length trends with the preference value. Across six environments, inference and decoding take 0.1-0.8 ms per query after candidate preparation. On UR5, executed path length decreases as the preference value increases, as expected as well.
  9. We clarified the theoretical argument, explaining the connection between the conditioned objective and the density-ratio property. This includes explanations for how waypoint inference can be performed from the Gaussian marginal and Markov assumptions.

We hope these additional experiments are sufficient to address the reviewers’ feedback as the discussion period concludes.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.