acceptodds
Under review as a conference paper at ICLR 2027

Latent Space Planning with Sequential Monte Carlo

Abstract

Planning with learned world models often relies on trajectory optimization on latent rollouts of action sequences through differentiable dynamics. Stochastic variants inject noise into these updates to escape spurious basins, but unlike zero-order planners, they do not maintain a population. In contrast, sequential Monte Carlo (SMC) performs population-level inference, where importance weighting and resampling progressively steer a candidate population toward regions favored by the target distribution and can be combined with gradient frameworks. In this work, we propose latent-space planning as SMC, enabling inference through steering and optimization. Because the cost determines where the population is steered, we also study how costs influence planning, specifically, the () objective typically used by planners and a structured cost inspired by local progress measures from classical planners. Across five environments and two world models, we find that SMC improves goal-reaching success over zero-order, gradient-based, and stochastic gradient-based planners, with the effect of the cost structure becoming more pronounced in more complex settings. We find that SMC improves average goal-reaching success by upto , and percentage points over zero-order, gradient-based and stochastic gradient-based planners, respectively.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World Models, latent space planning, sequential monte-carlo

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_WT5HOpenReview ↗
Rating
2
Confidence
2(−1)
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary
  • The paper applies SMC to latent-space planning of a pretrained world model, refining candidate action sequences through stochastic updates, importance weighting, and resampling.
  • It also constructs a latent graph from offline data and derives a new cost function.
  • Experiments show that the proposed planner achieves higher average goal-reaching success rates than gradient descent, CEM, and GRASP, using the same proposed cost function.
Critical strengths
  • The motivation and approach for combining exploration over multiple candidate action sequences with gradient-based optimization through SMC are clear.
  • The method’s effectiveness is demonstrated with two different pretrained world models.
Critical weaknesses
  • Unclear algorithmic contribution of the SMC planner. Existing work [1] already combines gradient-guided proposals with particle weighting and resampling. The paper should clarify which aspects of its SMC planner are new beyond applying these components to action-sequence planning.
  • Incomplete experimental analysis. The cost ablation covers only the proposed planner. It remains unclear whether SMC retains its advantage under the standard L2 cost and whether the proposed cost similarly improves competing planners.

[1] Raghav Singhal et al. “A General Framework for Inference-time Scaling and Steering of Diffusion Models.” ICML, 2025.

Questions for authors
  1. Could the authors clarify how the proposed SMC planner builds on existing gradient-guided SMC methods and which algorithmic aspects are new?
  2. Does SMC retain its relative advantage under the L2 cost? Does the proposed cost also improve GD, CEM, and GRASP?
  3. Minor clarification (not counted in the assessment): Table 2 reduces N from 3 to 2 for DINO-WM at H=15, making resampling impossible with τ=0.5 since ESS≥1. Is resampling intentionally disabled in this setting, or is a different resampling rule used?
6 comments

W1/Q1 — algorithmic contribution of the SMC planner

We thank the Reviewer for their detailed assessment and take this opportunity to firstly clarify that SMC is not the planner, but a key component within a stochastic optimization framework that forms the planner. As the reviewer correctly points out, the reweighting and resampling paradigm has been used for inference-time steering and distribution tilting in generative models, and more recently extended to optimization problems. This literature spans three lines.

(1). Inference-time steering and reward alignment in generative models [1, 2], with related work on diffusion-based sampling [3, 4] and on training latent diffusion models [5].

(2). SMC as a component of stochastic optimization, which is the line we build on [6, 7, 8], resting on the SMC samplers of [9], annealed flow transport [10] and sequential controlled Langevin diffusions [11].

(3). SMC in planning, control and reinforcement learning [12, 13, 14].

A key challenge with applying SMC over action spaces is that robots may have different action spaces (e.g., a 2D/3D navigation robot vs. a 7-DoF manipulator), requiring the kernels to be re-defined, thus affecting their generalizability. Reformulating SMC within the JEPA latent space instead achieves zero-shot generalizability to different environments. We show that neither the cost function nor the SMC planner needs to be redefined under new environments, making this a fully generalizable form of SMC optimization.

We therefore reformulate the Expectation-Maximization paradigm to optimize (latent) action sequences. The SMC sits as a key component within this framework. Our modeling paradigm defines the kernel by treating an entire action sequence as a single particle, thus operating in a higher-dimensional space where the environment and robot intrinsics do not require redefining the kernel, enabling zero-shot generalizability. Our objective is not just inference-time tilting, but stochastic optimization using the weights proposed by SMC.

References

[1] Ψ\Psi-Sampler: Initial Particle Sampling for SMC-Based Inference-Time Reward Alignment in Score Models. Taehoon Yoon, Yunhong Min, Kyeongmin Yeo, Minhyuk Sung. Advances in Neural Information Processing Systems, 2026.

[2] Inference-Time Scaling of Discrete Diffusion Models via Importance Weighting and Optimal Proposal Design. Zijing Ou, Chinmay Pani, Yingzhen Li. arXiv:2505.22524, 2026.

[3] Reverse Diffusion Sequential Monte Carlo Samplers. Luhuan Wu, Yi Han, Christian Andersson Naesseth, John Cunningham. Advances in Neural Information Processing Systems, 2026.

[4] Diffusion Path Samplers via Sequential Monte Carlo. James Matthew Young, Paula Cordero-Encinar, Sebastian Reich, Andrew Duncan, O. Deniz Akyildiz. arXiv:2601.21951, 2026.

[5] Training Latent Diffusion Models with Interacting Particle Algorithms. Tim Y. J. Wang, Juan Kuntz, O. Deniz Akyildiz. arXiv:2505.12412, 2026.

[6] Learning Latent Variable Models via Jarzynski-adjusted Langevin Algorithm. James Cuin, Davide Carbone, O. Deniz Akyildiz. Advances in Neural Information Processing Systems, 2025.

[7] Efficient Stochastic Optimisation via Sequential Monte Carlo. James Cuin, Davide Carbone, Yanbo Tang, O. Deniz Akyildiz. arXiv:2601.22003, 2026.

[8] Utilising Gradient-Based Proposals within Sequential Monte Carlo Samplers for Training of Partial Bayesian Neural Networks. Andrew Millard, Joshua Murphy, Zheng Zhao, Simon Maskell. ICASSP, 2026.

[9] Sequential Monte Carlo Samplers. Pierre Del Moral, Arnaud Doucet, Ajay Jasra. Journal of the Royal Statistical Society Series B, 2006.

[10] Annealed Flow Transport Monte Carlo. Michael Arbel, Alexander G. D. G. Matthews, Arnaud Doucet. arXiv:2102.07501, 2021.

[11] Sequential Controlled Langevin Diffusions. Junhua Chen, Lorenz Richter, Julius Berner, Denis Blessing, Gerhard Neumann, Anima Anandkumar. arXiv:2412.07081, 2025.

[12] Probabilistic Planning with Sequential Monte Carlo Methods. Alexandre Piché, Valentin Thomas, Cyril Ibrahim, Yoshua Bengio, Chris Pal. International Conference on Learning Representations, 2018.

[13] Sequential Monte Carlo for Policy Optimization in Continuous POMDPs. Hany Abdulsamad, Sahel Mohammad Iqbal, Simo Sarkka. Advances in Neural Information Processing Systems, 2026.

[14] Tempered Sequential Monte Carlo for Trajectory and Policy Optimization with Differentiable Dynamics. Heng Yang. arXiv:2604.21456, 2026.

Does SMC retain its relative advantage under the L2 cost? Does the proposed cost also improve GD, CEM, and GRASP?

We address this point in Section 4.4 and Figure 6, where every environment and horizon is run under both cost structures. Under the L2L_2 cost our planner averages 72.6% over the twelve settings, ahead of GD by 21.7 points, CEM by 27.3 and GRASP by 42.7, and level with MPPI at 72.3%. The PDE cost adds 7.2 points on average and takes the planner to 79.8%.

On whether the cost helps the baselines, Appendix G.3 and Figure 8 report GD, CEM and MPPI under the same roadmap cost on nine paired cells of 100 seeds each. The only significant change is GD on Reacher at H=15H{=}15, from 15% to 55% (p<0.001p<0.001), and the other eight cells stay within 9 points with p≥0.21p\ge0.21. GRASP cannot take the cost, since its objective never evaluates a goal term and its terminal state is pinned at the goal, so a cost-to-go there is zero by construction. We are happy to elaborate further if specific aspects are unclear to the Reviewer.

Minor clarification — resampling at N=2N{=}2

Resampling is not disabled, but is done when ESS<τN\mathrm{ESS}<\tau N with ESS=1/∑iwi2\mathrm{ESS}=1/\sum_i w_i^2, the standard rule. At N=2N=2, this rule can never fire since ESS≥1=τN\mathrm{ESS}\ge 1=\tau N. We thank the reviewer for the catch. We also note that in the revision, every planner holds 12 particles in every environment and horizon, so this configuration no longer appears.

We sincerely appreciate the time and effort the reviewers have devoted to reading and evaluating our paper and rebuttals. We hope our responses and additional experiments have addressed their concerns, and we would be happy to provide further clarifications or answer any additional questions the reviewers may have.

Reviewer_RfjdOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper considers planning using latent space world models using sequential monte carlo. The authors define the problem of planning using a latent space world model and a cost function. Where previous methods have used "zero order models", i.e. direct optimization, the authors introduce a stochastic population using SMC. The authors also consider the problem of defining a useful cost loss surface, since the topology of the world model may not be encoded in latent space as expected. To this end, the authors introduce a graph based method, based on encoded real world states and their associated costs, to come to a usable loss in latent space. The authors then apply their solutions to a series of benchmarks.

Critical strengths
  • The method seems to work on the benchmarks discussed.
  • The presentation is clear.
  • The method is a sensible expansion of prior work, by introducing a population of particles rathe than direct optimization or single particle stochastic methods
Critical weaknesses
  • The quality of the contribution is good, but overall seems somewhat small.
Questions for authors
  • Generally the way I am used to SMC, it runs over the time axis t, i.e. the t in vec(a) = (a0 ... aT) for example like in Piche et al (2018), but that does not seem to be the case here, it seems to be optimizing plans in an outer loop. While not necessarily a problem, I'm curious if the authors can point to other work where SMC is simply used for optimization? And in this case, how is the K chosen? How does one know when to stop?
  • The author's latent loss has a different conclusion on pusht vs pointmaze. I think I might understand why this is the case for pointmaze, but it's not clear to me why the result in PushT is worse. Can the authors elaborate and explain? I'd like to hear the author's take on both aspects (better on pointmaze AND worse on PushT).
  • In the experiments in Tab. 1, how many particles are used? And is GD given the appropriate extra amount of compute in terms of starting points or optimization steps?
4 comments

We thank the reviewer for the constructive feedback and the thoughtful questions. We respond to each point below.

Q1

The reviewer's reading is correct. Our SMC steps are planner iterations, not environment steps, and each particle is a complete action sequence.

Among works that leverage SMC for optimization problems, the closest are SOSMC [1], which runs SMC over a sequence of targets indexed by the parameter that an optimizer updates. JALA-EM [2] does the same for maximum marginal likelihood estimation.

Further examples are [3], which uses gradient-based SMC proposals to train partial Bayesian neural networks, [4], which trains latent diffusion models with interacting particle algorithms, and [5], which places SMC samplers inside variational inference for state space models. In control, [6] uses SMC for policy optimization in continuous POMDPs and [7] for trajectory and policy optimization with differentiable dynamics. The SMC samplers of [8] and annealed flow transport [9] provide the underlying sampler in each case.

In our planner, we treat KK as a fixed compute budget and cap it at 150 iterations for every planner. An optimization terminates on the first success. The trajectory is evaluated by a simulator after every optimization step. The simulator uses predefined environment configurations to determine whether a plan succeeds. Thus, the optimization terminates either when KK reaches 150 steps or when the simulator flags the trajectory as a success (Section 4.1).

Q2

We observe that the roadmap cost helps when Euclidean distance in the latent space does not represent the true path distance. Point maze and Push-T differ in this manner. In Point maze, motion is restricted by walls, so its geodesics differ from Euclidean distances. Two states can lie close together in the latent space yet many steps apart in the environment. As a result, connectivity through the data carries information that a straight line does not. In Push-T, the T is moved through relatively unobstructed space, so latent Euclidean distance already orders states by their distance to the goal about as well as the roadmap does. Therefore, the cost-to-go adds nothing extra.

We measured this directly. Over 60 held-out goals, the Spearman correlation of each ordering with the observed number of steps to the goal is 0.807 for the roadmap cost against 0.422 for Euclidean distance on Point maze, with a paired gain of +0.385±0.051+0.385\pm0.051, and +0.208±0.066+0.208\pm0.066 on Reacher. On Push-T, the two are indistinguishable, 0.595 against 0.608, with a paired difference of −0.013±0.037-0.013\pm0.037.

References

[1] Efficient Stochastic Optimisation via Sequential Monte Carlo. James Cuin, Davide Carbone, Yanbo Tang, O. Deniz Akyildiz. arXiv:2601.22003, 2026.

[2] Learning Latent Variable Models via Jarzynski-adjusted Langevin Algorithm. James Cuin, Davide Carbone, O. Deniz Akyildiz. Advances in Neural Information Processing Systems, 2025.

[3] Utilising Gradient-Based Proposals within Sequential Monte Carlo Samplers for Training of Partial Bayesian Neural Networks. Andrew Millard, Joshua Murphy, Zheng Zhao, Simon Maskell. ICASSP, 2026.

[4] Training Latent Diffusion Models with Interacting Particle Algorithms. Tim Y. J. Wang, Juan Kuntz, O. Deniz Akyildiz. arXiv:2505.12412, 2026.

[5] Scalable Bayesian Learning for State Space Models using Variational Inference with SMC Samplers. Marcel Hirt, Petros Dellaportas. International Conference on Artificial Intelligence and Statistics, 2019.

[6] Sequential Monte Carlo for Policy Optimization in Continuous POMDPs. Hany Abdulsamad, Sahel Mohammad Iqbal, Simo Sarkka. Advances in Neural Information Processing Systems, 2026.

[7] Tempered Sequential Monte Carlo for Trajectory and Policy Optimization with Differentiable Dynamics. Heng Yang. arXiv:2604.21456, 2026.

[8] Sequential Monte Carlo Samplers. Pierre Del Moral, Arnaud Doucet, Ajay Jasra. Journal of the Royal Statistical Society Series B, 2006.

[9] Annealed Flow Transport Monte Carlo. Michael Arbel, Alexander G. D. G. Matthews, Arnaud Doucet. arXiv:2102.07501, 2021.

Q3

Table 1 uses 12 particles per planner, with a ceiling of 150 iterations. For GD, this is 12 candidate action sequences per iteration with the gradient averaged over them. This amounts to 1800 world-model rollouts and 150 Adam steps per trial. On Point maze at H=10H{=}10, we find that an iteration costs 0.87 s for GD, against 0.78 s for ours. We evaluate GD under two settings, a 12-candidate optimization and the single particle of its released configuration, to ensure fairness to the corresponding models. We find that GD improves with 12 candidates, indicating that the equal-particle protocol is the more favorable of the two for GD. Nevertheless, our approach outperforms GD in both settings, by 28.8 percentage points at 12 candidates and by 43.1 at the single particle of its released configuration. Appendix G.3 reports the other baselines in the same way, where we lead GRASP by 52.9 points at its released single particle and CEM by 39.1 points at an equal number of world-model rollouts.

We sincerely appreciate the time and effort the reviewers have devoted to reading and evaluating our paper and rebuttals. We hope our responses and additional experiments have addressed their concerns, and we would be happy to provide further clarifications or answer any additional questions the reviewers may have.

The authors have addressed my concerns but I will maintain my score.

Reviewer_en8BOpenReview ↗
Rating
3(+2)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a population-based planner for frozen world models. Action sequences are treated as particles, moved by noisy gradient updates, reweighted by their planning costs, and resampled when the effective sample size is low. A roadmap constructed from encoded offline data supplies an additional latent cost-to-go. The method is motivated as SMC inference toward a cost-defined distribution and evaluated against gradient descent, GRASP, and CEM across five environments using two world-model families.

Critical strengths
  1. The algorithm combines complementary planning operations in a useful way. Section 3.3 uses gradient-based moves to improve individual action sequences and cost-based weighting and resampling to select among the population. Separating the differentiable move objective LL from the weighting cost CC also gives a concrete role to richer costs in population selection. Section 3.4's roadmap cost addresses a meaningful limitation of latent Euclidean distance: proximity in representation space need not reflect reachability through the environment. These are worthwhile design choices even though the claimed target-distribution interpretation remains unresolved.

  2. The experiments show substantial gains in some settings and expose relevant limits. Table 1 reports 92% success on Reacher at H=10H=10, compared with 38% for both GD and CEM. The study spans two world-model families, and Figure 6 separately examines the effect of the roadmap cost. The paper also reports cases where CEM is better, including Point maze and Wall at H=15H=15, and cases where the roadmap cost hurts. This provides useful evidence about the proposed planner's behavior rather than only presenting favorable examples.

Critical weaknesses

W1. The stated SMC target is not established by the proposed updates. Equation (1) defines π(a)∝q0(a)exp⁡[−C(a)]\pi(a)\propto q_0(a)\exp[-C(a)], and Section 3.2 claims that the terminal distribution equals this target. Equation (6), however, repeatedly multiplies weights by exp⁡[−C(ak−1)/α]\exp[-C(a_{k-1})/\alpha], without showing the change-of-measure correction needed to obtain that marginal under the move kernel. In the identity-move limit, KK such updates produce a distribution proportional to q0(a)exp⁡[−KC(a)/α]q_0(a)\exp[-KC(a)/\alpha], which generally differs from Equation (1). Appendix A.3 establishes a Feynman–Kac representation weighted by costs along the entire optimization path, but it does not establish that its terminal marginal is the target in Equation (1). Thus, the algorithm may be a useful population optimizer, but the central fixed-target inference justification does not follow from the analysis given.

W2. The main performance comparison does not establish comparable computation. Table 1 describes a matched 150-iteration budget, but an iteration has different costs for the compared algorithms. Appendix B uses eight particles for Reacher, so 150 iterations allow 1,200 particle updates before accounting for smoothing and differentiation. Appendix C describes CEM as 30 candidates over five outer iterations, or 150 total rollouts, while also stating that all planners receive 150 iterations. These descriptions do not specify a common computational budget. This matters directly to the claimed benefit of the population planner: the success gains may partly reflect additional search computation. The iteration curves in Figure 4 do not resolve this, since their horizontal axis is not a shared unit of work. Together with W1, this prevents the current formulation and evaluation from supporting acceptance.

Questions for authors

Q1 (W1). What is the terminal action-sequence distribution induced by Equations (5)–(6), and under what conditions does it equal Equation (1)? Please reconcile the repeated-cost weighting, including the identity-move limit, with the fixed target and the path-space result in Appendix A.3.

Q2 (W2). What exact CEM and particle-planner budgets produced Table 1, and how many world-model evaluations and gradient computations do they entail? Do the main success-rate gains remain at comparable computation or decision time, with all planners using the same reported cost-to-go objective?

5 comments

Q1 (W1) — the terminal action-sequence distribution and the repeated-cost weighting

We thank the Reviewer for their thorough feedback. We have revised the exposition in Section 3 to align the notation and derivation with the Appendix and the established formulations in [1] and [2].

Specifically, our SMC move and weight recursion follow Eqs. (6) and (7) of Proposition 1 in [1]. The optimization step follows from their Eq. (8). Their Remark 1 further identifies this recursion as a special case of Eq. (12) in [3]. The same recursion is established in [2], Eq. (8) and Theorem 1, which we adopt in our Eq. (4). The derivations in Appendices A.2 and A.3 follow the corresponding treatment in [2].

Crucially, Eq. (4) of [1] establishes the property requested by the reviewer: the weighted ensemble is unbiased with respect to the target distribution at every step, for any sequence of potentials with finite normalizing constants. Thus, at the final step, the ensemble is unbiased with respect to our target in Eq. (1), evaluated at the current plan. Moreover, in the identity-move limit, the incremental weight in Eq. (7) of [1] reduces to the ratio of consecutive targets. The resulting weights telescope, ensuring that the cost is applied only once, regardless of the number of iterations. The accumulation the reviewer describes would arise from dividing the current target by the proposal distribution at every step. Our weights divide it by the previous target instead, so the cost does not accumulate.

Distinction from prior work. The key difference is the optimization variable: [1] optimizes model parameters, whereas we optimize the plan itself. Consequently, the target changes between iterations because the reference plan changes. Line 3 of Algorithm 1 accounts for this change in closed form.

Q2 — budget and comparable computation

To generate Table 1, we consider the following settings for CEM and other planners. We use 12 candidates and 150 iterations for all baselines. For both CEM and MPPI, this is 1800 model calls and no gradient computations. For GD, it is 1800 model calls and 150 gradient backpropagations, each differentiating that iteration's 12 rollouts. For GRASP, it is 1825 model calls and 175 gradient backpropagations, with the extra 25 being for the full-gradient refinement that its released configuration runs once after iteration 100.

In contrast, for ours, we have the same 1800 model calls, each differentiated, and 150 gradient optimization steps. Thus, each iteration consists of an equal number of model calls and gradient computations.

We evaluate success under an identical optimization objective, computed over 150 iterations. However, because these models are parallelized differently, wall-clock times do not accurately reflect the compute budget. Nevertheless, we computed wall-clock time on a representative setting (Point-Maze H=10) and observe that an iteration of the planning loop takes 0.13 s for GRASP, 0.45 s for CEM, 0.78 s for ours, and 0.87 s for GD. Crucially, we note that capping on wall-clock time instead of iterations would allocate varying compute budgets to the planners, making their optimization settings non-identical.

In our main results, we highlight this by presenting table results under identical optimization iterations and candidate/particle counts. In Appendix G.3, we present GD and GRASP under the single particle of their released configurations and CEM at an equal number of world-model rollouts. Averaged over the twelve settings, our planner is ahead by 7.4 to 49.8 percentage points under the equal-particle protocol and by 39.1 to 52.9 points under those configurations.

References

[1] Learning Latent Variable Models via Jarzynski-adjusted Langevin Algorithm. James Cuin, Davide Carbone, O. Deniz Akyildiz. Advances in Neural Information Processing Systems, 2025.

[2] Efficient Stochastic Optimisation via Sequential Monte Carlo. James Cuin, Davide Carbone, Yanbo Tang, O. Deniz Akyildiz. arXiv:2601.22003, 2026.

[3] Sequential Monte Carlo Samplers. Pierre Del Moral, Arnaud Doucet, Ajay Jasra. Journal of the Royal Statistical Society Series B, 2006.

Reviewer_en8BOpenReview ↗

Response to Authors

Thanks for the response and the revised paper. The new formulation addresses the main mathematical issue I raised, and the added experiments make the results more convincing.

Equation (4) and Algorithm 1 now account for both the changing target and the particle move. In the identity-move limit, the target ratios cancel across iterations, addressing the repeated-cost issue. However, the claim that the ensemble is unbiased at every step is too strong. Appendix A.3 already notes that finite-particle guarantees have not been established for the adaptive targets used here. The cited identities also do not establish unbiasedness of the finite, self-normalized particle estimate. Section 4.1 of Cuin et al. (2025) discusses this bias. Please make sure the claim is consistent with these limits.

In the revised version, I find that the target, particle moves and weight updates differ quite a bit from the initial submission. Did the original implementation already use these updates, or did you change the algorithm for the revised experiments? Please say which results were rerun and confirm that the revised Table 1 uses the updates now described.

For Q2, Table 1 now uses 12 candidates and up to 150 main-loop iterations for each method. This resolves my concern about the population and iteration budgets in the main comparison. Adding MPPI is also useful.

The claim of an identical optimization objective still needs correction. Table 1 uses the PDE cost for your planner, while the baselines mainly use goal distance. With the same estimator choices as Table 1, the L2 entries in Table 9 average 72.6%, close to MPPI's 72.3%. The 7.4-point gain in the main table therefore reflects the planner and cost design together. Figure 8 adds useful evidence here. Combining its reported changes with Table 1 gives 93% versus 81% on Push-T at H=5H=5, and 89% versus 69% on Reacher at H=15H=15, for your method and MPPI with the PDE cost. These comparisons support the benefit of the combination.

One budget inconsistency remains in Section 4.1. In Appendix G.3, CEM has a budget of 30×5=15030\times5=150 rollouts, while your method keeps the 12×150=180012\times150=1800 budget from Table 1. Please correct the description of this comparison as having equal rollout budgets.

Overall, I think the revision gives stronger support for the contribution. I'd be willing to move to a positive recommendation once these remaining points are clarified.

We thank the reviewers for the productive feedback and discussion. In the following response, we address the points put forth by the reviewer.

P1 Unbiasedness

We thank the reviewer for the correction regarding unbiasedness claims in our formulation, and we have made sure that the paper reflects this throughout.

P2 Table 1 Results

In the original submission, the reported numbers came from an earlier implementation of the planner, whose moves were unadjusted Langevin steps on the planning objective. To ensure completeness of our experiments, we investigate the extent to which the kernel choice impacts performance. To this end, we also compared the different kernels mentioned in Appendix G.1 within Algorithm 1, where JALA achieves the highest success rate in four of the five settings. Consequently, the main paper reports JALA throughout. Similarly, we experimented with various optimization settings to determine what would be fairest to the baselines, and converged on the setting with 12 particles and 150 optimization steps. Thus, following the suggestions of the reviewers, we have updated every cell in Table 1 to reflect performance under this setting. We have also updated Figures 6 and 9 accordingly. We confirm that Table 1 uses Equations (3), (4) and (6), and Algorithm 1 as printed.

P4 Optimization objective

In order to highlight the performance under both cost structures, we have updated Table 1 by adding a row for our planner under the L2L_2 cost. We have also updated the “Baselines” paragraph of Section 4.1 to reflect what Table 1 uses as the objective for each baseline. We reported the L2L_2 numbers for the baselines in Table 1. For completeness, Appendix G.4 reports the results of applying our PDE cost to GD, CEM and MPPI at the same 12-candidate budget, paired with their Table 1 runs by seed.

P5 CEM budget wording

We have updated the CEM budget statement accordingly.

Summary of changes to the paper

  • Appendix and Section 3 do not contain strong unbiasedness claims.

  • Table 1 adds a row for our planner under the L2L_2 cost, and both rows are labelled by their cost.

  • Section 4.1 states that each baseline optimizes the objective reported by its authors, and Appendix G.4 examines the effect of our PDE cost on GD, CEM and MPPI.

  • Section 4.1 describes the CEM comparison of Appendix G.3 as at an equal rollout budget to a single-particle gradient planner.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.