- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper deals with sample-based planning in continuous control with latent world models. The paper first demonstrates the planners are subject to the phenomenon of proposal overgeneration, where generating candidate actions for the planner results in an increased likelihood of selecting infeasible action sequences. The paper then introduces ASAR, a method for constructing candidate actions outside the candidate set. ASAR generates actions by mixing two groups of actions: i) the action with the lowest cost, and ii) a weighted mix of local neighborhoods of actions. Then ASAR is evaluated in several continuous control domains.
Critical strengths
- I found the idea of measuring proposal overgeneration interesting. I think taking this into consideration can lead to new work regarding candidate generation for sample based planners.
Critical weaknesses
The critical weakness is the readability of the paper. I took several passes through the paper and I only got a bare understanding of what is going on in the paper; in fact, I may not have understood it correctly because it was so difficult to read. This has definitely hindered my ability to properly assess the soundness of ASAR and evaluate the claims made in the paper.
The two parts of the paper --- proposal overestimation and action construction --- appear to be disjoint. I do not understand how one part relates to the other or how understanding proposal overestimation influences the functioning of ASAR.
I also question the comparison of ASAR against the baseline CEM. I think the idea behind the experiments were to show that augmenting the action set of CEM using ASAR improves its performance, if I understood the setup correctly. However, it just seems like ASAR gets some extra computation because it uses these computations to generate actions. Would CEM perform better if given the same amount of total computation? Again, I may have misunderstood the setup since the explanation in the paper is lacking.
Table 1 is hard to understand based on the description in the corresponding section. The description reads as if Top-20 exposure is the the fraction of pools with feasible actions in the set of low cost actions. First off the budget reduces this value. Second, the paper states that reachability improves this fraction. But that's only at higher budgets --- at lower budgets, it reduces this value. I am also not sure how these pools are made. Equation 6 is an incorrect application of the probability chain rule.
Table 3 also does not make sense --- I do not understand the confidence intervals as written.
The equations and theory are not properly defined. I cannot assess their correctness.
Questions for authors
No questions to ask.
4 comments
- Application of the Probability Chain Rule in Equation (6)
Equation (6) is a direct application of the probability chain rule and does not require an independence assumption.
Let:
P denote the event that at least one feasible candidate exists in the entire candidate pool. T denote the event that at least one feasible candidate exists among the top-k candidates ranked by predicted cost.
Whenever $ T $ occurs, $ P $ must also occur. Therefore, $ T $ is a subset of $ P $. For $ \Pr(P) > 0 $, we have:
This equality follows directly from the definition of conditional probability and does not assume independence between P and T.
The derivation is already provided in Appendix D.1. We will also add the explicit event definitions and intermediate steps to the main text to make the argument easier to verify.
We hope these clarifications address the reviewer's concerns regarding the motivation, computational fairness, and technical correctness of our approach.
I would like to thank the authors for taking their time answer my questions.
Re. 1: I understand that in the paper, this was the intended message. However, given the paper as it is, this is not apparent in the paper. It reads as if there is no flow between the two.
Re. 3, 4: I am glad that these are clarified.
Re. 2: I am happy that the authors ran these experiments as I believe they are a critical evaluation against the baseline.
Re. 5: This explanation makes sense now. I think that a line explicitly stating that needs to be included before equation 6.
I again thank the authors for their rebuttal. However, I still maintain my score. While the authors mention they will add proper definitons and formalizations, I was not able to assess the paper on its correctness given what I read.
Revised Manuscript and Corrections to the Candidate-Budget Audit
We thank the reviewer for the follow-up and for acknowledging our clarifications and additional experiments.
We understand that your remaining concern is whether the technical correctness of our method can be properly assessed from the manuscript itself, rather than from our rebuttal explanations alone.
To address this concern, we have now uploaded a revised PDF with improvements to the writing and presentation, particularly focusing on making the connection between overgeneration and ASAR clearer and improving the presentation of the technical arguments and experimental results.
During our re-examination of the experiments, we also found that some of the "feasibility" labels in the previous candidate-budget audit were generated based on relative rankings among candidates and therefore could not be directly interpreted as indicating whether a candidate satisfied a fixed physical success criterion.
Specifically, this issue affected the following parts of the manuscript:
Page 4, §4: The 32-candidate-pool budget audit and the corresponding Top-1 description.
Page 5, Table 1 and Figure 2: The corresponding Position / Position+yaw results.
Appendix G.2, Table 9: The corresponding two rows.
Appendix G.3, Table 11: The results reported in this table.
We have corrected this issue, rerun the affected experiments, and updated the corresponding results and descriptions in the revised manuscript.
We appreciate your concern that the correctness of the work should be assessable directly from the manuscript. Since the revised PDF is now available, we would be grateful if you could take another look and reconsider your assessment in light of these changes.
Response to Concerns on the ASAR Mechanism, Computational Fairness, and Technical Clarifications
We thank the reviewer for their time and comments. We would like to address the concerns and clarify several important points regarding our method and experimental results.
Overgeneration illustrates that sampling more candidate actions during planning can expand the range of possible actions, but also increases the chance of selecting actions associated with extreme model prediction errors. In our work, overgeneration therefore serves as a mechanism-level diagnosis, suggesting that execution failures may arise during the mapping from the candidate action set to the action ultimately executed.
ASAR takes this observation one step further. While retaining the original world model and CEM cost function, we introduce local support and action reconstruction into this mapping to address the overgeneration problem.
Our core intuition can be illustrated with two candidate actions, A and B. Suppose A has many similar neighboring actions that the model also predicts can accomplish the task, whereas even a small modification to B causes the model to predict failure. Given the continuity of physical dynamics, we hypothesize that A is more likely to generalize to nearby actions and to be genuinely feasible in the physical environment.
To examine this hypothesis, we conduct a controlled evaluation matching states, predicted costs, and sampling origins. We observe a 10.03% improvement in empirical physical feasibility associated with local support.
This provides empirical evidence for our central intuition: local support contains information about physical feasibility that is not fully captured by an individual candidate's predicted cost.
To directly address the concern regarding computational fairness, we have conducted an additional experiment under identical hardware configurations and fixed computation budgets.
Specifically, we run vanilla CEM and ASAR on the same NVIDIA RTX 6000 Ada GPU and dual-socket AMD EPYC 7763 system, with CPU execution restricted to two threads. Both methods are allocated the same 0.5-second computation budget. We evaluate both methods on the Hard Cube task over 128 randomly initialized trials under this matched computational setting.
Vanilla CEM: 22/128 successful trials (17.19%).
ASAR: 32/128 successful trials (25.00%).
Under the same computational budget, ASAR corresponds to a 7.81% improvement in success rate. And this result demonstrates that ASAR outperforms vanilla CEM under the same computation-time budget in this controlled experiment.
We would like to clarify the definition of Top-20 exposure in Table 1. Top-20 exposure measures the proportion of action selections in which at least one feasible action appears among the 20 candidates with the lowest predicted costs.
Therefore, the existence of feasible actions in the full candidate pool is not inconsistent with a decrease in Top-20 exposure. This is because newly sampled infeasible actions may receive lower predicted costs and consequently enter the top 20, displacing feasible actions from the lowest-cost candidates.
In summary, the existence of feasible actions measures sampling coverage alone, whereas Top-20 exposure jointly reflects sampling coverage and the quality of candidate ranking by predicted cost.
We would like to clarify that the confidence intervals in Table 3 represent the 95% confidence intervals of the success-rate differences between ASAR and the corresponding baseline in each row, rather than the confidence intervals of ASAR's own success rate.
For example, +0.039 [0.016, 0.065] indicates that ASAR improves the success rate by 3.9 percentage points relative to the baseline, with a 95% confidence interval for this difference ranging from 1.6 to 6.5 percentage points.
To avoid ambiguity, we will explicitly clarify the comparison direction, statistical unit, and seed-grouped bootstrap procedure in the table header and caption.