- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies how the importance of exact world-model evaluations changes during population-based planning. The authors fit a local linear response model from a small number of exact queries and use it to replace evaluations at different search stages. Early replacement is generally more damaging than late replacement on PushT and Two-Room, whereas Reacher does not show the same pattern. The analysis relates these results to proposal contraction. On PushT, representation error decreases during search while candidate-ranking agreement changes little. The authors then introduce Exact-to-Local CEM, which retains exact evaluations early, approximates intermediate rounds, and evaluates the final population exactly. Experiments compare this schedule with native CEM, smaller populations, truncated search, and adaptive stopping, with additional stage interventions in iCEM and MPPI. The main finding is that late computation can be reduced without a detectable loss of success in the tested settings, with different trade-offs in planning time and final model cost.
Critical strengths
-
The paper shows that the effect of approximate evaluations depends on when they are introduced. Table 1 shows larger losses from early than late replacement in several PushT and Two-Room settings. On PushT, decreasing representation error is not accompanied by a substantial improvement in ranking agreement (Table 6), yet late replacement can preserve success. This suggests that the effect on the search, beyond approximation accuracy itself, matters when allocating exact evaluations. Reacher provides a boundary case, since early replacement does not reduce success there.
-
The local model in Section 4 replaces world-model evaluations while retaining the original cost function and CEM update rule. The operator is fitted from the current decision’s own queries and requires no retraining of the world model. This makes it useful both for studying the effects of replacement and for implementing E2L. Appendix H supports the design through elite-overlap and proposal-drift measurements, and the timing experiments show that the resulting planner can reduce complete planning time.
Critical weaknesses
My main reservation concerns the additional practical benefit of E2L over stopping earlier. In Table 2, E12 saves about 3% of planning time on PushT, whereas R13 saves about 12% with similar observed success rates. E12 has a smaller reported mean increase in final model cost relative to full CEM (+0.19 versus +0.69), but it also takes longer. The comparison therefore establishes a trade-off without resolving whether approximate refinement makes better use of the available planning time. This leaves the analysis of search stages as the stronger contribution.
Questions for authors
In Table 2, E12 and R13 use nearly the same number of exact queries, but E12 takes 0.152 s per decision compared with 0.138 s for R13. How does E2L compare with truncated native CEM in final model cost when their planning times are similar? A comparison on the same evaluation sources would clarify whether the approximate refinement rounds offer a benefit at a fixed planning-time budget.
4 comments
Thank you for the detailed response. It addresses most of my concerns. However, after rereading the paper, I have one further question.
The paper mainly reports absolute changes in Cost but does not provide the baseline Cost values, making it difficult to assess the magnitude of these changes. Could the authors also report the baseline Cost and the corresponding percentage changes? I also noticed that in some experiments, Cost changes substantially while the success rate remains nearly unchanged. Could this be due to the relatively loose success criteria of these tasks? For tasks requiring higher precision, could similar changes in Cost lead to a substantial drop in success rate? Would E2L still maintain its advantage in such settings?
Sorry for bringing this up a bit late. If you still have time, I'd appreciate your thoughts on these questions.
Thank you for rereading the paper and for this further question. We are glad our response addressed most of your concerns, and the question is not late at all.
1. Baseline cost and percentage change
New Appendix S gives the native cost, written below as mean (median), and each change in the same form. A source's cost averages the exact final cost over its decisions.
Section 6.4 and Table 2, pre-success cost at comparable planning time:
| Task | Comparison | Native | E2L | Change | E2L lower on |
|---|---|---|---|---|---|
| PushT | E14/8 vs R20 | 12.20 (2.83) | 12.26 (2.29) | +0.5% (−19.3%) | 76% |
| PushT | E24/8 vs R35 | 10.22 (1.66) | 10.21 (1.50) | −0.1% (−9.4%) | 78% |
| PushT | E36/8 vs R45 | 9.78 (1.41) | 9.31 (1.24) | −4.8% (−12.3%) | 96% |
| Two-Room | E16/8 vs R20 | 61.21 (10.45) | 54.84 (2.52) | −10.4% (−75.9%) | 79% |
| Two-Room | E24/8 vs R36 | 45.77 (2.03) | 46.58 (1.15) | +1.8% (−43.2%) | 72% |
| Two-Room | E40/8 vs R45 | 43.54 (1.28) | 42.06 (0.79) | −3.4% (−38.5%) | 93% |
The last column is the share of sources where E2L-CEM costs less. The mean is set by sources neither arm solves: on low-budget PushT, 254 such sources average 106, while on the 3,061 sources both arms solve, E2L-CEM lowers the median cost from 2.59 to 2.10. Native CEM with 60 rounds reaches 9.28 (1.25) on PushT and 42.14 (0.92) on Two-Room, which high-budget E2L-CEM matches.
Section 6.2 and Table 1, cost change from stage substitution:
| Task and planner | Native | Early | Late |
|---|---|---|---|
| PushT CEM R=60 | 8.84 (1.24) | +185.0% | +6.4% |
| Two-Room CEM R=60 | 20.63 (0.51) | +230.9% | +1.5% |
| PushT CEM | 14.60 (3.83) | +37.7% | +4.9% |
| PushT iCEM | 71.04 (35.06) | +78.4% | +11.5% |
| PushT MPPI | 16.60 (5.70) | +178.8% | +9.0% |
| PushT CEM K=15 | 14.83 (3.35) | +27.4% | +12.0% |
| PushT CEM K=60 | 19.59 (6.62) | +16.3% | +3.3% |
| PushT LeWM | 11.01 (3.00) | +46.8% | +11.1% |
| Two-Room CEM | 18.38 (0.86) | +91.1% | +10.1% |
| Two-Room MPPI | 8.36 (0.43) | +970.4% | −31.2% |
| Reacher MPPI | 23.06 (0.44) | +98.4% | −1.1% |
| Reacher CEM | 35.37 (0.56) | +20.7% | −32.0% |
In Section 6.3, stopping native CEM after 13, 11, 7 and 3 of 15 rounds raises the PushT mean of 20.84 (4.22) by 0.3%, 15.6%, 57.6% and 141.5%, and stopping after 15 of 30 rounds raises the Two-Room mean of 14.93 (0.79) by 53.5%. On Section 6.2's 2,000 sources, early noise and early local scoring raise the first-decision mean of 12.63 (2.82) by 23.7% and 38.4%.
So early substitution raises cost by 16% to 970%, and late substitution by at most 12% where its interval excludes zero. At comparable planning time, E2L-CEM lowers cost on 72% to 96% of sources.
2. Why cost can change while success barely moves
The success criteria are the standard benchmark thresholds. PushT requires a joint position error of agent and block below 20 pixels and an angle error below 20 degrees, Two-Room a goal distance below 16 pixels, and Reacher both joint errors below 0.05 rad. As the paper argues, early rounds decide whether a plan reaches the goal region, which these thresholds record, and later rounds refine within it, which mainly the cost records. Stopping native CEM at 20 instead of 60 rounds raises cost on 98% of PushT sources and its mean by 31.5%, but lowers success by only 0.56 points. At one quarter of the tolerance it lowers success by 7.76 points. On Two-Room the mean rises by 45.2% and success falls from 2,483 to 2,430 (p=2.4×10⁻⁶). So a large enough cost gap changes success, and the stricter the criterion, the more of the refinement it sees.
3. E2L under stricter success criteria
Yes, and the advantage of E2L-CEM at comparable planning time grows as the criterion tightens. PushT runs all 50 steps after success and saves every state, so we rejudged the 3,375 trajectories of Table 2 (new Appendix S). We fixed the threshold scales 1, 3/4, 1/2 and 1/4 before computing.
| Comparison | 1× | 3/4 | 1/2 | 1/4 |
|---|---|---|---|---|
| E14/8 vs R20 | −0.30 | +0.21 | +0.03 | +2.87 [1.45, 4.27] |
| E24/8 vs R35 | −0.24 | −0.21 | −0.09 | −0.59 [−1.87, 0.68] |
| E36/8 vs R45 | +0.21 | +0.24 | +0.59 | +1.13 [0.03, 2.16] |
| R20 vs R60 | −0.56 | −1.36 | −2.96 | −7.76 [−9.27, −6.16] |
Entries are success differences in points, with 95% paired bootstrap intervals. Native CEM with 60 rounds solves 92.3%, 90.2%, 84.0% and 53.9% at the four tolerances. At one quarter of the tolerance, low-budget E2L-CEM solves 2.87 points more than native CEM with 20 rounds in 3.5% less time, and high-budget E2L-CEM solves 1.13 points more than native CEM with 45 rounds. Middle-budget E2L-CEM matches native CEM with 35 rounds at every tolerance in 12.5% less time. High-budget E2L-CEM matches native CEM with 60 rounds at every tolerance, 54.0% against 53.9% at one quarter, in 24% less time.
So stopping earlier gives up the refinement that a strict criterion rewards, and E2L-CEM, which keeps it with local updates, solves more at comparable planning time.
Thank you again for the detailed response and additional experiments. They have addressed my remaining concerns, and I will maintain my current positive rating.
Response to Reviewer FrP4
Thank you for the accurate summary. You point out that the effect of an approximate evaluation depends on when it enters the search, which is the central observation of our paper. As you also note from Table 6 of the submission, where exact evaluations are allocated, an approximation's effect on the search matters beyond its accuracy. The revision tests the practical consequence of this finding: whether, at comparable planning time, cheap local refinement trades cost against time better than truncated native CEM.
1. Additional practical benefit of E2L over early stopping.
Yes. At the middle budget on PushT, E2L-CEM uses 12.5% less planning time with no detectable difference in pre-success cost, −0.01 with 95% interval −0.25 to 0.22. At the low budget on Two-Room, it lowers pre-success cost by 6.37, interval 5.31 to 7.45, with 4.7% less planning time. At the high budget, its cost is below native CEM with 45 rounds, 9.31 against 9.78 on PushT and 42.06 against 43.54 on Two-Room, with 1.4% and 5.4% more planning time, and its observed mean is close to the 9.28 and 42.14 of native CEM with 60 rounds, with 24% and 7% less planning time. Section 6.4 now reads:
Development runs selected the same schedule, E24/8, independently on both tasks: the first 24 of 60 rounds are exact and the local operator is refitted every 8 rounds after that. So at budgets where truncation already preserves success, the added local updates recover most of the model cost that truncation loses.
2. Final model cost of E2L against truncated native CEM at comparable planning time on the same sources.
At comparable planning time on the same sources, E2L-CEM has lower pre-success cost at three of the six budgets, both high budgets and the low budget on Two-Room. On PushT at the low and middle budgets no difference was detected, and at the middle budget on Two-Room its cost is 0.81 higher with 12.2% less time. Each time ratio is measured with both planners on the same GPU, at the low and high budgets over 25 disjoint timing sources (Table 2 footnote, Appendix C). Two-Room uses 2,950 sources with N=150, and PushT 3,375 sources with N=300.
Within each budget no success difference is significant. Across budgets, lower cost goes with more successes: high-budget E2L-CEM solves 3,126 PushT and 2,483 Two-Room episodes against 3,096 and 2,430 for native CEM with 20 rounds (p=1.4×10⁻³ and 6.9×10⁻⁶).
The E12 against R13 comparison you mention matched exact queries. E12 takes 0.152 s per decision and R13 takes 0.138 s, because planning time also includes fitting and local updates. The new experiments therefore choose comparison points by complete planning time and test whether added local refinement improves the cost and time trade-off, and the original comparison remains in Appendix E. So at comparable planning times, E2L-CEM significantly lowers pre-success cost relative to the evaluated truncated-CEM references at both high-budget points and at the low-budget Two-Room point.