- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces SPACE, a framework for improving the efficiency of hierarchical robot planning through adaptive allocation of computational effort. SPACE combines a lightweight Local Planner with a more capable Frontier Planner. The Local Planner first predicts whether to retain the current subtask or generate a new one. When a new subtask is needed, a learned Reliability Router evaluates its correctness using token-level confidence and semantic or numeric differences among candidate predictions, invoking the Frontier Planner when necessary. Experiments on RoboMME, RoboDojo, and three real-world manipulation tasks demonstrate improvements in task success and substantial reductions in high-level planning latency compared with the local-planning baseline.
Critical strengths
-
The paper addresses an important practical problem in hierarchical robot planning: unnecessary repeated reasoning and the high inference cost of powerful planners. The proposed framework reduces high-level planning latency while improving task success, demonstrating some value for long-horizon robotic tasks.
-
The authors evaluate SPACE on two simulation benchmarks and three real-world manipulation tasks. The component ablations further demonstrate that transition-aware planning and reliability-based routing contribute to planning efficiency and task performance, respectively.
Critical weaknesses
-
Limited methodological novelty and insufficient analysis of the key decision mechanisms. The proposed framework primarily combines existing concepts, including early-exit decoding, binary transition prediction, confidence-based model routing, and semantic uncertainty estimation. Although their integration is practically useful, the methodological novelty relative to existing adaptive computation and hierarchical planning approaches appears limited. Furthermore, the Hold mechanism is central to the efficiency improvements. Yet, the paper does not provide a direct evaluation of its classification reliability or analyze the consequences of incorrect Hold decisions. Since the Reliability Router is activated only after Advance predictions, incorrect Hold decisions cannot be corrected by the stronger planner.
-
Concerns regarding experimental comparability and reported results. According to Appendix D, the Frontier-Only baseline is evaluated on a 192-episode subset of RoboMME, whereas the main SPACE and MemER results use 800 episodes. These results are compared directly in Table 1, making the claimed performance comparability difficult to establish without matched-subset results. Furthermore, Appendix C.1 reports 25 evaluation episodes per RoboDojo task, while Table 7 includes success rates such as 37.0%, 43.6%, and 97.6%. These values are not consistent with binary success rates computed from exactly 25 episodes unless additional averaging or evaluation procedures were used. These issues require clarification because they directly affect the reliability and interpretation of the experimental comparisons.
Questions for authors
-
Evaluation protocol and statistical consistency. Could the authors clarify how the RoboDojo success rates in Table 7 were computed? Given the stated evaluation of 25 episodes per task, binary success rates should change in increments of 4 percentage points.
-
Fairness of baseline comparisons. Could the authors report SPACE, MemER, and Frontier-Only results on the same 192-episode RoboMME subset, ideally with uncertainty estimates? Additionally, the Frontier Planner receives task-specific prompt information and a more complete visual history. How do these differences in available information affect the comparison?
-
Effectiveness and cost of the routing mechanism. How does the learned Reliability Router compare with simpler routing strategies, such as random routing or entropy-based thresholding, under the same Frontier Planner budget? Could the authors also report planning latency averaged over all real-world episodes, including failed rollouts, rather than only successful ones?
4 comments
[W2-1,Q2-1] Additional Results on matched size.
Thank you for highlighting the difference in evaluation size. The original results used 800 episodes for SPACE and MemER [1] and 192 episodes for Frontier-only. We now report all three methods on the same 192 RoboMME episodes, retaining the original Frontier-only rollouts.
\def\srrow#1#2#3#4#5{\begin{array}{ccccc}\srcell{#1}&\srcell{#2}&\srcell{#3}&\srcell{#4}&\srcell{#5}\end{array}} \begin{array}{lcc} \hline \begin{array}{c}\vphantom{\text{Task success rate}}\newline\text{Method}\end{array} & \begin{array}{c}\vphantom{\text{Task success rate}}\newline\text{Time (s)}\end{array} & \begin{array}{c}\text{Task success rate (\\%)}\ \pm\ \text{bootstrap SE}\newline\hline\srrow{\text{Count.}}{\text{Perm.}}{\text{Refer.}}{\text{Imit.}}{{\textbf{Avg.}}}\end{array} \newline \hline \pi_{0.5} & \text{—} & \srrow{29.17\pm5.04}{22.92\pm5.24}{27.08\pm5.50}{6.25\pm2.40}{21.35\pm2.36} \newline \text{w/ Local Planner (MemER)} & 62.90 & \srrow{43.75\pm5.03}{45.83\pm5.20}{45.83\pm5.80}{27.08\pm3.81}{40.63\pm2.51} \newline \text{w/ Frontier Planner} & 422.58 & \srrow{33.33\pm5.15}{52.08\pm3.34}{66.67\pm4.58}{45.83\pm5.15}{49.48\pm2.31} \newline \text{\textbf{w/ SPACE (Ours)}} & \mathbf{40.51} & \srrow{35.42\pm3.41}{72.92\pm3.51}{70.83\pm4.41}{29.17\pm5.34}{\mathbf{52.08}\pm2.12} \newline \hline \end{array}$$ On the same 192 episodes, SPACE achieves a higher success rate than Frontier-only while substantially reducing planning time. We will include this comparison and explicitly distinguish the evaluation sizes in the main text and appendix. [1] Sridhar et al. Scaling up Memory for Robotic Control via Experience Retrieval. ICLR, 2026. --- **[W2-2,Q1] RoboDojo scores and the expected increments of 4 percentage points.** We would like to clarify that the RoboDojo results follow the official partial-credit Score metric rather than binary Success Rate. We report the mean Score over 25 episodes per task and will correct the metric label from Success Rate to Score in the revision. --- **[Q2-2] Effects of differences in prompt information and visual history on the comparison.** We conducted an additional experiment to assess how input context affects the comparison. We gave the Local Planner the same task-specific prompt information and more complete visual history sampled at 2 fps as the Frontier Planner, and compared this setting with SPACE on the same 192 RoboMME episodes. $$\begin{array}{lccc} \hline \text{Method} & \begin{array}{c}\text{Task success} \\\\ (\\%)\end{array} & \begin{array}{c}\text{Planning latency} \\\\ \text{(s)}\end{array} & \begin{array}{c}\text{Frontier usage} \\\\ (\\%)\end{array} \\\\ \hline \text{SPACE + Local Planner with full context} & 8.85 & 58.39 & 43.6 \\\\ \text{SPACE (Ours)} & \mathbf{52.08} & \mathbf{40.51} & 48.1 \\\\ \hline \end{array}$$ When full visual history and task-specific knowledge were added to the Local Planner, task success dropped and planning latency increased, as shown in the table. These results show that providing more information does not necessarily improve performance and that the current Local Planner struggles to effectively use the expanded context. --- **[Q3-1] Additional evaluation on random routing and entropy-based thresholding.** We appreciate your suggestion to compare the Reliability Router with random routing and entropy-based thresholding. We conducted additional experiments comparing the three strategies on the same 192 RoboMME episodes. SPACE and entropy-based routing are calibrated to the same Frontier invocation rate, although actual usage differs during rollouts. As shown in the table below, SPACE achieved higher task success than both random routing and entropy-based thresholding. $$\begin{array}{lcc} \hline \text{Method} & \begin{array}{c}\text{Task} \\\\ \text{success (\\%)}\end{array} & \begin{array}{c}\text{Frontier} \\\\ \text{usage (\\%)}\end{array} \\\\ \hline \text{Local Planner (MemER)} & 40.6 & 0.0 \\\\ \text{Random routing} & 43.2 & 44.7 \\\\ \text{Entropy-based thresholding} & 46.4 & 51.5 \\\\ \text{\textbf{SPACE}} & \mathbf{52.1} & 48.1 \\\\ \hline \end{array}$$ **[Q3-2] Planning latency including failed real-world rollouts.** We report planning latency for successful real-world episodes to show the total high-level planning inference time needed to complete each task. Failed rollouts ended at timeout, and their planning time was excluded from the reported average. We will clarify this measurement scope in the revised appendix. --- We sincerely appreciate the time and effort you took to engage with our work. If there are any remaining concerns beyond those already discussed, we would be grateful if you could share them so that we may address them more thoroughly during the remainder of the rebuttal period.Thank you for the detailed rebuttal and additional experiments. I appreciate the efforts to address my concerns. The matched 192-episode evaluation, clarification of the RoboDojo Score metric, and comparisons with random and entropy-based routing substantially improve the experimental evidence. However, several concerns remain:
-
The additional experiments reveal that SPACE detects only 19.1% of ground-truth transitions at the actual transition step, although recall increases to 84.0% within the next three calls. Since incorrect Hold predictions bypass the Reliability Router, could the authors further clarify the distribution of transition delays and their consequences for task execution? The oracle experiment is useful, but it does not fully characterize the severity of delayed transitions.
-
My original question specifically requested average planning latency across all real-world episodes, including failures. The rebuttal explains why failed episodes were excluded but does not provide the requested results. Could the authors report the accumulated planning time averaged over all attempted episodes under the same evaluation protocol?
Dear Reviewer AoQB,
We are glad that our responses have substantially addressed your concerns regarding the experimental evidence. Here, we address your remaining questions below.
[Q1] Transition delays and their effects on task execution.
Following your suggestion, we analyzed the distribution of transition delays on RoboMME. We also evaluated task success on RoboMME by providing the high-level planner with ground-truth transition calls at varying delays.
Specifically, we first compute the cumulative percentage of ground-truth subtask transitions that SPACE detected by each planning call. As shown in the table below, most delayed detections occurred one or two planning calls after the ground-truth transition (0.8–1.6 seconds).
\def\rerow#1#2#3#4#5#6{\begin{array}{cccccc}\recell{#1}&\recell{#2}&\recell{#3}&\recell{#4}&\recell{#5}&\recell{#6}\end{array}} \begin{array}{lc} \hline \begin{array}{c}\vphantom{\text{Transition recall}}\newline\text{Method}\end{array} & \begin{array}{c} \text{Transition recall (\\%) (increase in percentage points)}\newline \hline \rerow{\text{At transition}}{\text{By +1 call}}{\text{By +2 calls}}{\text{By +3 calls}}{\text{By +4 calls}}{\text{Full window}} \end{array} \newline \hline \text{SPACE} & \rerow{19.1}{50.5(+31.4)}{77.7(+27.2)}{84.0(+6.3)}{85.6(+1.6)}{88.7(+3.1)} \newline \hline \end{array}$$ Next, to assess how these transition delays affect task execution, we additionally evaluated task success on 192 RoboMME episodes, varying the allowed delay after ground-truth transition. The Local Planner’s Advance prediction is used if it occurs by the specified call. Otherwise, the oracle forces Advance at that call. In these settings, the VLA continued executing the preceding subtask for up to 0.8, 1.6, and 2.4 seconds, respectively. The table below shows similar task success across the oracle transition correction settings except ground-truth transition call (“At transition”). Forcing Advance at the “At transition” yielded the lowest task success. In this setting, the Frontier Planner did not update the preceding subtask in 27 of 103 cases, compared with 9–17 of 103 under delay settings. This suggests that the planner may recognize subtask transition from current visual observation later than the state based oracle transition. $$\def\orcell#1{\mathop{#1}\limits^{\textstyle\hphantom{\text{At transition}}}} \def\orrow#1#2#3#4{\begin{array}{cccc}\orcell{#1}&\orcell{#2}&\orcell{#3}&\orcell{#4}\end{array}} \begin{array}{lcc} \hline \begin{array}{c}\vphantom{\text{Oracle transition correction}}\newline\text{Metric}\end{array} & \begin{array}{c}\vphantom{\text{Oracle transition correction}}\newline\text{SPACE}\end{array} & \begin{array}{c} \text{Oracle transition correction}\newline \hline \orrow{\text{At transition}}{\text{By +1 call}}{\text{By +2 calls}}{\text{By +3 calls}} \end{array} \newline \hline \text{Task success (\\%)} & 52.08 & \orrow{48.44}{56.77}{57.81}{55.73} \newline \hline \end{array}$$ --- **[Q2] Planning time including failed real-world rollouts.** Thank you for your clarification. Following your suggestion, we additionally report the mean accumulated high-level planning time per episode across all attempts, including failures. For failed episodes, we measured the time spent on high-level planning up to the timeout, which was 60 seconds for Normal episodes and 120 seconds for Hard episodes. As shown in the table below, SPACE had lower mean planning latency than MemER on all three tasks at both difficulty levels. $$\def\taskcell#1{\mathop{#1}\limits^{\textstyle\hphantom{\mathbf{00.00\pm00.00}}}} \def\taskrow#1#2{\begin{array}{cc}\taskcell{#1}&\taskcell{#2}\end{array}} \def\taskhead#1{\begin{array}{c}\text{#1}\newline\hline\taskrow{\text{Normal}}{\text{Hard}}\end{array}} \begin{array}{lcccc} \hline \begin{array}{c}\vphantom{\text{Tower of London}}\newline\text{Method}\end{array} & \taskhead{Tower of London} & \taskhead{Solve Equations} & \taskhead{Shell Game} & \begin{array}{c}\vphantom{\text{Tower of London}}\newline\text{Overall}\end{array} \newline \hline \text{MemER} & \taskrow{27.99\pm0.71}{59.63\pm0.99} & \taskrow{29.16\pm0.81}{60.20\pm1.82} & \taskrow{37.62\pm0.75}{38.69\pm0.62} & 41.63\pm0.30 \newline \text{SPACE} & \taskrow{\mathbf{9.19\pm0.03}}{\mathbf{46.40\pm6.50}} & \taskrow{\mathbf{14.18\pm0.98}}{\mathbf{43.01\pm16.04}} & \taskrow{\mathbf{29.51\pm0.71}}{\mathbf{34.86\pm1.63}} & \mathbf{29.91\pm2.25} \newline \hline \end{array}$$ --- If you have any further comments or suggestions, please let us know. We are committed to improving the quality of our work, and we value your feedback. Thank you very much, Authors
Dear reviewer AoQB,
We sincerely appreciate your efforts and insightful comments to improve the manuscript.
[W1-1] Concerns on limited methodological novelty.
We would like to clarify that SPACE is the first framework that selectively allocates computational effort when using frontier high-level planners for robot planning. Its methodological novelty lies in allocating computational effort based on subtask transitions and the reliability of subtask predictions. Figure 4 and Table 3 in the paper show that first-token termination reduces planning time compared with full generation, and using semantic differences improves task success over confidence-only routing. Our additional experiments show higher task success with fewer Frontier calls than entropy-based routing. These results demonstrate that these design choices are crucial for SPACE to reduce computational effort while maintaining task performance.
[W1-2] Reliability of Hold predictions and consequences of incorrect decisions.
Following your suggestion, we conducted two additional experiments on RoboMME to evaluate the reliability of Hold/Advance predictions and the effect of transition errors on task success.
First, we compared SPACE’s predictions with simulator-provided ground-truth history context in 192 rollouts. We report transition recall, the percentage of ground-truth transitions detected. When the current subtask was supposed to continue, the Local Planner correctly predicted Hold on 97.6% of calls. As shown in the table below, the Local Planner detected most transitions it had missed on the first call within the next three calls. By the end of this window, its detection rate was close to that of MemER [1], which generates a full subtask at every call.
\def\detrow#1#2{\begin{array}{cc}\detcell{#1}&\detcell{#2}\end{array}} \begin{array}{lc} \hline \begin{array}{c}\vphantom{\text{Transition recall (\\%)}}\newline\text{Method}\end{array} & \begin{array}{c}\text{Transition recall (\\%)}\newline\hline\detrow{\text{At transition}}{\text{Through next 3 calls}}\end{array} \newline \hline \text{SPACE} & \detrow{19.1}{\mathbf{84.0}} \newline \text{MemER} & \detrow{23.2}{\mathbf{86.0}} \newline \hline \end{array}$$ Second, we tested whether correcting transition decisions improves task success. In the same episodes, we replaced incorrect Hold/Advance tokens with the simulator’s ground-truth transition tokens. After Advance, the Local Planner still decoded its own subtask and the original Router was applied. As shown in the table below, correcting Hold/Advance predictions improved task success from 52.08% to 55.73%. $$\begin{array}{lc} \hline \text{Method} & \text{Task success (\\%)} \\\\ \hline \text{SPACE} & 52.08 \\\\ \text{SPACE + oracle transition} & 55.73 \\\\ \hline \end{array}$$ Although the Router cannot correct an incorrect Hold directly, the Local Planner often detects the missed transition within the next planning timesteps. The oracle results show that correcting Hold/Advance errors can further improve task success. We will include both evaluations in the revision. [1] Sridhar et al. Scaling up Memory for Robotic Control via Experience Retrieval. ICLR, 2026.