- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
This paper presents a problem of determining execution depth: “how many actions should an agent execute open-loop before it replans?” The authors argue that this should be learned rather than use of fixed value. So the problem can be seen as a decision process with a hard budget K on the number of policy calls. PACE adds two haeds to Qwen VLM: a depth head that picks h and an action decoder that classifies over a fixed per-environment command list, outputs 8 actions and executes only the first h. On sliding Puzzle, Sokoban, ALFWorld and ScienceWorld, PACE beats the best fixed depth while using slightly fewer decisions.
Since many methods have relied on fixed execution horizon, the problem that this paper focuses on seems worth addressing. However, I’m not persuaded that the use of adaptive depth brings advantage because a hand-picked K is too small to finish most tasks, and there is no intermediate feedback even though the simulators produce it for free. Furthermore, the main comparison does not isolate what it claims to isolate. Together with fairly limited novelty relative to prior work on execution horizons, this leads me to lean towards rejection.
Critical strengths
- The problem statement is easy to understand.
- The evaluation has performed in diverse environment with diverse settings, providing multifaced evidence of the effect of the proposed method.
Critical weaknesses
- The idea of learning how long to commit to an action, jointly with the action itself, is not new. In FiGAR (Sharma et al., 2017), the policy is factorized into an action component and a repetition component. This is very close to the PACE design. The main difference is that PACE executes a sequence of different actions open-loop, whereas FiGAR repeats a single action. Dynamic Execution Horizon Prediction for Chunk-based Robot Policies (Zhao et al., 2026) also already report that learned or adaptive horizons beat fixed ones and follow the task phase. What remains is applying the idea to discrete VLM agents, with a closed action vocabulary and a hard budget. That reads to me more as an engineering combination than a new insight.’
- I suspect that the experimental setup is constructed favoring adaptive depth. The authors set K below the length of the reference solutions. For example, in ScienceWorld, the references take 6-22, 23-32, and 35-153 actions for short, medium, and long groups, with decision budgets K = 6, 9, and 18, respectively. With h = 1, the agent can execute at most K actions, which is fewer than most tasks require. It therefore scores at or near 0%, no matter how good its individual decisions are. Under such a budget every policy is forced to execute long chunks, so it seems that a policy using long chunks where they are safe and short ones where they are risky has more advantage over single fixed depth setting. A comparison with close-loop execution (h=1) under a larger budge seems required.
- In the experiment, the agent never uses the feedback that the simulators already return after every action (invalid actions do not stop a chunk). Compared to the simple rule like stopping the chunk as soon as the environment reports and invalid action, what advantages can be brought by the use of PACE?
Questions for authors
- What do the authors see as the main novel idea of PACE beyond the mentioned previous works?
- Can you clarify when adaptive depth helps and when it does not? For example, how does the gain change with reversibility, with observability, or with different K?
- The paper models cost as a fixed limit on the number of decisions: once the agent has made K decisions, the episode fails. However, the cost of calling a model is usually time or money in practical. It would help to report the actual time or token cost of PACE.
- All fixed-depth baselines reuse PACE’s checkpoint, which was trained to mix depths, and simply force a single depth at test time. Could you train a fixed-depth policy with RL under the same budget and compute, at least for the best h, and compare it with PACE? Without this comparison, it is hard to tell whether the gain comes from adaptive depth or from the RL training itself.
5 comments
Response to Reviewer z8no (2/2)
Q2 (when does adaptive depth help, and how does the gain change with reversibility, observability and K?). We answer this in detail in General Response G5. In short:
- Theory. For a learned policy, choosing the depth by state can help at any K (Prop. 1); under a budget, it can help even at the optimum (Prop. 2). PACE improves over the same-checkpoint best fixed depth in all seven settings: +8.6 to +15.6 pp in six and +3.1 pp on Sokoban (+8.5 to +13.1 and +1.6 vs. the strongest baseline in Table 2); the gain is positive in all seven, and against the same-checkpoint best fixed depth it is significant in 2 by McNemar's test and in 3 by the bootstrap CI.
- K. Evaluated at larger budgets than used in training, PACE leads the best fixed depth by 4.7–15.0 pp in all seven settings (Table 3). At the relaxed budgets, successful episodes finish well before the limit, yet PACE still beats step-wise control in all seven settings and still mixes depths (G3).
- Observability. Not every new observation is worth a decision (Flaspohler et al., 2020). At the relaxed budgets, step-wise control, which observes after every action, still has lower success than PACE in both partially observable domains (G5).
- Reversibility. The gain is smallest on Sokoban, where deadlocks cannot be undone and cause 68–79% of failures; there, learned timing mainly saves cost (25.5% fewer environment steps per episode). ScienceWorld is also irreversible, yet its gains are much larger than on Sokoban, so reversibility alone does not set the gain (G5).
- What the selected depths show (new appendix table). At the main budgets, PACE uses at least two depths in 58–98% of episodes, and two independently trained backbones most often select the same depth in six of seven settings, so the learned depth depends on the task.
We did not vary observability or reversibility within a domain, so comparisons across settings are correlational, not causal.
Q3 (actual time or token cost). We count decisions because each decision is one full forward pass of the 7B VLM backbone over the current context, which dominates compute, memory traffic and latency.
- The extra cost per decision is negligible. The depth head is one linear layer, and the action decoder is a two-layer module over a closed action vocabulary.
- PACE generates no free text. Each decision outputs one depth class and eight action classes, so the output length is constant, with no reasoning tokens. The cost is set by the number of decisions and the context length.
- Decision count measures backbone calls and serves as a proxy for inference compute. Executing more actions in a chunk needs no further model call.
- Token cost: PACE uses fewer input tokens per success in all seven settings, measured in the text settings and bounded in the puzzles. We counted the input tokens of every decision in the evaluation logs, including failed episodes.
- Text settings (ALFWorld, ScienceWorld). We count the prompt plus the text observation. Tokens per decision differ by less than 9% between PACE and the best fixed depth, so the gap comes mainly from the number of decisions: the best fixed depth needs 1.16–1.38× more input tokens per success (e.g., 10,072 vs. 7,545 on ALFWorld seen).
- Puzzles. The input is a fixed prompt plus one image of the current board, with no earlier frames. The board size varies across instances but not within an episode, and the logs do not store image tokens, so we bound the ratio: whether image tokens are constant, proportional to the board area, or a mix of the two, the best fixed depth needs 1.39× more input tokens per success on Sliding Puzzle and 1.08–1.10× more on Sokoban.
- Counting model calls is common practice. Recent agent benchmarks also cap the number of model calls rather than environment steps, with several actions allowed per call (e.g., CubeBench (Gao et al., 2025) caps the model calls per run, each of which may execute several moves).
Table 14 reports decisions per success, including the cost of failed episodes. Our formal results also do not depend on counting decisions: they hold when each decision has a cost, such as the time, money or tokens of one model call, as long as one decision costs less than half the value of success (new per-decision-cost Remark, App. B).
Summary of changes. In response to this review, we added the closed-loop comparison at relaxed budgets, the invalid-action stop rule for three policies without a learned depth, an analysis of when chunking helps (G5), fixed-depth policies RL-trained at their own depth, statistics of the selected depths, the measured token cost, and new formal results (Props. 2–3 and two Remarks, App. B). We also revised the abstract, contributions and related work to state the novelty more precisely. We hope these results address the reviewer's concerns, and we would be grateful if the reviewer would consider them in the final assessment.
Dear Reviewer z8no,
With the discussion period nearing its end, we would like to follow up to see if our response has resolved your questions. We would be glad to provide any further clarification if needed.
Kind regards,
The Authors
Response to Reviewer z8no (1/2)
We thank the reviewer for the careful review, and for finding the problem statement easy to understand and the evaluation across diverse environments and settings to provide multifaceted evidence. All three weaknesses are addressed with new experiments or new formal results; points shared with other reviewers are covered in detail in the General Response (G1–G5).
W1 / Q1 (novelty relative to FiGAR and DEHP). We agree that learning how long to commit, and that adaptive horizons can beat fixed ones, is known; the original submission cites this lineage (§2, Remark 3), and PACE does not claim it. What PACE adds is what changes when each decision has a cost, as when every decision is a call to a large model:
The experiments support both points and show that the differences from FiGAR (Sharma et al., 2017) and DEHP matter empirically:
Details: General Response G1.
W2 (the budget favors adaptive depth; closed-loop h = 1 under a larger budget). We ran exactly this comparison. With relaxed budgets (K = 50; 80 / 150 for ScienceWorld medium / long), step-wise control (h = 1) still loses to PACE in all seven settings, by 5.2 to 65.6 pp (4.9% vs. 70.5% on ScienceWorld long), and it needs more decisions per success in all seven. On the puzzles, we also trained dedicated h = 1 policies with GRPO at K = 50; PACE beats them as well (+11.0 / +7.8 pp). At these budgets, successful episodes finish well before the limit (a median of 3–10 decisions for PACE), yet PACE still leads and still mixes depths (75–100% of episodes use at least two). The advantage of PACE thus persists at the larger budgets we tested. The tight main budgets are deliberate: we study how to spend a small number of decisions, each a full forward pass of a 7B VLM. Details: General Responses G3 (Table R4) and G5.
W3 (stopping a chunk at the first invalid action). We evaluated this rule on ALFWorld for PACE and for three policies without a learned depth: fixed-depth policies RL-trained at h = 7 and h = 8, and an action-only policy (see General Response G5, Table R10). The rule replaces the learned decision of when to stop with a fixed trigger: stop and replan at the first rejected action.
Q4 (a fixed-depth policy RL-trained under the same budget and compute). We trained fixed-depth policies with RL at their own depth, from the same SFT checkpoint and with the same budget, RL cycles, hyperparameters and checkpoint selection. We did this for the SFT-best depth and for the RL-best depth of Table 2, including depths outside H.
Details: General Response G2, Tables R2–R3.