- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Preference-conditioned adaptive commitment (PACE) is a framework that learns when an LLM agent should replan versus commit to executing its current plan for multiple steps (to achieve balance in compute cost during inference and end task performance). Rather than using a fixed replanning frequency, PACE dynamically selects a commitment horizon based on the current workflow state and a user-specified preference between task quality and inference efficiency.
To train this controller, PACE restores intermediate execution states and evaluates other commitment horizons under a shared continuation policy. These are used to train separate quality-retention and cost-saving predictors, whose outputs are combined according to the specified preference. The resulting policy is further refined through PPO, while the underlying LLM executor remains frozen. Experiments on ALFworld and WebShop report improved quality-cost trade offs compared with fixed horizon and learned commitment baselines.
Critical strengths
-
Preference-conditioned control: PACE allows a single controller to adapt its planning decisions to different quality-cost preferences without retraining the underlying LLM. The reported improvements on ALFWorld and WebShop also suggest the proposed method's potential effectiveness.
-
Counterfactual-based learning: The proposed restored-state counterfactual evaluation provides a principled way to compare alternative commitment horizons under the same workflow state and continuation policy, offering useful supervision for adaptive replanning.
-
Sound motivation: I think the motivation of looking into quality-cost preference trade offs is convincing enough of a motivation.
Critical weaknesses
Missing References:
- Paglieri et al. (2025), Learning When to Plan: Efficiently Allocating Test-Time Compute for LLM Agents - Studies adaptive planning frequency in LLM agents, directly overlapping with PACE's central motivation. https://arxiv.org/abs/2509.03581
- Liu et al. (2026), When Replanning Becomes the Bottleneck: Budgeted Replanning for Embodied Agents - Proposes BRACE, which learns when and how to replan under computational constraints, raising questions about PACE's novelty and advantages over existing adaptive replanning methods. https://proceedings.mlr.press/v306/liu26fo.html
Questions for authors
Q1) How does PACE fundamentally differ from existing adaptive replanning approaches, particularly the ones in the Missing References in Critical Weaknesses? What specific advantages does PACE offer compared to these existing formulations?
Q2) Can the authors provide direct evidence that PACE actually adjusts its commitment horizons in response to intermediate observations? For example, how does the selected horizon change when unexpected observations invalidate an existing plan?
Q3) Could the authors compare PACE against a preference-conditioned controller that uses only the initial task information and execution progress, without access to intermediate observations? This, I think, will serve as a helpful baseline that would help isolate whether the reported improvements genuinely come from feedback-dependent adaptation.
Q4) I believe this question is important since the authors claim inference cost (might overlap with Q23; Ccould the authors provide the exact executor model, hardware configuration, decoding parameters, and inference setup? Additionally, could they report the number of planning calls and token consumption to clarify where the computational savings originate? I believe this will also be necessary to isolate where the efficiency contributions actually occur.
Q5) Could the authors provide confidence intervals or statistical significance tests for the improvements over the strongest baselines, particularly CF-Direct + PPO and Multi-Scalar PPO? I know this may take some time (and I personally don't like asking these kinds of requests during review) but it does say in the Appendix A.2 that tThe benchmark evaluation uses three independent training seeds and four executions per task and preference" (which I believe should give us the confidence intervals and mean)
Thank you for the questions regarding prior work, intermediate observations, and computational cost.
Q1. Differences from Learning When to Plan and BRACE
PACE trains a lightweight, preference-conditioned commitment policy around a frozen executor using restored-state counterfactual supervision during training. In contrast, Learning When to Plan fine-tunes a unified LLM for a fixed trade-off, while BRACE manages replanning modes and token/latency budgets.
Seed-0 results on identical splits and backbone (
Qwen2.5-7B-Instruct, g_Q = 0.5):PACE achieves the lowest inference latency across both environments and outperforms CF-Direct + PPO in quality, while keeping the 7B LLM completely frozen and supporting continuous runtime preference control.
Q2. Horizon changes in response to intermediate observations
When an action fails mid-segment, control returns to PACE to re-evaluate the horizon from the updated history. In WebShop (seed-0, g_Q = 0.5), across 183 early-return boundaries caused by invalid actions, the mean selected horizon contracted from 3.25 to 2.10 (shortening in 66 cases; e.g., dropping from 8 to 1 after opening an unexpected product page). This confirms that PACE adaptively shortens commitments based on intermediate execution feedback.
Q3. Controller using only initial task information and execution progress
We implemented Initial+Progress, which receives only the initial task, initial observation, g_Q, and execution progress (excluding subsequent observations and action history), while keeping the executor unchanged.
At g_Q = 0.5, intermediate observation feedback drives clear gains in partially observable, multi-step ALFWorld (+7.49% success, -15.23s), whereas shallower search-and-select workflows in WebShop depend primarily on initial query choices under single-seed evaluation.
Q4. Executor configuration, planning calls, and token consumption
All runs use
Qwen2.5-7B-Instruct(bfloat16, RTX 4090, Hugging Face SDPA, batch size <= 4, 4,096-token left-truncated input, greedy decoding up to 384 tokens, EOS left-padding). Per-episode means (including failed episodes) are:PACE reduces planning calls by 21.7% and input tokens by 18.0% in ALFWorld while maintaining compact budgets in WebShop.
Q5. Confidence intervals against CF-Direct + PPO and Multi-Scalar PPO
Using 10,000 paired task-bootstrap resamples across all 134 ALFWorld and 500 WebShop tasks at g_Q = 0.5 (with inference time as cost):
[0.0472, 0.2427][-0.0275, 0.0390]PACE shows a statistically significant utility gain over CF-Direct + PPO in ALFWorld and a positive mean advantage in WebShop. Due to the high compute cost of counterfactual rollout collection across all 634 native benchmark tasks, our statistical analysis focuses on this primary baseline; multi-seed intervals for Multi-Scalar PPO will be added in the final revision.