- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies the assumption in test-time adaptation for LLM agents -- stronger models explore better than the weaker counterparts. The studies examine how the benefits of model capability depend on the information available about the target location and the size of the inspectable candidate space. From the observations, the authors propose a hybrid strategy that combines task-informed guidance and broad inspection to improve performance from LLM agent exploration. The authors evaluate the proposed scope-then-cover exploration strategy on three agentic benchmarks and show performance gain over actor-only baseline.
Critical strengths
- This paper addresses a practically important problem in test-time adaptation for LLM agents.
- The authors evaluate the proposed scope-then-cover method across various agentic benchmarks which span different agentic skills -- the evaluation offers comprehensive coverage.
- The formulation of the budget-prior framework is sound and easy to follow.
Critical weaknesses
- The experiments only test one model family (i.e., Qwen3), therefore "stronger model" here refers to "larger model" and it remains unclear if the proposed method would generalize across model families. For instance, the discovery approach may differ across models (e.g., scope size selected by different models may exhibit different model biases).
- The proposed scope-then-cover strategy does not compare against other exploration-based baseline methods under matched computational budgets in the main evaluation. In addition, the performance gains may just come from additional test-time compute budget -- it is unclear if the performance lift mainly comes from the design of scope-then-cover strategy, or just additional compute budget.
- The methodological novelty appears to be limited, specifically the proposed method of using stronger model for task-informed scoping and weaker model for covering appears to be the main novelty of this work. However, delegating subtasks to different agents depending on task difficulty has been extensively studied in prior work to improve effectiveness or efficiency. Therefore it remains unclear if the decomposition strategy in scope-then-cover offers much methodological novelty. Moreover, the budget-prior formulation mainly formalizes intuitive relationships among search space size, scope recall, inspection budget, rather than providing new theoretical insights. Hence the overall contribution and novelty of this paper may be limited.
- The authors match exploration budget across model sizes as wall-clock time in the controlled experiment (Appendix D.2, fixed budget is times per-inspection wall-clock under a single H100 GPU). However, monetary cost is also an important metric to be considered for practical agent exploration, particularly for running multiple scout agents in parallel. It would strengthen the comprehensiveness of the studies, and offer valuable insights in practical settings, if the authors can consider measuring using monetary cost, and compare cost-performance trade-offs.
Questions for authors
- Figure 2 main caption is missing.
- In ALFWorld randomized probe in Figure 2a, how do you randomize object placements in the experiment setup?
Thank you for your constructive comments. We respond to each concern in detail below.
W1. Generalization beyond the Qwen family
We added a DeepSeek-Flash actor and compared the following configurations on the same 200 WebShop tasks and 274 ALFWorld tasks. Both scout configurations use self-prompt exploration.
Qwen3-4B scouts improve task performance in both environments and achieve results close to those of DeepSeek-Flash scouts. This extends the evidence to a non-Qwen actor and demonstrates cooperation across model families.
Due to time constraints, the corresponding three-arm SWE-bench evaluation has not yet been completed. We will update the results once it is complete.
W2. Exploration baselines and compute allocation
Our added comparison includes using the actor model itself, DeepSeek-Flash, in the scout role. With the same DeepSeek-Flash actor and task sets, both scout configurations use self-prompt exploration: DeepSeek-Flash scouts achieve 68.03% on WebShop and 93.43% on ALFWorld, compared with 69.07% and 93.80% using Qwen3-4B scouts. This directly compares allocating scoped exploration to the same strong model versus a weaker model. Their similar task performance supports the use of weaker models for scoped information gathering, while the actor retains responsibility for task understanding and subsequent decisions.
The controlled WebShop probe separately examines compute allocation under a matched time budget. Based on measured per-inspection times, the 0.6B, 4B, and 8B models receive 30, 12, and 8 inspections, respectively. This tests the role of increased inspection coverage by weaker models within the same time allowance. Together, the two experiments examine the capability required for the exploration role and the capability-coverage trade-off under a fixed budget. W4 reports the monetary cost and task performance of the fully API-based configurations.
W3. Contribution beyond existing multi-agent delegation
Our contribution is to characterize when weaker models can take over information gathering within a scope selected by a stronger actor. We implement this division of labor through an explicit explore action: the actor specifies the missing information and search scope, scouts inspect that scope and return evidence, and the actor uses the evidence to make subsequent decisions.
We analyze and test when this division of labor is effective. Controlled probes examine whether greater inspection coverage can compensate for weaker model capability; return-content ablations examine which information helps the actor; and the added strong-actor comparison tests whether weaker scouts can fulfill this role in full tasks.
The theory further gives a condition for this division of labor to help. Under the proposition's uniformity and reliable-inspection assumptions, discovery lift in the non-saturated regime is characterized by r/rho: narrowing the scope improves over uniform inspection at the same budget when the retained target probability exceeds the scope fraction, i.e., r > rho. This provides a testable quantitative basis for the trade-off among scope size, target recall, and inspection budget.
W4. Monetary cost and performance
We estimate API costs for the fully API-based configurations in W1 using recorded token usage, including recorded reruns. Cached input, uncached input, and output are priced at $0.003, $0.15, and $0.60 per million tokens, respectively, using the historical off-peak reference rates in the experiment scripts.
Adding DeepSeek-Flash scouts increases the WebShop score by 6.42 points at approximately 6.6% higher API cost, and ALFWorld success by 2.55 percentage points at approximately 15.6% higher API cost. These results quantify the performance-cost trade-off of delegated exploration.
Qwen3-4B scouts are locally hosted. Their recorded allocation across eight H20 replicas is 0.026556 GPU-hours per WebShop task and 0.008605 per ALFWorld task, including loading and idle time. Converting this allocation to a total monetary cost requires the GPU-hour price in addition to the actor's API charges; the API-only comparison above therefore uses the two fully API-based configurations.
Q1. Figure 2 caption
Thank you for pointing this out. We will add the missing caption.
Q2. ALFWorld probe randomization
Thank you for pointing this out. We will clarify the randomization procedure.