acceptodds
Under review as a conference paper at ICLR 2027

Measure What Changes the Decision: Bounded Lookahead for Queried-Only Deployment

Abstract

When an action must be measured before it can be deployed, information acquisition and resource allocation become coupled. A measurement budget Q controls which action outcomes are known, while a separate budget B couples the measured actions selected for deployment. Under this coupling, a query can have little immediate value yet make a later query useful. Exact one-step Expected Deployable-Utility Gain (EDUG) cannot price this delayed payoff. We therefore introduce A2-EDUG, a bounded two-step policy that ranks all queries by their immediate decision value, evaluates two-query plans for the four best candidates using 256 posterior draws, executes only the selected first query, and replans. With the belief model, query budget, and deployment solver held fixed, A2-EDUG reduces average regret along the query path by 11.99% on fresh DIV2K (95% group-bootstrap interval [6.26%, 17.55%]), by 16.16% on a speaker-disjoint FSDD confirmation, and by 7.72% ([5.78%, 9.57%]) in a 240-episode FSDD scale study. In both systems, the policy is about 3% worse after its first query and becomes better after the second, matching the predicted delayed-payoff mechanism. A control matched to the same number of exact solver calls and a monotone k = 1, 2, 4, 8 candidate sweep show that the gain comes from two-step planning, not simply from running more one-step evaluations.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
active information acquisition, value of information, nonmyopic planning, budget-constrained decision making, resource allocation, learned image compression, speech representation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_uqUWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies acquisition when an action becomes deployable only after it has been measured, with separate measurement and global deployment budgets. A2-EDUG shortlists queries using immediate expected deployable-utility gain, evaluates a bounded two-step continuation for four candidates, executes the first query, and replans. A counterexample illustrates arbitrarily poor one-step behavior. Image-compression and speech-representation experiments report lower query-path regret under a fixed belief model and measurement budget, with matched solver-call and approximation controls.

Critical strengths
  1. The queried-only deployment formulation captures a useful distinction: measuring an action also makes it available for deployment. The counterexample clearly illustrates how a global deployment budget can create complementarities that one-step acquisition misses.

  2. The empirical controls are careful. The paper fixes the belief model and deployment solver and includes matched solver-call, exhaustive-planning, shortlist, and fixed-second-query comparisons. These results support the usefulness of a two-query horizon within the tested setting.

Critical weaknesses
  1. The methodological contribution of the lookahead construction could be articulated more clearly. The approach combines familiar shortlisting and two-step rollout components, and practical two-step lookahead is already studied in cited work such as Wu and Frazier (2019), albeit in a different setting. The competitive fixed-second-query control suggests that much of the demonstrated gain comes from extending the horizon; it does not establish a separate benefit from the adaptive continuation machinery. Explaining the methodological insight specific to queried-only deployment would strengthen the contribution.

  2. An important question is how reliably the candidate-selection rule captures the complementarity motivating the method. Ranking candidates by immediate value can exclude actions whose value emerges only jointly, while Proposition 3 uses an instance where the top-four shortlist is exhaustive. The existing shortlist-recall and exhaustive-planning analyses acknowledge this approximation, but all episodes still contain four items and 13 actions per item. It would be valuable to clarify how well this behavior is expected to extend to larger coupled decisions. Given the roughly 24.7-second planning cost versus about 10 seconds for one-step EDUG (Table 2), such evidence would also help readers identify the regimes where the extra planning is worthwhile.

Questions for authors
  1. Could you clarify the methodological insight beyond standard two-step rollout, particularly in light of the competitive fixed-second-query control?

  2. When does the immediate-value shortlist retain actions whose benefit is mainly complementary? Is there evidence beyond four-item allocations, or a structural condition supporting this design choice?

  3. Is the equal-weight planning score in Eq. (10) intentionally a surrogate for trapezoidal AURC in Eq. (5)? For two queries and fixed initial regret, AURC weights the first and second post-query regrets 2:1. Clarifying this relationship would help assess the method’s objective.

Reviewer_z18AOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

In this paper, the authors study the problem of choosing a sequence of actions under a measurement budget and a deployment budget. The goal is to minimize the total loss subject to the budget constraints.

The authors proposed a two-step policy (A2-EDUG) and demonstrated that it is better than the myopic (one-step) policy (EDUG) on two benchmarks.

Critical strengths

None

Critical weaknesses

The first critical weakness is that the novelty of the paper is not established, due to the lack of literature review and the limited comparison with baselines in the experiments. This work is related to various streams of works such as active learning, Bayesian optimization, and multi-armed bandits. However, there is no discussion of related work. In addition, the only baseline in the experiments is the one-step policy, which is by design worse than a two-step policy.

The second critical weakness is the soundness of the paper, including various definitions and design choices without motivation or explanation. Below is a non-exhaustive list of such issues.

  • In the abstract, the author said that the proposed A2-EDUG "evaluates two-query plans for the four best candidates using 256 posterior draws". Where do the hyperparameters of "four" best candidates and "256" posterior draws come from?
  • In Line 072, why is the value of two queries D + (1/2) C?
  • In Eqn. (5), why is the AURC defined in this way, where r_1, ..., r_{Q-1} are counted twice and r_0 and r_Q are counted once?
  • There are too many terminologies without definitions, such as "mandatory anchors", "paid outcomes", "measured cells", "queried-only optimum", etc.
  • In Eqn. (8), Bayes risk \mathcal{R} is not defined.
Questions for authors

Please see my comments on Critical Weaknesses.

Reviewer_PVh6OpenReview ↗
Rating
4
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies the problem of minimizing the average reconstruction error of selected options under a budget to choose among those options, with image compression and audio reconstruction as two canonical examples. The paper proposes a new heuristic, A2-EDUG, on top of the existing EDUG baseline. Built upon the insight that simulating two options jointly instead of one option alone is beneficial, A2-EDUG shows empirically lower area-under-the-regret-curve than the baseline.

Critical strengths
  • The paper, especially the intro, clearly delineates the importance and current limitations of solving the reconstruction error minimization problem.
  • The proposed solution is well motivated: jointly simulating two options is more informative than simulating one with measured computation overhead.
  • The experiments include comparison of different hyperparameters and controls in different domains, establishing the strength of the proposed method.
Critical weaknesses
  • The paper does not justify the tradeoff between efficiency and performance, as A2-EDUG being a two-step heuristic shows twice to four times as much the planning time with only 10-15% AURC reduction.
  • The proposed method does not outperform a simple control, as fixing the second query matches A2-EDUG with same top-k query in both time and error reduction rate. However, both are acknowledged to be variants of the same novel proposal.
Questions for authors
  • Can you provide a theoretical analysis of EDUG and A2-EDUG's time complexity as the search space becomes large? This way we can clearly see the efficiency tradeoff.
  • Can you discuss the advantages of A2-EDUG over the fixed-second-query method?
1 comment

We sincerely thank the reviewer for the careful and constructive assessment. We especially appreciate the reviewer’s recognition of both the motivation for jointly simulating two options and the breadth of our experimental controls. The comments on planning cost and the fixed-second control identify two important questions that the manuscript did not explain clearly enough. We address these questions below.

Complexity and the efficiency tradeoff. Let UqU_q denote the number of unseen cells at query depth qq, SS the number of posterior draws, kk the shortlist size, TallocT_{alloc} the cost of one exact queried-only deployment solve, and TrankT_{rank} the cost of scoring one candidate under a conditional-mean continuation. In our experiments, S=256S=256 and the primary setting is k=4k=4.

Up to lower-order shortlist-selection and posterior-conditioning terms, the leading cost of one-step EDUG is TEDUG(q)=O(UqSTalloc)T_{EDUG}(q)=O(U_q S T_{alloc}). A2-EDUG first performs the same all-candidate EDUG scoring and then evaluates kSkS conditional continuations: TA2(q)=O(UqSTalloc+kSUqTrank+kSTalloc)T_{A2}(q)=O(U_q S T_{alloc}+kS U_q T_{rank}+kS T_{alloc}).

With the per-solve costs held fixed, both methods are linear in UqU_q when kk and SS are fixed. The additional cost is controlled by kk and the Monte Carlo budget SS. If kk grows with the search space, the shortlist-dependent continuation term can become quadratic in UqU_q. This decomposition separates planner overhead from the exact deployment solver, whose own cost depends on the multiple-choice knapsack implementation and on NN, MM, and BB.

The existing per-trajectory planning audit makes the practical frontier explicit. Relative to EDUG, k=4k=4 takes 2.45× the planning time on DIV2K and 2.62× on FSDD, while reducing AURC by 11.99% and 16.25%, respectively. The cheaper k=2k=2 setting gives 9.37% and 10.60% reductions, whereas k=8k=8 gives 15.56% and 18.75% at roughly 4.5–4.8× the EDUG time. Thus, k=4k=4 is an operating point on a tunable compute–quality frontier, not a claim of universal efficiency.

Fixed-second control. We agree that the fixed-second control is an important comparison. It retains the two-query horizon and the same k=4k=4 planning setting while removing outcome-dependent re-selection of the second query. With 568.8k solver calls and essentially the same planning time, it gives a 13.33% reduction on DIV2K and a 15.44% reduction on FSDD, compared with 11.99% and 16.25% for outcome-adaptive A2-EDUG.

The conceptual advantage of outcome-adaptive continuation is available when the best second query changes with the simulated first outcome; when the best continuation is outcome-invariant, adaptive continuation reduces to a fixed continuation. In our current experiments, the difference between the two policies is small, and the paired intervals for the A2-EDUG versus fixed-second difference cross zero.

We will therefore make the claim more precise in the revision: the evidence supports the value of a bounded two-query lookahead horizon in queried-only deployment, but it does not establish a separate benefit from outcome-adaptive branching in the tested domains. We will add the precise fixed-second definition, move the compute–quality interpretation into the main discussion, and include the complexity accounting above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.