acceptodds
Under review as a conference paper at ICLR 2027

Separating Answer Selection from Computation in Adaptive Language-Model Policies

Abstract

A confidence-guided policy can improve answered accuracy by choosing which answers to return as well as by allocating additional reasoning. We test a fixed controller through interventions that preserve cached branch outcomes, exact query counts and answer coverage. Across two open-weight backbones on MMLU and ARC Challenge, the 25%-query, 80%-coverage controller exceeds state scrambles by 7.50 macro accuracy points. Query-only and abstain-only controls yield gains of 1.55 and 6.94 points at their respective budgets. The results establish that item-aligned state matters for this controller under fixed budgets.

Primary area:
foundation or frontier models, including LLMs
Keywords:
test-time computation, selective prediction, confidence-based routing, causal evaluation, language models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_oAAcOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The authors audit a fixed language-model controller that decides which questions receive additional reasoning and which answers are returned. In this paper, paired cached answers and state-scrambling interventions hold query counts and answer coverage fixed, while a two-stage decomposition attributes the observed accuracy effect to query allocation and answer selection. Experiments use two primary models on MMLU and ARC Challenge, with additional prompt, length-cap and checkpoint analyses.

Critical strengths
  • Cached outcomes for both branches and exact query/coverage budgets separate state alignment from changes in branch capability.
  • The authors report corrections and corruptions, stronger first-pass-confidence baselines, selected refitting checks, and six prompt/cap settings.
Critical weaknesses
  • The sequential policy discards 284 of 550 queried answers, while the joint allocator and no-verifier controller achieve higher accuracy point estimates. The central answer-versus-query decomposition is not repeated for either alternative, leaving unclear how much of its conclusion follows from the chosen policy.
  • The study applies familiar feature-reassignment and path-averaging ideas to one controller family on two reused multiple-choice question windows, with Qwen reaching every tested generation cap. It does not demonstrate that this audit materially changes conclusions about an existing adaptive-compute method.
  • The paper does not give literal prompts/chat templates or the logistic regularization strength, and Appendix F.3 does not display the four intervention-cell accuracies promised by Table 2. (there is also no supplementary material).
Questions for authors

See above.

Reviewer_ZffxOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the relative contributions of computation allocation and answer selection in adaptive language-model policies. The authors construct a fixed controller that predicts the correctness of direct and deliberated responses, allocates a 25% deliberation budget, and selects 80% of questions to answer. They introduce a state-scrambling intervention that permutes item-level confidence features while holding the controller parameters, cached branch outcomes, and allocation budgets fixed. Experiments on MMLU and ARC Challenge with Qwen3-4B and Granite-3.3-2B show a 7.50 percentage-point improvement in answered accuracy over state-scrambled controls. A same-budget decomposition attributes approximately 6.35 points to answer selection and 1.12 points to computation allocation. Additional analyses examine feature baselines, alternative prompts, longer deliberation budgets, and a third model.

The main takeaway is that item-aligned confidence information influences both routing and answer selection, with answer selection accounting for most of the observed contrast under the studied controller.

Critical strengths

S1. Explicit separation of two policy decisions.**

The four-cell intervention distinguishes state dependence at the computation-allocation and answer-selection stages. The path-averaged decomposition is more informative than simply comparing an adaptive system against a nonadaptive baseline.

S2. Multiple robustness analyses.**

The paper goes beyond its primary experiment by examining alternative feature sets, input-stratified permutations, controller refitting, longer deliberation branches, different prompts, and a third backbone. These experiments strengthen the conclusion that the observed alignment effect is not restricted to one prompt or model.

Critical weaknesses

Main weakness: The central result compares a controller receiving correctly aligned confidence features against the same controller receiving features randomly reassigned from other examples. However, the controller is explicitly trained to exploit associations between these features and answer correctness. Destroying their alignment is therefore expected to degrade performance. Although this is a valid diagnostic of feature dependence, it is a relatively weak test of whether the proposed architecture offers a meaningful advantage over simpler policies.

More importantly, the paper's own experiments show that: The state-only controller achieves 0.7710 accuracy, slightly higher than the full controller's 0.7694. Removing both verifier-derived features produces 0.7750 accuracy, again higher than the full controller. The no-query selective policy already achieves 0.7600 accuracy, compared with 0.7694 for the full controller.

These results suggest that the more elaborate controller provides little demonstrated improvement over relatively simple first-pass confidence-based selection. The strong 7.50-point improvement over scrambled states therefore should not be interpreted as evidence of a similarly large practical improvement over competitive policies.

The format of the paper seems not to align with the requirement from ICLR.

Questions for authors

Please see above weakness for my concerns.

Reviewer_W3ZuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper audits a fixed two-stage controller that decides which questions receive a deliberated answer (exactly 25% of items) and which answers are returned (exactly 80%). A direct and a deliberated answer are cached for every question. Two logistic predictors of branch correctness are fit on five pre-deliberation confidence features and three surface features. The audit scrambles complete state vectors across items while holding fitted parameters, branch outcomes and budgets fixed, and decomposes the resulting loss into query-allocation and answer-selection credits.

The claimed findings are: (i) the intact controller beats state scrambles by 7.50 macro accuracy points across Qwen3-4B and Granite-3.3-2B on MMLU and ARC-Challenge; (ii) answer selection accounts for most of this effect (6.35 points) versus query allocation (1.12); (iii) first-pass confidence is a competitive baseline and the verifier's incremental value is unresolved; (iv) adding queries to a no-query selective policy at 80% coverage gains 0.94 points.

Critical strengths
  1. The paired-branch, exact-budget design isolates what it sets out to isolate. Both branches are generated before any fitting, query and coverage counts are exact, fitted parameters are frozen, and the pass rule (Δ ≥ 0.020, one-sided p ≤ 0.05, at least 3/4 nonnegative cells) was fixed before evaluation. Branch capability and budget are therefore removed as explanations for the scramble gap. The reported numbers are internally consistent across the main tables and appendices, and the released branch bank makes the analysis replayable.

  2. The paper reports results that cut against its own framing, and one of them is its most informative result. The full controller (0.7694) is numerically below the no-verifier (0.7750) and state-only (0.7710) variants, querying adds only 0.94 points over no-query selective answering, and 284 of 550 queried items are later abstained. The Phi-3.5-mini bank is the cleanest evidence in the paper: deliberation lowers accuracy (MMLU 0.6771 to 0.5771, ARC 0.8750 to 0.7125), yet the alignment contrast stays positive (0.0760, 0.0522). This shows the gain does not depend on deliberation helping. All 24 model/task/setting contrasts across six prompt and token-cap settings are also positive.

Critical weaknesses
  1. The headline contrast is close to guaranteed and overstates the value of state. The scrambled controller (0.6945) is no better than the random matched policy (0.6907), and it is worse than the constant-state controllers (0.7204, 0.7202) and the input-only controller (0.7232). Scrambling therefore does more than remove information: it supplies misleading information to the frozen model, costing about 2.6 points beyond ablation. Against a state-ablated null, state is worth roughly 4.9 points (0.7694 − 0.7204), not 7.5. Even this is largely the known result that confidence-ranked abstention beats random abstention: direct no-query answering at 80% coverage already reaches 0.7600 versus 0.6907 for random. A pass threshold of Δ ≥ 0.02 against this null cannot realistically fail, and p = 0.0005 is the floor of 2,000 permutations. The abstract's "establish that item-aligned state matters" and Section 6's "causally improves" claim more than the test supports. The test measures the reliance of a frozen predictor on a bundle of five highly correlated features (up to 0.992, Table A1), not a causal effect of confidence.

  2. "Answer selection dominates computation" holds at a single operating point with a weak computation arm, and the audited controller is not shown to be better than simple alternatives.

    • The decomposition is labeled exploratory yet carries the title-level claim, and it is run only at q = 0.25, c = 0.80.
    • The interaction (−1.09 points) is as large as the entire query credit (1.12). From Table 2, query alignment is worth about 0.6 points when the answer stage is intact and about 1.7 when it is scrambled, so the two decisions act as substitutes and the 6.35 versus 1.12 split depends on the path-averaging convention.
    • The computation arm is weak. 2,164 of 2,200 deliberation traces hit the 96-token cap, and Qwen still reaches the cap at 512 tokens. The query signal is a difference of two logistic models. The oracle shows large unused query headroom (Table I2: 71.18 fitted versus 77.58 oracle at full coverage), and query credit grows from 1.12 to 2.12 at 384 tokens. The split therefore describes this budget, truncated branch and estimator, not the two functions in general.
    • The controller is not the best policy tested. It trails the no-verifier and state-only variants numerically, and its AURC (0.1473) is worse than first-pass state, entropy and max probability alone (0.1388, 0.1395, 0.1410). The joint allocator's gain interval includes zero. Net utility charges 0.05 per deliberation call but nothing for the verifier pass on every item, and the +0.0025 utility advantage over no-query answering would not survive that cost. Runtime and FLOPs are not measured.
Questions for authors
  1. Please report Δ against a state-ablated null (constant or mean state) alongside the scramble, and the four-cell decomposition against that null. How much of the 7.50 remains?
  2. Please report the order-specific query effect with the answer stage intact (the real controller's regime), not only the path-averaged credit.
  3. How were the MMLU windows chosen? "Offset 5600" and "question windows" suggest contiguous slices. MMLU is ordered by subject, so a contiguous slice would cover a narrow subject band and could differ systematically between development (validation) and evaluation (test). Please report subject coverage for both.
  4. What is the end-to-end cost (verifier on every item, 25% deliberation) of the full policy versus first-pass-confidence selective answering at 80% coverage, and does the 0.94-point gain survive that cost?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.