- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The authors audit a fixed language-model controller that decides which questions receive additional reasoning and which answers are returned. In this paper, paired cached answers and state-scrambling interventions hold query counts and answer coverage fixed, while a two-stage decomposition attributes the observed accuracy effect to query allocation and answer selection. Experiments use two primary models on MMLU and ARC Challenge, with additional prompt, length-cap and checkpoint analyses.
Critical strengths
- Cached outcomes for both branches and exact query/coverage budgets separate state alignment from changes in branch capability.
- The authors report corrections and corruptions, stronger first-pass-confidence baselines, selected refitting checks, and six prompt/cap settings.
Critical weaknesses
- The sequential policy discards 284 of 550 queried answers, while the joint allocator and no-verifier controller achieve higher accuracy point estimates. The central answer-versus-query decomposition is not repeated for either alternative, leaving unclear how much of its conclusion follows from the chosen policy.
- The study applies familiar feature-reassignment and path-averaging ideas to one controller family on two reused multiple-choice question windows, with Qwen reaching every tested generation cap. It does not demonstrate that this audit materially changes conclusions about an existing adaptive-compute method.
- The paper does not give literal prompts/chat templates or the logistic regularization strength, and Appendix F.3 does not display the four intervention-cell accuracies promised by Table 2. (there is also no supplementary material).
Questions for authors
See above.