- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how to allocate a shared completion-token budget across questions for a fixed language model. The proposed Solve-then-Learn framework first estimates question-specific accuracy and cost at several token caps, uses a Lagrangian solver to construct allocation labels, and trains a lightweight classifier to predict these labels before generation. A separate sequential executor enforces the total budget by reserving a minimum allowance for unanswered questions. The paper provides finite-table allocation and imitation-error bounds, together with a budget-feasibility guarantee. Experiments cover two mathematics datasets and three models. The primary results report improvements over the strongest evaluated baseline in 17 of 18 settings, averaging 3.90 percentage points, although these results use a cost-calibrated variant that differs from the direct-budget training procedure emphasized in the main description.
Critical strengths
1.The Lagrangian formulation explicitly coordinates compute allocation across questions without requiring diminishing returns in accuracy.
2.The lightweight policy and hard-budget executor support deployment without modifying the underlying model or sampling preliminary answers.
3.Experiments across three models and two datasets show consistent improvements, with ten comparisons against Random remaining significant after multiple-testing correction.
Critical weaknesses
1.The headline results use an earlier cost-calibrated variant, leaving the emphasized direct-budget training procedure insufficiently validated by the primary evaluation.
2.The absence of stronger learned-allocation baselines makes the specific contribution of oracle-label supervision difficult to establish.
3.Incomplete generation records and uncertainty estimates conditioned on fixed response pools limit reproducibility and assessment of robustness to fresh sampling.
Questions for authors
1.Which training variant is intended as the final method, and do the headline conclusions hold under a matched comparison of direct-budget and cost-calibrated training?
2.How does STL compare with learned utility or marginal-reward predictors using the same training data, input features, candidate caps, and budget executor?
3.Can the missing mini/nano sampling settings be recovered, and do the gains persist across independent training runs and newly generated response pools?