acceptodds
Withdrawn from ICLR 2027

Screen-Then-Select: Learning When Temperature Selection Helps LLM Reasoning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Sampling temperature controls the diversity of language-model reasoning, yet aggregate accuracy can conceal substantial differences in how inputs respond to temperature changes. We characterize this heterogeneity through multi-model MATH sweeps and high-sampling continuation experiments, connecting modest aggregate changes in a conventional temperature range to large local opportunities. Motivated by these findings, we propose Screen-Then-Select, an answer-assisted framework that separates screening for responsive candidates from selecting their temperatures. A small pool of reference-checked responses identifies mixed-success questions; a lightweight prompt-only selector then chooses a temperature for each fresh answer without updating the language model. In a MATH prefix study, correct-answer counts enrich inputs with large temperature responses. Trained on MATH and frozen for transfer, the selector improves individual-answer accuracy on screened GSM8K and OlympiadBench subsets relative to fixed temperatures selected for that metric on separate development questions, including a 7.1-percentage-point gain on GSM8K. Matched rejected controls show smaller observed gains, while policy decomposition identifies a shared temperature change as the source of the GSM8K improvement. Together, these results provide an empirical foundation and a lightweight framework for targeting temperature selection where its observed benefits are concentrated.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
llm, inference, decoding, temperature

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_azYkOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Screen-Then-Select is a framework for adaptive temperature selection in LLM mathematical reasoning. It first screens questions using correctness counts from sampled answers, then trains a lightweight MLP to predict a sampling temperature from prompt features. The method is evaluated on MATH, GSM8K, and OlympiadBench, with reported improvements on selected subsets.

Critical strengths
  1. The paper provides an empirical analysis of temperature sensitivity across questions, including high-sampling experiments and statistical controls.
Critical weaknesses
  1. Input-dependent temperature selection and adaptive decoding have been extensively studied in prior work. The proposed framework mainly introduces a correctness-based screening heuristic, without establishing a sufficiently technical contribution in the current version. On GSM8K, the learned selector assigns the same temperature (T=0.9) to every screened question. The reported improvement therefore comes from changing a fixed temperature rather than question-specific adaptation. Furthermore, the evaluation relies on a single rollout block without repeated training or generation runs (test-time scaling by BoN), I therefore have concerns about robustness.

  2. The screening procedure requires ground-truth answers and retains questions for which some sampled responses are already known to be correct. If correct answers can already be found, the practical motivation for generating additional answers with a learned temperature is unclear to me. The evaluation only covers small, selected subsets of the benchmarks, without showing end-to-end benefits under a matched inference budget.

Questions for authors
  1. Can the authors demonstrate meaningful improvements over a fixed temperature calibrated specifically for the screened population, with repeated runs to establish robustness? By the way, if only math benchmarks are included, the authors can write "LLM mathematical reasoning" instead of "LLM reasoning."

  2. What practical inference setting justifies correctness-assisted screening when verified correct answers are already available? How does the method compare with directly using those answers or allocating the same inference budget to additional sampling?

1 comment

Dear authors,

Please provide a rebuttal by the deadline if you want me to reaccess the paper.

Reviewer_bDzJOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper asks whether the benefits of changing sampling temperature are concentrated in a subset of questions, and whether a selector can learn which temperature to use. It first screens questions using repeated generations and reference answers, then applies a selector trained on MATH to choose a temperature for fresh responses. The experiments examine temperature responsiveness, enrichment through screening, and transfer to GSM8K and OlympiadBench.

Critical strengths
  1. The authors use randomization tests, independent split-half selection and evaluation, and question-clustered analyses to examine whether screened prefixes have larger temperature responses. These experiments provide more support than a temperature sweep or a few selected examples would.

  2. The appendices report label reliability, differences between same-pool and cross-pool evaluation, the contribution of a constant temperature, and negative results. These analyses help distinguish the existence of a temperature-selection opportunity from the ability to learn a useful selector.

Critical weaknesses
  1. All 30 screened GSM8K questions use T=0.9, so a constant temperature reproduces the full 7.08-percentage-point gain. MATH individual-answer accuracy is unchanged at 54.17%. OlympiadBench changes only 5/56 choices, yielding +1.56 points (95% CI [-0.67, 4.46]). The learned selector's added value over a suitable constant remains unestablished.

  2. The target-task fixed baselines are tuned on 64 unscreened development questions but evaluated on screened ones. Screened-development selection reduces the GSM8K gain to 2.08 points (95% CI [-4.17, 8.33]), yet leaves only 2 GSM8K and 9 OlympiadBench development questions (Appendix E.3). A better subgroup-wide temperature could explain the gain.

  3. The GSM8K fixed baseline scores 91.67% on rejected controls versus 58.33% on screened questions, leaving different room for improvement. Table 2 gives no confidence interval for the between-group gain difference. These target-task policy comparisons do not separate temperature responsiveness from difficulty and headroom.

  4. The screened test sets contain 30 GSM8K, 56 OlympiadBench, and 45 MATH questions. Target-task evaluation uses one rollout block of eight answers per question and policy. Table 19's interaction-label split-half correlations are 0.2466 and 0.1931, using eight answers per temperature per half. Replication across generation and training seeds is not shown.

  5. As acknowledged, screening needs reference answers and eight pilot generations per question, limiting applicability to unlabeled new questions. The paper does not report full-set accuracy when rejected questions retain the baseline policy, or account for screening cost in an end-to-end comparison.

  6. Against fixed temperatures tuned for SC@8, the differences are zero on GSM8K and -10.71 points on OlympiadBench (95% CI [-25.00, 3.57], Section 5.3). The latter does not establish inferiority, but superiority for majority-vote reasoning is also not demonstrated.

  7. Much of the writing reads to me as AI-generated and insufficiently edited. Repeated qualifications and shifts between prefix diagnostics and prompt-only policy evaluation make the main argument difficult to follow.

Questions for authors
  1. All 30 GSM8K questions use 0.9. Does the ungated MLP also predict 0.9 for all of them, or does the gate revert other predictions to the default?

  2. Why is the main fixed temperature selected on unscreened development data when the comparison is made on screened questions? How do the results distinguish recalibrating a common temperature from learning per-question choices?

  3. What is the 95% confidence interval for the screened-minus-rejected gain difference in Table 2? The separate intervals for each group do not answer this.

  4. If rejected questions retain the original fixed temperature, what is the accuracy change over the full original test set? Results on the screened subset and sampled rejected questions do not establish the overall gain.

  5. Appendix E.4 reports three wins, one loss, and one tie among the 5 OlympiadBench temperature changes. What is the accuracy difference for each question? Is the overall 1.56-point gain driven mainly by one question?

Reviewer_JYZ7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how the effect of sampling temperature varies across mathematical reasoning problems and proposes Screen-Then-Select. It first generates eight responses per question and uses reference answers to retain questions with three to five correct responses. A small MLP trained on MATH then uses prompt features to choose a temperature for each new answer from a frozen Qwen3-8B. The paper combines temperature sweeps across several models and experiments on fixed reasoning prefixes with method evaluations on screened GSM8K and OlympiadBench subsets. It reports gains over fixed temperatures chosen on separate development questions and distinguishes gains from a common temperature change from those due to question-specific choices. It also examines the reliability of the correctness-based training labels.

Critical strengths
  1. The paper distinguishes temperature effects from actual selector gains. Choosing temperatures on one half of the responses and evaluating them on the other, together with the label-noise analysis, provides useful checks (Table 1, Appendix F).
  2. The authors clearly report negative and inconclusive results, including the constant GSM8K policy and the uncertain OlympiadBench gain (pp. 7–9).
Critical weaknesses
  1. It is unclear how much the learned selector adds over a fixed temperature. All 30 screened GSM8K questions receive temperature 0.9, so the 7.08 percentage-point gain comes entirely from a common temperature change. MATH individual-answer accuracy is unchanged, and OlympiadBench gains 1.56 points with a 95% confidence interval of [−0.67, +4.46] (Appendix E).
  2. The GSM8K result depends on how the fixed baseline is chosen. Selecting it on screened development questions reduces the gain to 2.08 points [−4.17, +8.33], but only two development questions pass the screen (Table 16). This makes the comparison difficult to assess reliably. Table 2 also gives no confidence intervals for the difference in gains between screened and rejected groups. The temperature analysis is useful, but the benefit of learning a selector remains unclear. I am not yet convinced that the empirical findings alone are a sufficient contribution.
Questions for authors
  1. Could the authors use the existing OlympiadBench results to compare the final MLP against randomly reassigned temperatures with the same frequencies, and report the difference with a confidence interval?
  2. Are more independently screened development questions available to check the fixed-temperature baseline? Please also report confidence intervals for the differences in gains between the screened and rejected groups.
  3. The paper studies temperature effects on AIME and with larger Qwen3 models, but tests the final selector only on Qwen3-8B. Could the authors evaluate Screen-Then-Select on AIME using a larger model, such as Qwen3-14B or 32B, against a fixed temperature chosen on separate development questions?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.