acceptodds
Withdrawn from ICLR 2027

Beyond Correctness: Compute-Responsive Data Selection for Reasoning Model Post-Training

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Reasoning language models increasingly rely on test-time compute scaling, yet most existing work studies how to allocate inference computation while treating the model itself as fixed. We ask a complementary question: can reasoning models be post-trained to make better use of a given amount of test-time compute, and which examples are most useful for this goal? Our key observation is that reasoning problems differ substantially in their response to additional computation. Some are already solved under limited compute, others remain unresolved even after further reasoning, while a distinct subset changes from complete initial failure to success when additional reasoning is provided. We call these compute-responsive (CR) examples and introduce Compute-Responsive Data Selection (CRDS), a post-training data-selection principle that prioritizes examples with empirically demonstrated marginal utility from additional reasoning. On Qwen3-8B, SFT on CR examples recovers 45.3% of held-out low-compute failures, compared with 40.6% when training on already-solved examples. The advantage extends beyond the selected CR split: SFT-CR reaches 51.8% overall accuracy on an independently sampled test distribution, with the gain concentrated on compute-responsive problems. CR training also shifts the compute–accuracy frontier across inference budgets from 512 to 4,096 tokens, outperforming matched non-CR training throughout and reaching 48.1% recovery at the largest budget. The same pattern is supported across multiple data sources, remains visible under on-policy distillation, and extends to Llama-3.1-8B-Instruct when CR sets are reconstructed separately for that model. Better test-time compute utilization can therefore be learned through post-training, with recoverable responses to additional reasoning providing a particularly informative signal for selecting training examples.

Primary area:
foundation or frontier models, including LLMs
Keywords:
test-time compute, reasoning models, post-training, compute utilization, on-policy distillation, data selection

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_L7BYOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies post-training data selection for reasoning models from the angle of test-time compute utilization. They propose Compute-Responsive Data Selection (CRDS): fine-tune on the successful blind trajectories of CR problems (616 examples, LoRA SFT) and compare against size-matched controls (SFT-Wrong, SFT-Random, SFT-Already-Solved).

Critical strengths
  1. Careful construction of the CR signal. The paper is unusually candid about the confounds that make a naive "wrong at low budget, right at high budget" rule misleading (truncation/completion effects, deterministic audit reproducing the same error, grader-format false recoveries)
  2. Controlled, size-matched comparison with converging evidence. All SFT variants share the base model, 616 examples, LoRA config and optimizer, differing only in which trajectories are used, and the CR > Already-Solved > Random ordering recurs across the held-out split, the stratified mixed test set, four token budgets, per-source decomposition, an OPD objective with a 32B teacher, a second model family, and a 3-seed replication (Table 15).
Critical weaknesses
  1. Several appendix sections appear unfinished ("Missing latency values will be filled from the unified timing evaluation" in Table 12; "Exact examples will be inserted after final manual selection" in Table 13; Appendix L text).

  2. The central claim ("compute responsiveness" is the useful signal, not merely "train on problems the model currently fails, with correct solutions") is not isolated by the controls. The missing control is a difficulty-based or failure-based selection with correct targets that does not use the CR probe, e.g. SFT on Unresolved problems with gold/teacher solutions, or SFT on the same number of failed problems chosen by a cheaper difficulty proxy (low pass@k), or plain RFT on the full pool. Without this, the gain over SFT-Already-Solved is equally well explained by "hard, in-distribution problems with correct targets help more than easy ones", which is the known STaR/RFT result. The related-work section does not discuss STaR, RFT, expert iteration or rejection-sampling SFT.

  3. (a) The "overall" +1.2-point gain on the mixed set depends entirely on the arbitrary 14% CR share of that set, while the already-solved stratum drops by ~14 points; the compute-accuracy "frontier" (Table 3) is reported only on the 106 CR problems where the base model is 0% at every budget by construction, so it does not show that the model uses compute better on a general distribution. (b) Qwen3-8B run with thinking disabled, look forward to see the results under thinking mode.

Questions for authors

Q1. Can you add a control that separates "failed problem + correct target" from "compute-responsive"? For example, SFT on 616 Unresolved problems using gold NuminaMath/DeepScaleR solutions or Qwen3-32B solutions as targets, and/or SFT on 616 failed problems selected by a difficulty proxy without the CR probe

Q2. Where exactly do the SFT-Random targets come from (random problems from the whole pool with a correct self-generated trajectory? gold solutions?

Q3. How does SFT-CR perform on standard benchmarks (e.g. MATH-500, AIME24/25, GSM8K) at the four budgets compared with the base model and SFT-Already-Solved?

Q4. Qwen3-8B run with enable_thinking=False, Do the CR phenomenon and the SFT-CR advantage persist in thinking mode, where test-time compute is actually used by the model?

Reviewer_mxnmOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies whether post-training data selection can help reasoning models make better use of test-time compute. The motivation is, some problems can already been solved with very limited budget, some problems can not be solved even with more test-time budget, so it's valuable to prioritize training model with compute responsive (CR) examples. The definition of compute responsive example is: problems that cannot be solved within limited budget, and can be solved with a larger budget. This definition excludes failures caused solely by truncation.

The paper tests this sampling strategy with both supervised fine-tuning and on-policy distillation. They compare with model trained with same amount of random/all-correct/all-wrong samples, and show that training with compute responsive samples can improve the performance. The paper also reports results on a mixed evaluation set and a second model family.

Critical strengths
  • The motivation is clear and intuitive. Since additional compute does not benefit every problem equally, selecting examples that can be solved with additional reasoning is a reasonable direction for improving post-training.

  • The experiments cover several settings, including both SFT and OPD, different inference budgets, and two model families. The comparisons use matched training configurations, and CR-selected training consistently performs better than the alternatives considered.

Critical weaknesses
  • My main concern is whether the performance gain comes from the proposed sampling strategy or from differences in the training trajectories, such as whether they are complete and fit within the evaluation budget. In Table 12, the other SFT baselines have higher truncation rates after training than SFT-CR. I wonder whether differences in the training data contribute to this result. An important baseline would be SFT on randomly selected examples with non-truncated trajectories, ideally generated using the same procedure as the CR targets.

  • The writing is sometimes hard to follow because several concepts are mentioned before they are clearly defined. For example, “limited compute,” “deterministic revision,” and “audit-based revision” are introduced without enough explanation at first mention. The statements about repeated errors and formatting-induced false recoveries around lines 92–94 and 199–203 are also difficult to understand without reading the appendices. Brief explanations or examples in the main text would help.

  • I would like to better understand how much the results depend on the CR construction setting. The initial pass uses greedy decoding with a 2048 token budget, while the additional attempts use sampling, a larger budget, and sometimes revision prompts. It is therefore unclear which part of this procedure is most important for identifying useful training examples. The evaluation across different budgets is helpful, but it would also be useful to understand whether collecting CR examples with a different initial budget leads to similar results.

Questions for authors

Q1: In Figure 1(b) and around line 77, what exactly does “limited compute” mean? It would help to state the budget and decoding settings when this concept is first introduced.

Q2: For identifying a complete initial failure, the paper uses one greedy response. Have you checked whether the CR labels are stable across repeated sampled attempts or different seeds? Similarly, how much does the selected CR set change with the number of recovery attempts?

Q3: Around line 189, does blind re-solving mean generating a new trajectory from the original problem alone? For audit-based revision, does each attempt receive the same initial failed response, or does it build on previous revisions? Please make this distinction clearer in the main text.

Q4: How exactly are answers normalized? The appendix provides examples, but more details about the matching procedure and how ambiguous cases are handled would help.

Q5: Why was 2,048 chosen as the initial budget? Does it represent a practical inference setting? Although Table 3 shows improvements across evaluation budgets, would the method still work similarly if the training examples were selected using a different initial budget?

Reviewer_QjckOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This work proposes compute-responsive data selection to better use test-time compute by focusing on problems that would benefit from more compute, with compute here meaning prompt retries. They find that taking this extra step of selecting data samples based on this criterion provides performance improvements relative to naive SFT, even though it also helps with compute-responsive problems.

Critical strengths
  • I think there may be a good idea at the core of this paper: that if we pick the samples most amenable to extra compute and provide and allow them to learn from that extra compute, we can almost surgically target the model's weak points.
Critical weaknesses
  • First of all, I thank the authors for submitting their code. Unfortunately, a bug I found in it seriously undermines their findings. The answer verifier does not always correctly identify right/wrong answers. For example, for DeepScaleR 44625dac127f, the ground truth is 19, and the answer the model provided contains "Gerta is 19 years old" but that's graded as wrong. In another case, NuminaMath-CoT 9fe6d1bcc3e9 the ground truth is "A and D" but just "A" is graded correct. By my estimate this affects 14% of the samples. Basically, this calls everything that happens after you determine whether the samples were indeed compute-responsive into question. Normally, these wouldn't be too severe—most verifiers are imperfect in some way—but (see my fourth point), it seems that this is especially pertinent given the nature of your contribution: a way to select training sample that would be especially helpful in better learning. This was also largely self-inflicted since you could have used a verifier library like math-verify.

  • It's my understanding that you are creating the train/val/test splits that you train and evaluate on are based on the training data from Numina, DeepScaleR, etc. First, I would ask why you didn't just use normal test benchmarks like MATH-500. This calls into question the actual generalizability of this method. I also think this is problematic because, as I see it, you're "testing" on training data which the models have almost definitely seen before, before your own SFT even came into play.

  • Why is reasoning disabled? It seems to me that when doing this kind of work, you would at least want to understand what the model's actual best performance is. We know reasoning models do better than non-reasoning models on math tasks (which you focus on), so I'm perplexed by this decision.

  • A less severe concern regarding novelty: I may be missing something here, but this seems like rejection sampling with extra steps. While that is not bad per se, this works novelty takes a knock. And if my understanding is correct, the actual contribution here is how you select the samples.

  • A relatively minor concern: parts of the introduction seem repetitive. For example, paragraphs 3 and 4 could probably be merged. Figure 2 is a bit hard to follow. It seems like it's supposed to be a flowchart, but it doesn't follow flowchart conventions.

Questions for authors
  • Same as my third weakness: why is reasoning disabled?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.