acceptodds
Under review as a conference paper at ICLR 2027

A Primer in Post-Training Reasoning Data: What We Know About How It Works

Abstract

Post-training has become a major driver of recent progress in large reasoning models, and reasoning data are often a key determinant of its success. Research on these data has grown rapidly, yet the evidence remains fragmented. This primer synthesizes over 150 works around four questions: what forms reasoning data take, what makes them useful, how they are constructed, and what scaling studies establish. Across mathematics, code, science, and interactive tasks, we compare apparently conflicting findings to clarify the conditions under which conclusions hold and the questions that remain open. We distill this evidence into seven practical lessons and reporting recommendations to guide data construction, assess feedback reliability, and evaluate claims of data efficiency and capability improvement. We conclude by outlining directions for future research.

Primary area:
foundation or frontier models, including LLMs
Keywords:
post-training, reasoning data, large reasoning models, chain-of-thought, reinforcement learning with verifiable rewards, supervised fine-tuning, data curation, data scaling, survey

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_sLr7OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper synthesizes 156 works on post-training reasoning data, focusing on their forms, usefulness, construction, and scaling evidence. It examines how the value of reasoning data depends on the learner and the learning interface, distills seven practical lessons and reporting recommendations, and discusses apparently conflicting findings across existing studies. The paper also proposes three controlled comparisons for future research.

Critical strengths
  1. The survey is well organized and covers a broad range of reasoning-data practices. Its problem-oriented structure and learning-interface perspective help connect SFT, RLVR, and distillation. 
  2. The paper records important conditions behind reported comparisons and distinguishes unresolved findings.
Critical weaknesses
  1. The paper's positioning relative to existing surveys is insufficiently developed. Luo et al. (ACL 2025), "A Survey on Efficient Large Language Model Training: From Data-centric Perspectives," already review data selection, data quality, synthetic data, and distillation. Yu et al. (ACL 2026), "A Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and Solutions," cover data-efficient RL. These topics overlap substantially with Sections 4–6, yet the paper does not clearly explain how its scope and synthesis differ from these related surveys. An explicit comparison would help establish the distinctive contribution.  2.The overall quality of the figures and tables needs substantial improvement. Several figures appear insufficiently refined, with overly dense information, small fonts and annotations, and poor readability. In addition, some tables contain imprecise conceptual classifications. For example, Table B.5 categorizes training issues arising from reward aggregation and credit assignment as checker errors, without clearly distinguishing errors in the checker's judgments from problems in how reward signals are used during training.
Questions for authors

1.In the three controlled comparisons proposed in Section 7, which would the authors prioritize as the most informative next step, and why?

3 comments
AuthorsOpenReview ↗

Response to Reviewer sLr7

Thank you for the review. We are glad you found the organized synthesis of scattered evidence, and the learner and route as shared variables, valuable. Your two weaknesses pointed at things we could usefully fix: the positioning against Luo et al. and Yu et al. is now explicit in §2 and Table A.2, several figures are redrawn, and Table B.5 now separates a wrong verdict from a mishandled reward. Section 7 now also reports two of the three comparisons you asked about.

W1. Positioning.

We agree, and thank you for pointing to these two surveys. In brief: Luo et al. organize the literature by data operation, from selection to synthesis, and Yu et al. by RL technique under data scarcity; like other recent surveys, such as Chen et al. (2025a) on the capabilities of long chains of thought, they map which methods exist. The primer complements them with a question they leave open, one a practitioner meets whenever two results disagree: which of them holds for a given model and training method? We read each finding together with the learner it was measured on, the route by which its data reached that learner (SFT, RL or on-policy distillation) and what its comparison held fixed. Read this way, six of the seven conflicts we collect differ mainly in learner or route (Appendix B.2), and our new experiments test this reading directly: in every family, the best teacher for the smallest student is the worst for the largest. §2 now states this boundary in more detail, and Table A.2 sets these surveys side by side by the question each starts from.

W2. Figures.

Thank you for the suggestion. Figures 1–4 are redrawn at their printed size: no label in them is now smaller than 6 pt, where most were before, down to 3.9 pt in Figure 2. The new Figure 6 follows the same standard.

W2. Tables.

Thank you; Table B.5 now separates them. Summed step rewards and one score per response form their own group, Reward aggregation and granularity, and the caption states that this group falls outside the derivation above the table, which concerns errors in the checker's verdicts.

Q1. Which comparison first?

The teacher-by-student crossing, and we have now run it. Its prediction is the sharpest, since s1 and OpenThoughts point in different directions about which teacher is better. Across three teachers and ten students from three model families, the ranking reverses between the smallest and the largest student of each family. We also ran the comparison of random against systematic checker errors. Please see Q2 in our main response for both results.

We hope these changes address your concerns, and we would be glad to answer any further questions.

Thank you for the response. My concerns have been addressed and I will keep my score.

AuthorsOpenReview ↗

Thank you for your guidance

Thank you for reading our response and for confirming that your concerns have been addressed. We are sincerely grateful for your thoughtful guidance. We took each of your suggestions to heart and spent considerable time refining the paper in light of them, from how the primer positions itself to the legibility and design of our figures.

Your comments have made the paper clearer and more readable, and have raised the quality of its writing and presentation throughout. With your careful review and valuable suggestions, together with our team's dedicated effort in polishing the paper, we truly believe it will be helpful to, and appreciated by, many more readers. Thank you again for your time and your positive assessment.

Reviewer_DDpxOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

There has been substantial work on post-training in recent months/years, including around what data to use/create. Unfortunately, the methods and evidence are scattered, across literally (low) 100s of articles. The current paper sets out to address this.

"This primer synthesizes over 150 works around four questions: what forms reasoning data take, what makes them useful, how they are constructed, and what scaling studies they establish."

They distil the evidence into seven "practical lessons", with corresponding recommendations.

Critical strengths

The scattered nature of post-training results, methods, data and the like is a bit of a nightmare for working in the field. This synthesis will surely be extremely useful to the community.

Not only is the premise highly valuable, but the presentation is excellent too. It was clear as the reader what I was reading and why. The citations - the key part of the document - are done very well. All to often, authors write something like, "Post-training has been shown to improve performances on maths [3, 41, 72, 87]." This really gives very little information to what the references say. The authors here clearly explain what each citation shows. I click on many of them: the ones of particular interest to me, which I was able to discern from the text in the body of the paper; I saved many of these and shared with my post-training colleagues!

The tables, figures and the like were fine. To be honest, the text was so clear that these weren't even necessary. I'm not suggesting that they're removed, of course - just praising the writing!

The AI-use statement says that AI supported "translation checks", suggesting non-native English speakers wrote it. They have really done an excellent job. Just getting AI to write a document like this without precise guidance would provide something genuinely awful - absolute slop. The authors have clearly taken great care with the writing.

Critical weaknesses

The main weakness that I need to highlight is two papers that they cite from mid 2025:

  1. Spurious Rewards
  2. RL with One Training Example

These papers have both been debunked: Incorrect Baseline Evaluations Call into Question Recent LLM-RL Claims.

To be honest, both are a case of "seems too good to be true" ("RL with random reward helps" and "can use only one datapoint to train"). And, as is normally the case in such situations, they indeed are.

On the writing front, whilst it was excellent overall, a small contents would've been appreciated. Reading each section alone was very clear, but I had basically no idea what the next section was going to be about. There is a small "We organize around four questions", but then these are in §§2–6, which is five sections. A slight reworking of this final paragraph in §1 would help.

Similarly, the sections jump straight into the content. Even just one/two lines expanding on the title before the main content would help. Some sections have this already, but not all.

Last, the explicit "Takeaway" at the end of §3 was very nice - a clear summary of the section. It would be nice of (pretty much) all the sections had this.

Questions for authors

Given the two citation issues above, I did get it to extract all the references and do a quick check online to see if any others had been debunked/similar. I copy the report below.

To be honest, it didn't do a great job - eg, I gave it #1 as an example, including the Notion doc, and it didn't even find #2 on its own. So, the report shouldn't be thought of as comprehensive.

I did get agents to check it after it was written, for accuracy, but naturally individual claims should be verified manually. The two points raised in §Weaknesses are still the primary issues.

Findings requiring qualification or clarification

Priority: baseline adequacy in two prominently cited RL studies. The issue is whether their comparisons establish the claimed size and interpretation of the RL benefit against a well-elicited starting model. Critics need not demonstrate that RL worsens performance. Different evaluation protocols prevent causal subtraction of scores; they do not dispose of a challenge to the adequacy of the reported baseline.

Chandak, Goel and Prabhu’s Incorrect Baseline Evaluations Call into Question Recent LLM-RL Claims raises this objection for both Spurious Rewards and RL with One Training Example.

1. Spurious Rewards: the headline gain needs a stronger-baseline comparison

Confirmed prompt sensitivity and baseline-adequacy concern. The primer’s +21.4-point Qwen2.5-Math-7B result (§4.1, Figure 2a; §7) faithfully reproduces Spurious Rewards, v2, Figure 1, p 2: 49.4% before training and 70.8% after random-reward training. Yet the same source’s Appendix I, Tables 4–5, pp 25–26 reports 68.8% without training under another prompt.

The two untrained prompt scores differ by 19.4 points, numerically close to the 21.4-point within-protocol RL increase. These are distinct comparisons: the prompt difference does not decompose the RL gain, and 70.8−68.8 is not a causal estimate of RL’s effect. The reported 49.4→70.8 increase remains a within-protocol result, but does not quantify the benefit over a well-elicited starting model. Faithful transcription alone does not address that baseline-adequacy objection. The source acknowledges prompt sensitivity; the primer should attach this concrete qualification to its central example, rather than rely on its general evaluation discussion (§6.3).

Notion’s overview compares the different, non-Math Qwen2.5-7B model. Its numbers should not be transferred to this example: the Math-model concern above is independently supported by the primary paper itself.

2. RL with One Training Example: the stronger starting-model score is genuine

Confirmed stronger starting-model score under a different evaluation. The original paper, v1, §3.3/Table 4 reports DeepSeek-R1-Distill-Qwen-1.5B improving from 71.0% to 78.0% MATH500. The official model card separately reports 83.9% for the released starting model under its evaluation protocol, before the one-example training. Notion’s overview gives 71.9 rather than the source’s 71.0; that discrepancy does not invalidate the stronger-baseline objection.

The v1 comparison uses an 8,192-token cap and greedy MATH500 evaluation (§3.1): its reported seven-point increase is within those conditions. The model card specifies a 32,768-token cap and, for benchmarks requiring sampling, temperature 0.6, top-p 0.95 and 64 responses per query to estimate pass@1. Its 83.9 is not a matched counterfactual for v1. More informatively, the revised paper, v3, Table 4/Appendix B.5 compares 82.9% → 83.9% under its own matched 32k sampled protocol: a much smaller, one-point gain under those conditions. This supports the concern about the claimed gain’s dependence on evaluation, without erasing the within-v1 result. The trained 83.9 does not refute the independently reported starting-model 83.9.

Relevance to the primer: §6.1/Figure 5, Tables B.1–B.2 row 6.1a and B.6 principally use Qwen2.5-Math-1.5B, 36.0% → 73.6%, not that DeepSeek subexperiment. The DeepSeek comparison warrants checking baseline adequacy in one-example studies, but does not itself overturn the plotted Qwen result. The primer should make that model/protocol scope clear when using the study to motivate data efficiency. A corresponding stronger untrained Qwen baseline was not established in this audit.

3. The s1 uncertainty discussion omits published paired item-bootstrap evidence

Confirmed inferential omission, not an arithmetic error. Appendix B.1, p 29 says the selected-versus-random AIME24 difference cannot be separated from question-sampling noise without per-problem results. Its approximate independent-binomial standard errors recompute correctly. However, the published s1 paper, §5.1/Table 2, p 20280 already reports a 95% paired item-bootstrap interval of [−26.7, −3.3] percentage points for random 1K minus selected s1K, excluding zero.

The primer should report this item-level paired result alongside its independent-binomial approximation. Conversely, s1’s 59K-versus-selected intervals include zero; this does not establish equivalence. These item-based analyses do not quantify training-run or evaluation-implementation variability. s1 discusses the latter—evaluation nondeterminism—in Appendix C, p 20289, not training-seed robustness.

4. Figure 5’s normalisation is not uniquely specified

Unresolved reporting detail, not a demonstrated impossible value. The purple one-prompt OPD annotation is reproducible: the reported scores 42.4 → 51.7, against 53.7 for full data, retain 82.3% of the improvement (Hou et al., v2, Tables 1–2).

For the green RLVR curve, Figure 5 does not identify the metric, full-data reference and checkpoint convention underlying “96%”. Different legitimate choices give different fractions. For example, best-MATH500 scores 36.0 → 74.0 against 75.4 yield 96.4%, whereas best-six-task-average scores give other values (Wang et al., v3, Table 3, p 7; Tables 8–9, p 21). Providing the plotted values and normalisation would resolve this without new experiments.

Verified checks and evaluation limits

  • Denominators: single-run AIME scores in Figure 3a/Table B.2 are attainable on 30 questions. TTRL’s 12.9% and 40.2% are also attainable because they average 16 samples; requiring multiples of 1/30 would be incorrect.
  • Process errors: Figure 2b’s rates reproduce as 7/200, 94/500, 161/500 and 259/500. These are selected correct-final-answer cohorts, not error prevalence in arbitrary training corpora (ProcessBench, §3.3/Table 2).
  • Redrawn data: Figure 2c matches Skywork’s 16-rollout distributions (§6.2); Figure 2d matches ReSyn’s fixed-16K-instance ablation (§5.2/Table 5); Figure 3c reproduces R-Zero’s pseudo-label accuracy against its assumed GPT-4o oracle: 79%, 69% and 63% (§4.4/Table 5). This is not independently established mathematical correctness.
  • Derivations and budgets: the no-signal probabilities in Table C.4, advantages in C.1 and pass@1/pass@64 counterexample in B.7 recompute correctly. B.4’s noise identity is valid under its stated class-conditional assumptions, not a general result for clipped GRPO. B.6–B.7 expose rollout, selection and annotation burdens; quantities vary by study and some costs are not stated. One prompt does not mean one training observation.
  • Leakage interpretation: this primer trains no models or releases new training data. Its cited test-time adaptation and benchmark-selected checkpoints raise evaluation-exposure and selection concerns, not demonstrated hidden answer-label leakage in a new experiment. No new regression/R-squared estimate or empirical run-level error bars are presented; B.1 does calculate approximate analytical binomial standard errors.

What the criticism screen changes

Of the seven articles criticised in Notion, only Spurious Rewards and RL with One Training Example are cited in the primer; both support substantive claims. Their baseline concerns should be distinguished from the peripheral historical corrections below.

There are genuine historical source defects, but their citation roles matter. FinQA’s released retriever code had a label-leaking table-formatting bug corrected in 2022 (official update notes); UltraFeedback corrected overall-score labels (official repository). Both appear here as background entries in A.3/Table A.1, not measurements supporting the central seven lessons. Hu et al., v5, has two distinct Table 3 issues: the PDF, p 8 and HTML disagree on correlation signs (eg Qwen2.5-3B/GRACE DS: +0.781 versus −0.781); in the PDF, the five displayed Qwen2.5-3B/RSR values average −0.6918, versus the reported −0.603. Neither issue directly refutes the separate nine-teacher Figure 1 observation used by the primer (§5.2, §7, Table B.6). These specific defects do not justify calling all cited evidence collectively “debunked”.

For positioning, Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning, Findings ACL 2025, §7/Figures 4–6 already examines teacher–student and task dependence in CoT distillation but is absent from the bibliography. It already documents the narrow observation that stronger teachers need not transfer best. The primer’s broader cross-route synthesis and reporting framework remain distinct; this comparison alone does not settle its overall novelty or value. This is older peer-reviewed work; arXiv-only work and work peer-reviewed on or after 17 July 2026 were not treated as mandatory comparisons.

Unchecked: underlying prediction logs, immutable dataset/checkpoint provenance and the companion search/figure records were not supplied. Numerical feasibility and source agreement do not independently reproduce model runs, seed variability or every source’s contamination controls. Detailed Notion section bodies were inaccessible; its overview/table and the relevant primary-source numbers, not every allegation, were checked. Uncorroborated allegations and inaccessible evidence remain unresolved, not confirmed defects.

3 comments
AuthorsOpenReview ↗

Response to Reviewer DDpx

Thank you for the care behind this review, and for reporting the limits of the citation screen. We are glad you found that our citations state what each source shows. Following your review we changed four places in the paper: the baseline qualifications on Spurious Rewards and on RL with One Training Example, the s1 paired intervals, and the normalisation behind Figure 5.

1. Spurious Rewards, baseline.

We agree, and the revision attaches the qualification where you asked for it. §4.1 now reports that the same untrained Qwen2.5-Math-7B scores 49.4% to 68.8% across five prompts, and that the 21.4 points are measured from the lowest; §7 and the caption of Figure 2(a) now specify the default prompt. We also agree with your two distinctions: the prompt difference does not decompose the RL gain, and 70.8 − 68.8 is not a causal estimate.

What the primer takes from this study is a contrast reported inside it: the same reward type helps Qwen and hurts Llama and OLMo. The baseline question bears on the size of the Qwen gain, which §4.1 now reports with its protocol.

2. RL with One Training Example, starting score.

We agree, and §6.1 now gives the protocol-dependent numbers. For DeepSeek-R1-Distill-Qwen-1.5B in that study, the gain is 3.8 points at an 8K-token evaluation limit and 1.0 at 32K, taken from the current arXiv version (76.7 → 80.5 and 82.9 → 83.9). The 32K comparison uses the source's own untrained model under its own protocol, so it is matched in a way the model card's 83.9 is not.

The primer's primary example is a different model. §6.1, Figure 5 and Appendix B.6 rest on Qwen2.5-Math-1.5B, whose base scores 36.0 on MATH500 under the source's protocol.

3. s1, paired intervals.

You are right; Appendix B.1 is corrected. The submitted version said that the "Wait", 59K → 1K and selected → random gaps cannot be separated from question-sampling noise without per-problem results. Muennighoff et al. report paired bootstrap intervals for two of the three: −26.7 to −3.3 points for random 1K minus s1K, excluding zero, and −13.3 to 20.0 for 59K minus s1K. The revised B.1 now gives both intervals and restricts the original statement to the "Wait" gap.

4. Figure 5, normalisation.

Agreed; the caption and Appendix B.6 now state it. The plotted quantity is the gain over the base model as a share of the full-data gain (7.5K MATH prompts for RLVR, 17K for distillation), with sampled responses fixed within each study. The RLVR curve uses MATH500 at each run's best MATH500 checkpoint, giving the 96.4% you reconstruct (36.0 → 74.0 against 75.4), and the distillation curve the six-benchmark average. Appendix B.6 gives the plotted values, notes where its other figures for that study use a different checkpoint convention, and records the conditions on each plotted point.

Missing citation and historical defects.

The missing citation is now in: Chen et al.'s Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning is cited in §5.2 and in Table A.2.

On the three historical defects, we agree with your reading of the citation roles. FinQA and UltraFeedback appear only in the background inventory of Table A.1. From Hu et al. we use the nine-teacher observation of their Figure 1 (§5.2, §7, Table B.6).

Writing.

Thank you for these suggestions. §2 and §3 now open with a short overview, as §4 to §6 already did, and §4 to §6 now close with a Takeaway box, as §3 did. We did not add a table of contents, since the main text runs to the page limit, but contribution (i) now says which sections answer the four questions.

Thank you again for a careful and supportive review; your checks have made the paper's numbers more precise. We are glad to clarify anything further.

I'm glad that you were able to make positive changes based off my review. Typically, it's appropriate to increase a score once points in a review have been addressed - indeed, the original rating was given on the version without the corrections - but since I was already at maximum, I'll have to leave it at that!

AuthorsOpenReview ↗

Thank you for your support

Thank you for your kind follow-up and for your support of the paper. Your careful checks of the numbers and protocols made the revision substantially more precise: following them, our team carefully rechecked every reported number, and the conditions it holds under, throughout the paper. We truly appreciate the time and care you put into your review.

We are also very grateful for your recommendation, and we believe that with it, this timely and practical primer will reach more of the readers who need it.

Reviewer_gHBEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This is a primer on post-training reasoning data. The authors collected 156 works and organise the material around the four questions: what forms reasoning data take and what a record contains, what makes them useful, how they are built, and what scaling evidence establishes. The unit of analysis is one record: a question, a reasoning trace and a checkable endpoint, plus optional feedback and provenance. The authors call the combination of learner, target and update rule a record's "learning interface", and a recipe a "route".

Critical strengths
  1. Using learner and route as shared variables turns scattered evidence into instances of one proposition. Placing 4.1's "random rewards lift Qwen but hurt Llama and OLMo" next to 4.3's "problems that RL discards teach most in on-policy distillation" in one frame says more than another list of datasets, and it yields a practical recommendation: report the learner, the target, the update rule and the route,.
  2. The analyses in Appendices B.4 and B.7 can be reused on their own. B.4 turns "random versus systematic error" from an intuition into a written expected update with a testable prediction: random errors scale by 1 - α - β, systematic ones change direction. B.7 shows that finding and selecting count different events, which explains a set of seemingly contradictory results on whether RL adds capability.
Critical weaknesses
  1. There is no new method, data or experiment, and no quantitative pooling of the cited results; the contribution is classification, explanation and advice.
  2. The central claim is not tested. Of the three comparisons in Section 7, the teacher by student crossing is affordable on the paper's own numbers: s1 trains on 1K questions for 7 H100-hours, so three teachers by three students is about nine runs, tens of GPU-hours, and Section 5.1 uses that same figure to argue for small datasets. The second comparison, injecting random and systematic errors at matched rates, already has its prediction written in B.4; one run on a small public model would show the direction. Without either, the sentence "correctness alone does not determine teaching value" remains an assumption with strong anecdotal support and no test by the authors.
Questions for authors
  1. Where is the boundary with contemporaneous surveys? The main text mentions Chen et al. (2025a) in passing and does not distinguish itself from other 2025 surveys on RLVR or reasoning data. Is the increment the frame, the standard of evidence, or the reporting conventions?
  2. Could you run a teacher by student pilot? On the s1 numbers cited in 5.1, three teachers by three students is about nine runs and tens of GPU-hours. If the ranking holds in most crossed cells, would you conclude that usefulness belongs to the data? Can the reversal condition in Section 7 be quantified, for example as a number of cells that must agree?
  3. The "up to 31%" in 5.5 conflicts with A.2's rule against figures. B.1 already switches to the HealthBench-1k ablation. Why does the main text keep the original number, and which other numbers in the main text come from figures rather than text or tables?
2 comments
AuthorsOpenReview ↗

Response to Reviewer gHBE

Thank you for the close reading, and for singling out Appendices B.4 and B.7 as reusable on their own. We ran both experiments you proposed.

W2. Is the central claim tested?

Yes. We have added direct experimental evidence for the central claim. Following your suggestion, we crossed three teachers with ten students rather than three, two seeds per cell, and observed ranking reversals in all three model families under a fixed training recipe. We also ran the checker-error comparison, and it went in the direction Appendix B.4 predicts: random false accepts slowed learning without reversing it, while the systematic checker, once exploited, reversed it. As Appendix E.4 states, these runs cannot separate the kind of error from the much higher rate it reached. Q2 of our main response summarizes both experiments; Appendix E gives the designs, controls and full results.

Q2. Does the ranking hold, and can the reversal condition be quantified?

The teacher ranking is not stable across students, so we do not conclude that usefulness belongs to the data alone. As Q2 of our main response shows, it reverses between the smallest and the largest student of every family. Ordered by student size within a family, both differences against Gemini change sign once: R1 overtakes Gemini between 1.5B and 7B in Qwen2.5, 1B and 8B in Llama 3, and 1B and 12B in Gemma 3. As a count of cells, as you suggest: across the ten students, R1 versus Gemini is significantly positive for 5 and negative for 5 (3 and 4 after Holm correction), and QwQ versus Gemini for 3 and 6 under both; a ranking that belonged to the data would keep one sign in all of them.

W1. No new experiment, and no quantitative pooling.

The revision now adds two controlled experiments (W2 above). We do not pool the cited results because they do not estimate a common quantity. They differ in learner, route, benchmark, budget and evaluation protocol, and their effects change with these factors, at times in sign (§4.1; Table E.1). Pooling would average these factors away. The paper aims at the opposite: to keep them in view, within the framework described below.

Appendix B.1 documents what was held fixed and what changed in selected comparisons; Appendices B.4 and B.7 state what a kind of evidence can and cannot show.

Q1. Where is the boundary with contemporaneous surveys, and what is the increment?

Primarily the frame: we read each finding together with the learner it was measured on and the route by which its data reached that learner. The standard of evidence and the reporting conventions follow from it: the frame requires recording what each comparison held fixed (Appendix B.1), and the reporting recommendations of Table 1 ask the same of future work.

Where the boundary lies. Recent surveys organize the field by what is done: Chen et al. (2025a) by the capabilities of long chains of thought, Luo et al. by data operation, from selection to synthesis, and Zhang et al. (2025b) and Yu et al. by RL technique. They map which methods exist. The primer complements them with a question they leave open, one a practitioner meets whenever two results disagree: which of them holds for my model and my training method? We read each finding together with the learner it was measured on, the route by which its data reached that learner (SFT, RL or on-policy distillation) and what its comparison held fixed. Read this way, six of the seven conflicts we collect differ mainly in learner or route (Appendix B.2). Your first strength puts the boundary well: placing §4.1's random rewards, which lift Qwen but hurt Llama and OLMo, next to §4.3's problems that RL discards yet that often teach most under on-policy distillation "says more than another list of datasets". The new experiments test this reading directly: in every family, the best teacher for the smallest student is the worst for the largest. §2 now states this boundary, and Table A.2 sets the surveys side by side by the question each starts from.

Q3. Why keep the "up to 31%", and which other numbers come from figures?

The 31% is stated in the source's text, not read from a figure. Gunjal et al. report "relative improvements of up to 31% on HealthBench" in their abstract and again in §5. Only the underlying full-HealthBench scores are confined to a figure. The HealthBench-1k row in Appendix B.1 is a separate ablation; B.1 now says so and names the abstract as the source of the 31%.

No number in the main text's prose comes from a figure. We rechecked every numeric result the main text takes from other work: each appears in a source's prose or tables, or is our own arithmetic on such values. Appendix A.2 now reports this.

Your review directly shaped these experiments, and we thank you for it. We would be glad to answer any further questions.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer gHBE,

Thank you again for your detailed and conscientious review. Your suggestions directly strengthened the chain of evidence behind our experiments and sharpened the paper's boundaries. Our team values your guidance greatly: we worked carefully to take each of your comments on board, and they have taken the paper a clear step further. We are grateful for this contribution.

In the revision we have answered each of your questions in the terms you proposed: the two comparisons you outlined are now run and reported (Section 7, Figure 6, Appendix E), with the reversal condition stated as a count of cells, and the boundary with contemporaneous surveys and the provenance of the numbers we report are now made explicit in the paper.

As the discussion period closes on November 18 (AoE), we are reaching out early in the hope of a fuller exchange with you. We would be glad to know whether our response resolves your concerns, and happy to provide any further clarification.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Response to All Reviewers

We thank all reviewers for their careful reading and constructive suggestions. We are glad that reviewers value an organized synthesis of scattered evidence (DDpx, sLr7), the learner and the route as shared variables, with the conditions each comparison records (gHBE, sLr7), the analyses of Appendices B.4 and B.7 as reusable on their own (gHBE), and citations that state what each source shows (DDpx).

We answer here the two questions raised by more than one reviewer and list the main additions. Added and reworded passages are printed in blue in the revised PDF.

  • Two controlled experiments (§7, Figure 6, Appendix E): three teachers crossed with ten students from three model families, and random against systematic checker errors under GRPO (Q2 below).
  • Positioning (§2, Table A.2): the scope paragraph now names recent surveys, and a new table sets them side by side by the question each starts from (Q1 below).
  • Corrections and conditions (§4.1, §4.2, §6.1, §7, Figures 2 and 5, Appendices A.2, B.1, B.6): reported numbers now carry the prompt, model or protocol they hold under, and Appendix B.1 adds the paired intervals that s1 publishes.
  • Presentation (contribution (i), §2–§6, Figures 1–4, Table B.5): contribution (i) maps the four questions to Sections 3–6, §2 and §3 gain short openings, §4–§6 close with a Takeaway box, Figures 1–4 are redrawn, and Table B.5 separates errors in the checker's verdicts from reward aggregation.

Q1. How does the primer differ from contemporaneous surveys? (gHBE, sLr7)

Our main contribution is a framework for interpreting evidence on reasoning data through the learner and learning route. It connects apparently conflicting findings, clarifies what each comparison establishes, and yields seven practical lessons with reporting recommendations (Table 1). For example, problems discarded by RL for yielding no reward signal can still teach through on-policy distillation (§4.3). Section 2 and the new Table A.2 now make this positioning explicit alongside recent surveys, including Luo et al. and Yu et al.

Q2. Is the central claim tested? (gHBE, sLr7)

Yes. We have now run two of the comparisons proposed in Section 7. The teacher–student crossing, including controls that hold final answers fixed, directly tests learner dependence; the GRPO experiment examines the effects of random and systematic checker errors (Figure 6; Appendix E).

The ranking of teachers reverses with the student. We crossed three teachers, Gemini and DeepSeek-R1 from s1K-1.1 and QwQ-32B, the teacher OpenThoughts prefers, with ten students from 0.5B to 12B in three families; every cell uses the same 985 questions, the same recipe and two seeds. The smallest and the largest student of each family (MATH500 accuracy, %):

StudentGeminiR1QwQ
Qwen2.5-0.5B14.79.12.9
Qwen2.5-7B76.183.282.9
Llama-3.2-1B21.216.39.5
Llama-3.1-8B53.365.268.1
Gemma-3-1B24.010.37.8
Gemma-3-12B83.286.487.8

In each family the ranking reverses between the smallest and the largest student, and in Llama it inverts exactly: Gemini is the best teacher for the smallest student of every family and the worst for the largest. The ranking also moves among the students that fine-tuning improves, with QwQ 3.0 points behind R1 on Llama-3.2-3B and 3.0 ahead on Llama-3.1-8B. Six of the ten students fall below their untrained baseline under every teacher, so for those the ranking reflects relative harm rather than improvement; Appendix E reports this.

The same final answers yield different teaching outcomes. With final answers held fixed, removing the reasoning hurt the largest student in every family and helped the smallest in two (averaged over Gemini and R1), so correctness alone did not decide the outcome. The smallest students' truncated outputs often end in repetition loops, and for Llama-3.2-1B the cost of full traces persisted when the evaluation cap doubled to 64K; this points to failing to finish long reasoning, without isolating trace length as the cause (Appendix E.3).

An initially calibrated checker can be exploited during training. Random false accepts at 20% and 40% cost at most 2.7 points against a clean checker. A systematic checker based on answer length, calibrated to 20% false accepts on the initial policy, was exploited until it accepted nearly every wrong answer; accuracy fell 34.7 points below its starting level. This experiment shows that initial calibration alone does not ensure robustness during training: systematic checker errors can be amplified as the policy adapts. It does not isolate the effect of error structure at matched realized false-accept rates (Appendix E.4).

Appendix E gives the design and the limitations, and Table E.1 the thirty differences with intervals.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.