acceptodds
Under review as a conference paper at ICLR 2027

Computation Modules for Generation: A Design Space for Label-Free Self-Distillation

Abstract

Self-distillation offers a promising route to language model self-improvement, but its effectiveness depends on the supervision a model can provide for itself. While ground-truth labels are not accessible, existing approaches often construct this supervision through heuristic generation procedures, leaving open how additional computation should be organized to make the model a more informative teacher. We introduce *Compound Computation for Amplification*, a framework for organizing self-improvement into generation and learning. At its core is a *computation module for generation*, which organizes model calls and programmatic operations as a directed acyclic graph to construct supervision from unlabeled input problems. By connecting the computation used for generation, the content it produces, and subsequent learning, the framework enables generation quality and downstream training outcomes to be studied separately. Within this design space, we compare different strategies for scaling computation and find that higher answer accuracy does not consistently lead to better self-distillation. Our observations suggest that changes in response coverage and reasoning patterns may also contribute to differences in supervision quality. Motivated by this finding, we develop *Computation-Augmented Self-Distillation (CASD)*, a concrete label-free on-policy self-distillation method within our framework, whose computation module for generation is designed to construct more effective supervision. Experiments on mathematical reasoning show that *CASD* achieves performance comparable to or better than baselines trained with ground-truth labels and outperforms all evaluated label-free baselines. These results demonstrate the potential of organizing generation-time computation to construct more effective supervision for self-improvement.

Primary area:
foundation or frontier models, including LLMs
Keywords:
On Policy Self Distillation, Post Training

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ZY2YOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies a simple approach to label-free self-distillation. Specifically, the authors first generate 16 independent reasoning trajectories, apply Recursive Self-Aggregation (RSA), and use majority voting to select a pseudo-answer. The aggregated result determines the pseudo-answer, while a matching trajectory from the initial generation is preferred as privileged context for on-policy self-distillation. The authors evaluate CASD on Qwen3-4B and Qwen3-8B across seven mathematical reasoning benchmarks. Experimental results show that CASD outperforms the evaluated label-free baselines and achieves better performance than methods using ground-truth supervision on most benchmarks.

Critical strengths
  1. The paper addresses an important research problem with a simple and straightforward approach, achieving relatively consistent improvements across multiple benchmarks.

  2. The idea of using the final aggregated results to select a reliable pseudo-answer while preferring an initial reasoning trajectory as privileged context for student training is interesting and insightful.

Critical weaknesses
  1. RSA and OPSD are both established methods, and the main novelty of CASD appears to lie in separating pseudo-answer selection from privileged-context selection. Although the paper cites CANON [1] and other consensus-based self-distillation methods in Section 2.2, the label-free baselines in Table 3 only include TTRL and Intuitor. A direct comparison with CANON under matched generation budgets would help clarify the methodological contribution.

  2. The paper observes that aggregation reduces success coverage and shortens reasoning trajectories, but the relationship between these changes and downstream student performance remains unclear. More controlled experiments or theoretical analysis would help explain why selecting earlier reasoning trajectories improves self-distillation and identify the actual sources of the performance gains.

  3. CASD generates up to 64 trajectories per problem, with later generations conditioned on previous responses, introducing substantial computational overhead. It is unclear whether simpler approaches, such as increasing parallel sampling or using standard consensus-based self-distillation, could achieve comparable improvements under the same computational budget. Furthermore, experiments are limited to two Qwen models, and evaluations on additional model families would strengthen the evidence for generalizability.

[1] Gkountouras, John, Josip Jukić, and Ivan Titov. "Consensus as Privileged Context for Label-Free Self-Distillation." arXiv preprint arXiv:2607.13643 (2026).

Questions for authors
  1. Does CASD outperform CANON under the same generation compute budget?

  2. Could the authors provide more controlled experiments or theoretical analysis to better explain the sources of the performance improvements?

1 comment
AuthorsOpenReview ↗

Response to Reviewer ZY2Y

We appreciate the reviewer’s suggestions and discuss each point below.

W1: Lack direct comparison with CANON.

Thanks for pointing out this! Please check Comparison to other label free baselines in our global rebuttal.

W2: The relationship between rollout changes and downstream student performance remains unclear. Could the authors provide more controlled experiments or theoretical analysis to better explain the sources of the performance improvements?

Thanks for raising this concern. We add token level controlled experiments For the student-generated responses, we check the output token length, the markers per response, and the markers per 10k tokens of the proposed method and several baselines.

Here are the results:

MethodMean output tokensMarkers / responseMarkers / 10k tokens
Base2,9592.137.21
SFT31,966236.9374.12
GRPO11,71330.1325.72
OPSD21,170103.6848.98
Initial-consensus self-distillation6,30114.7323.38
Terminal-consensus self-distillation8,52819.1322.44
CASD9,85929.5930.01

Markers per response measures how often explicit reflection appears in an answer; markers per 10k tokens measures its density, accounting for differences in response length.

The SFT and OPSD models produce reasoning traces that are substantially longer than necessary and use reflection markers far more frequently than needed, exhibiting signs of overthinking.

Appendix D further provides matched-prefix teacher case studies, illustrating how initial references can increase support for diverse reflective expressions and guide the teacher back to checking intermediate errors.

W3.1 && Q1: Does CASD outperform CANON and increase parallel sampling under the same generation compute budget?

Thank you for highlighting this point. The generation compute budget is an important consideration, and we agree that incorporating this context strengthens our evaluation. We have added comparisons with additional baselines under matched sampling budgets in the Comparison to other label-free baselines section of our global rebuttal. Specifically, we evaluate two label-free approaches: CANON using an equivalent sampling budget and an expanded parallel sampling baseline. CASD consistently outperforms both methods.

W3.2: Generalizability of our method. To further assess the generalizability of our approach across generation modules, we evaluated CASD after replacing RSA with PDR. The results demonstrate that:

MethodAIME25HMMT25AIME26HMMT26avg
RSA-CASD (paper method)57.78%42.50%67.50%42.42%52.55%
PDR-CASD60.00%39.72%66.39%40.15%51.57%

The results indicate that the CASD approach remains robust even when the generation module is swapped. While PDR-CASD shows a slight decrease in overall average performance (51.57%) compared to the original RSA-CASD (52.55%), it remains highly competitive, demonstrating that the CASD framework is effectively generalizable across different generation methods.

Reviewer_JkLJOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to use additional generation-time computation to construct better supervision for self-distillation. It proposes CASD, which uses iterative aggregation to select a pseudo-answer, then preferentially retrieves a matching solution from the initial candidate pool as the teacher’s reference during on-policy self-distillation. Experiments on mathematical reasoning compare generation procedures, aggregation depths, and reference-selection strategies. The results show that higher reference accuracy does not always produce better students, and CASD outperforms OPSD and the evaluated label-free baselines.

Critical strengths
  • CASD constructs training references without ground-truth answers or external verification.

  • The comparison with terminal consensus holds pseudo-answers fixed, providing clear evidence for the benefit of reference selection (table 4).

  • Consistent empirical gains. CASD improves over OPSD across the tested models and benchmarks and compares favorably with other baselines.

Critical weaknesses
  • The paper’s central motivating observations overlap substantially with prior work showing that privileged context can change reasoning style, suppress uncertainty or self-correction, and impair downstream reasoning despite correct supervision [1, 2, 3]. The RSA-depth experiments extend these observations to a particular generation pipeline, but do not clearly establish what additional understanding they provide. Thus, the main methodological contribution is the selection rule that uses terminal consensus to choose a matching initial-pool reference.

  • Table 4 provides a useful controlled comparison. However, the connection between this benefit and the motivating observations about response length and reflection remains untested. Figure 3 measures generated candidate populations; the paper does not show whether initial references lead to corresponding preservation of these behaviors in the trained student or changes in the teacher’s supervision. Furthermore, the downstream reference-selection ablation is limited to RSA, while the comparison with other generation modules considers only generation accuracy. It is not unclear whether CASD captures a transferable principle about useful reference properties or a benefit specific to RSA and its interaction with the generating model. These gaps limit the guidance the study provides about when and why the selection strategy should help.

[1] Kim et al. Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs? arXiv:2603.24472.

[2] Kaur et al. Rethinking On-Policy Self-Distillation for Thinking Models. arXiv:2607.05184.

[3] Gkountouras et al. Consensus as Privileged Context for Label-Free Self-Distillation. arXiv:2607.13643.

Questions for authors
  • Could the authors report mean response lengths, reflection-marker counts, and length-normalized marker frequencies for the base model, SFT, GRPO, OPSD, initial consensus, terminal consensus, and CASD under identical evaluation settings? Does CASD preserve more explicit reflection than terminal consensus, as would be expected if its gains stem from retaining these behaviors? A comparison of teacher predictions on identical student prefixes under initial and terminal references may provide further evidence

  • The authors may add an oracle answer-selection ablation using the ground-truth checker (eg the one already employed for GRPO). Later RSA stages determine the pseudo-answer and provide fallback references. A useful control would retain all training questions and existing fallback references, while replacing an incorrect initial-pool reference with a uniformly sampled correct initial candidate whenever one is available. This would assess the benefit of improving answer selection without changing question coverage. If reference correctness increases but downstream student performance declines, it would suggest that properties of the selected references beyond final-answer correctness contribute to their usefulness.

  • Does the benefit of selecting an initial matching reference persist with another generation module, such as PDR or MatryoshkaThinking? Could the authors also examine whether differences among the modules reflect the generating model’s ability to aggregate or verify candidate solutions? eg, could difficulties with workspace construction or recursive verification in qwen3 4b partly explain the weaker results of PDR or MatryoshkaThinking? This would help distinguish the merits of the algorithm from how effectively the model executes its component operations.

3 comments
AuthorsOpenReview ↗

Response to Reviewer JkLJ (Part 1/3)

We thank the reviewer for the careful review and respond to the points raised below.

W1: Comparison to previous work.

We acknowledge that the decrease of uncertainty and the change of reasoning pattern is well studied by [1]. However, it focuses more on what the reasoning pattern changes after training. Instead, we focus on a complementary question: what will the reasoning pattern of privileged information change the training result?

Regarding [2], we agree that privileged context can alter deliberative behavior and the token-level learning signal. Our study addresses a complementary construction problem: when privileged references must themselves be generated without labels, how should additional generation-time computation be used? Holding the downstream training procedure fixed, our depth study shows that selected-reference accuracy continues to improve while student performance peaks and then declines. This motivates evaluating generation procedures by their downstream teaching value, rather than answer accuracy alone.

Regarding [3], CANON forms a consensus and selects its reference from the same independently sampled population. Our distinction concerns the source of the reference across generation stages: CASD uses terminal consensus to identify a pseudo-answer, then preferentially retrieves an answer-matching trajectory from the initial population. Thus, additional aggregation informs answer selection without requiring its rewritten trajectory to become the teacher’s reference. This cross-stage construction is the specific design question investigated by our generation-depth study.

[1] Kim et al. Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs? arXiv:2603.24472.

[2] Kaur et al. Rethinking On-Policy Self-Distillation for Thinking Models. arXiv:2607.05184.

[3] Gkountouras et al. Consensus as Privileged Context for Label-Free Self-Distillation. arXiv:2607.13643.

W2 && Q1.1: The connection between this benefit and the motivating observations about response length and reflection. && Could the authors report mean response lengths, reflection-marker counts, and length-normalized marker frequencies for Different methods.

Yes. We compare Qwen3-4B students on AIME 2025 and HMMT 2025 under identical evaluation settings. CASD preserves more explicit reflection than terminal-consensus self-distillation, both per response and after normalization by output length.

MethodMean output tokensMarkers / responseMarkers / 10k tokens
Base2,9592.137.21
SFT31,966236.9374.12
GRPO11,71330.1325.72
OPSD21,170103.6848.98
Initial-consensus self-distillation6,30114.7323.38
Terminal-consensus self-distillation8,52819.1322.44
CASD9,85929.5930.01

Markers per response measures how often explicit reflection appears in an answer; markers per 10k tokens measures its density, accounting for differences in response length.

The SFT and OPSD models produce reasoning traces that are substantially longer than necessary and use reflection markers far more frequently than needed, exhibiting signs of overthinking.

Q1.2: A comparison of teacher predictions on identical student prefixes under initial and terminal references may provide further evidence

Thanks for the question! In the revised pdf, Appendix D further provides matched-prefix teacher case studies, illustrating how initial references can increase support for diverse reflective expressions and guide the teacher back to checking intermediate errors. We hope these experiments would provide further evidence on how the teacher's prediction of different reference answers would affect the supervision signal.

AuthorsOpenReview ↗

Response to Reviewer JkLJ (Part 2/3)

Q2: The authors may add an oracle answer-selection ablation using the ground-truth checker.

Thank you for the suggestion. In the oracle ablation, we replace the majority-vote pseudo-answer with the ground-truth answer for reference selection. We prioritize correct self-generated trajectories from the initial round and fall back to the terminal round when necessary. If neither round contains a correct trajectory, we retain the standard CASD selection. The results for the 8B model are shown below (Avg@12 accuracy, %).

BenchmarkCASD (8B)Oracle (8B)Δ
AIME 202557.7861.11+3.33
HMMT 202542.5043.06+0.56
AIME 202667.5068.06+0.56
HMMT 202642.4243.69+1.26
Macro Average52.5553.98+1.43

Oracle selection improves macro-average accuracy by 1.43 percentage points. CASD achieves comparable performance using only unlabeled problems, making it applicable to settings where ground-truth annotations are unavailable.

Our OPSD baseline (in Table 3) provides a complementary comparison: it directly uses dataset-provided ground-truth solutions as the teacher’s privileged context, rather than selecting self-generated trajectories.

Q3.1: Generalizability of our method beyond RSA

More experiments in PDR and MastryoshkaThinking.

Thanks. We try PDR on CASD as a base test time scaling method to see if CASD still works. Also, for MastryoshkaThinking we didn’t try because the majority voting of the final round’s accuracy is even worse than the first round. On PDR-CASD, we have

MethodAIME25HMMT25AIME26HMMT26avg
RSA-CASD (paper method)57.78%42.50%67.50%42.42%52.55%
PDR-CASD60.00%39.72%66.39%40.15%51.57%
The results indicate that the CASD approach remains robust even when the generation module is swapped. While PDR-CASD shows a slight decrease in overall average performance (51.57%) compared to the original RSA-CASD (52.55%), it remains highly competitive, demonstrating that the CASD framework is effectively generalizable across different generation methods.
AuthorsOpenReview ↗

Response to Reviewer JkLJ (Part 3/3)

Q3.2: Whether differences among the modules reflect the generating model’s ability to aggregate or verify candidate solutions?

We acknowledged that workspace construction and recursive verification might be at different difficulty levels. We analyzed existing rollouts in Section 4.2 on 60 problems: 30 from AIME 2025 and 30 from HMMT 2025. We preserved each method’s native workflow and traced its actual inputs to subsequent outputs: sampled candidates → aggregation for RSA; candidates → workspace → refinement for PDR; and the cumulative accepted pool → aggregation for MatryoshkaThinking. Each entry below reports conditional output accuracy (%), followed by the number of eligible problems in parentheses. We first averaged output correctness within each processing step, then across eligible steps within each problem, and finally equally across problems. Thus, the percentages are not simply the fraction of problems solved. Problem counts differ because each condition occurs on different subsets of the native trajectories. “Majority” means more than half of the input candidates; an incorrect majority requires one particular incorrect answer to exceed this threshold.

Input conditionRSAPDRMatryoshkaThinking
At least one correct candidate84.81% (25)73.56% (39)72.07% (37)
Correct candidates form a majority97.81% (23)97.29% (30)91.67% (29)
An incorrect answer forms a majority, but a correct candidate remains18.64% (7)26.04% (6)0.00% (5)
No correct candidate, including unparsed answers9.28% (40)0.40% (31)0.00% (23)
All candidates have parsed but incorrect answers3.84% (36)0.00% (30)0.00% (18)

Two patterns are relevant to the observed performance differences. First, available correct evidence is less consistently converted into correct outputs in PDR and MatryoshkaThinking: their conditional accuracies are 73.56% and 72.07%, versus 84.81% for RSA. Second, recovery without a correct input candidate is less frequently observed, including when all input answers are explicitly parsed and incorrect. These behaviors are plausible contributors to the performance gaps.

We are also curious about if another stronger model performs similar patterns. We evaluate DeepSeek-V4.1-Flash-1017 on the same 60 problems, with thinking disabled. Each method retains its prompts and sampling settings from Section 4.2.

The table reports generation accuracy (%) before self-distillation. RSA averages correctness across its terminal candidates. PDR and MatryoshkaThinking evaluate their final solution.

ModelMethodAIME25HMMT25Average
Qwen3-4BRSA66.0439.7952.92
Qwen3-4BPDR60.0036.6748.33
Qwen3-4BMatryoshkaThinking53.3336.6745.00
DeepSeekRSA85.6384.3885.00
DeepSeekPDR90.0090.0090.00
DeepSeekMatryoshkaThinking93.3390.0091.67

With DeepSeek, PDR and MatryoshkaThinking outperform RSA on both datasets. Thus, RSA’s advantage with Qwen3-4B does not persist under these configurations.

We also apply the same process metrics defined above. Each entry reports conditional output accuracy (%), with the number of eligible problems in parentheses.

Input conditionRSAPDRMatryoshkaThinking
At least one correct candidate87.37% (40)98.41% (55)98.33% (55)
Correct candidates form a majority98.64% (38)99.38% (54)99.59% (54)
An incorrect answer forms a majority, but a correct candidate remains44.44% (3)100.00% (2)45.83% (1)
No correct candidate, including unparsed answers8.72% (20)3.57% (7)0.00% (4)
All candidates have parsed but incorrect answers8.46% (20)3.57% (7)0.00% (4)

When a correct candidate is available, PDR’s conditional accuracy rises from 73.56% to 98.41%. MatryoshkaThinking improves from 72.07% to 98.33%. Its verifier also becomes more selective: the acceptance rate for candidates with incorrect answers falls from 52.49% to 19.49%.

These results are consistent with model capability contributing to the observed performance gaps. These experiments provide more evidence on the reviewer's hypothesis that Qwen3-4B might lack some ability which is necessary for PDR and MatryoshkaThinking (eg, workspace construction and self verification).

Reviewer_GHBZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper separates a DAG-based generation module from the learning procedure in label-free self-distillation. In the tested configurations, parallel sampling achieves greater success coverage than feedback-driven sequential revision, while deeper RSA stages increase average accuracy but reduce coverage and shorten trajectories. Selected-context accuracy rises with depth, yet student performance peaks at depth 2 and declines at depth 3. Motivated by this finding, CASD selects a pseudo-answer by majority vote over the terminal RSA population, but preferentially uses a matching response from the initial 16 samples as privileged context, falling back to the terminal population when necessary. A frozen, thinking-enabled teacher then supervises a non-thinking student on its own rollouts. Across Qwen3-4B/8B and seven reported mathematical benchmarks, CASD outperforms the evaluated label-free baselines and ground-truth-conditioned OPSD, while remaining competitive with other label-supervised methods. A 4B ablation holding generation records and terminal pseudo-answers fixed supports the context-selection design.

Critical strengths
  1. A controlled ablation supports separating answer selection from teaching-context selection. With generation records and terminal pseudo-answers held fixed, CASD raises Qwen3-4B AIME2026 Avg@12 from 29.17% to 53.06% and improves every reported benchmark over terminal consensus. This provides strong evidence that the choice of reference trajectory matters beyond pseudo-answer correctness.
  2. The method delivers substantial gains beyond the checkpoint-selection benchmark. On AIME2026, CASD reaches 67.50% at 8B and 53.06% at 4B, compared with the strongest evaluated label-supervised results of 51.11% and 47.50%. It also outperforms ground-truth-conditioned OPSD across all reported benchmarks at both sizes, supporting the value of its self-generated supervision.
Critical weaknesses
  1. The value of RSA is not established under a comparable generation budget. Over 90% of CASD’s contexts come from the initial population, so aggregation mainly informs pseudo-answer selection. Without a comparable-budget baseline using more independent samples and majority voting, it remains unclear whether RSA is necessary.
  2. The overall gain is not fully separated from thinking-teacher distillation. The teacher uses thinking mode while the student does not, but no same-teacher control without reference context is reported. Table 4 convincingly shows that context selection matters, yet does not isolate how much of the improvement over the base model comes from teacher-mode transfer.
  3. Baseline reporting limits the strength of the comparative claims. TTRL and Intuitor lack detailed configurations, while several label-supervised baselines underperform the base model on some benchmarks. Reference-solution characteristics, method-specific tuning, and total compute are insufficiently documented, making broader superiority claims difficult to assess.
Questions for authors
  1. Under a comparable generation budget, how does RSA compare with additional independent sampling and majority voting? Does it improve pseudo-answer quality, downstream student performance, or both?
  2. How does self-distillation perform with the same thinking teacher but no reference context? How much additional benefit does CASD’s context construction provide over this setting?
1 comment
AuthorsOpenReview ↗

Response to Reviewer GHBZ

We appreciate the reviewer’s thoughtful comments and provide our responses below.

W1 && Q1: The value of RSA is not established under a comparable generation budget. && Under a comparable generation budget, how does RSA compare with additional independent sampling and majority voting? Does it improve pseudo-answer quality, downstream student performance, or both?

Thanks for the suggestion! We adapt the sampling budget as the RSA adapted in CASD and used, which is that for each round of RSA we use 16 rollouts, and totally we have 64 rollouts. So we do 64 independent samplings in the baseline experiments.

Comparison in pseudo-answer quality. Increasing the parallel sampling budget from 16 to 64 rollouts yields only a modest improvement in selected-answer accuracy, from 48.44% to 49.63%. Both remain below the 53.42% achieved by RSA’s final-round selection.

Comparison in downstream student performance. Please see Comparison to other label free baselines in our global rebuttal.

W2 && Q2: The overall gain is not fully separated from thinking-teacher distillation.&& How does self-distillation perform with the same thinking teacher but no reference context? How much additional benefit does CASD’s context construction provide over this setting?

Thanks for raising this problem! We response in global rebuttal Self-Distilation without privileged information.

W3: TTRL and Intuitor lack detailed configurations, while several label-supervised baselines underperform the base model on some benchmarks.

Detailed config of TTRL and Intuitor. Thanks for pointing this out. This was indeed an oversight and we have now added the missing experimental details to Appendix B.2 in the revised pdf. For TTRL and Intuitor, we follow the core hyperparameters reported in their papers, adapted to Qwen3 model family, our OpenR1 training data, and 200 training iterations.

label-supervised baselines underperform base models. While improvements across all benchmarks are not guaranteed, the observed underperformance is confined to few benchmarks and should not be interpreted as a general trend. For SFT, we sample solutions using Deepseek V4 flash, which is a stronger external model. We think underperforming in some benchmarks is due to the mismatch of thinking mode. Appendix C of revised pdf provides this explanation through qualitative comparisons between V4 Flash targets and Qwen3 responses.

RFT fine-tunes on the model’s own correct samples, so it only reinforces solutions the model already produces, and the ~28% of problems the model never solves are excluded from training. These are exactly the problems that matter for AIME/HMMT/Apex. We also provide a loss curve for SFT and RFT in Appendix B.3, in which SFT exhibits a steadily decreasing loss, whereas RFT consistently maintains a relatively low loss. Both patterns are consistent with observations from typical hyperparameter-tuning practices.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Global Rebuttal

We thank reviewers GHBZ, JkLJ, and ZY2Y for their thoughtful feedback and constructive suggestions. We appreciate GHBZ’s recognition of the controlled evidence supporting the separation of answer selection from teaching-context selection, JkLJ’s acknowledgment of CASD’s label-free reference construction and consistent empirical gains, and ZY2Y’s appreciation of the method’s simplicity and the insight behind preferring initial reasoning trajectories as privileged context. Below, we address the shared concerns and present additional baseline comparisons and controlled ablations.

1. Self-Distilation without privileged information

We explore two approaches to self-distillation without the use of privileged information. We adapt the OPSD algorithm on Qwen3-4B, enabling the teacher's thinking mode. We remove privileged information in the following two ways:

  1. We remove the solution but retain the privileged prompt (i.e., the indicator stating that a reference solution follows: "Here is a reference solution to this problem").
  2. We remove both the solution and the privileged prompt. In this method, the model is provided only with the problem itself, which is identical to that seen by the student.

The result shows below, they all increase the performance over the base model or even comparable to OPSD but not as strong as CASD.

ConditionAIME25HMMT25AIME26HMMT26Avg
original-empty36.67%22.22%40.56%26.52%31.49%
clean-none30.28%17.78%28.33%16.92%23.33%
OPSD35.00%21.67%37.78%25.76%30.05%
CASD54.17%33.06%53.06%34.60%43.72%

2. Comparison to other label free baselines

We additionally compare CASD against CANON [1], evaluating both its default non-thinking teacher and a thinking-enabled teacher variant while keeping student thinking disabled. To isolate the effect of RSA, we also include an ablation baseline, SD i.i.d., which replaces RSA with independent sampling for reference construction. On the same 10,000-question pool, both CANON and SD i.i.d. draw 64 i.i.d. candidate responses per problem, matching CASD’s maximum reference-construction budget (16 candidates across four populations). We retain CANON’s reported core configuration in [1]: LoRA rank 64 with α=128\alpha=128, AdamW with a constant learning rate of 10−510^{-5} and no warmup, a 4,608-token response cap, and full-vocabulary JSD. We adapt its default 32-sample setting to 64 candidates and report the 200-step variants with teacher thinking either disabled or enabled, while keeping student thinking disabled. Under this setting, CASD substantially outperforms both CANON variants and SD i.i.d. across all benchmarks (58.26 vs. 30.08, 29.11, and 52.86 on average).

MethodAIME25HMMT25AIME26HMMT26AMC23MATHApexAvg
CANON, teacher off19.447.2215.0014.6564.5885.584.0830.08
CANON, teacher on19.729.4410.5612.1264.7983.753.3729.11
SD i.i.d.50.2830.2858.3335.3592.9294.508.3352.86
CASD57.7842.5067.5042.4293.7594.839.0458.26

[1] Gkountouras et al. Consensus as Privileged Context for Label-Free Self-Distillation. arXiv:2607.13643.

AuthorsOpenReview ↗

Summary of Discussion and Revisions

We sincerely thank all reviewers for their constructive feedback and the Area Chair for overseeing the review process. Across the reviews, all three reviewers recognized the empirical gains of CASD and the insight of decoupling pseudo-answer selection from privileged-context selection. Following our rebuttal and revised manuscript, Reviewer JkLJ confirmed that most concerns were resolved and maintained a positive score.

To assist the AC-Reviewer discussion, we summarize below how our new experiments and revisions directly address every major concern raised by Reviewers GHBZ, JkLJ, and ZY2Y:

  1. Value of RSA & Direct Comparison to CANON under Matched Budgets (GHBZ W1/Q1; ZY2Y W1, W3/Q1)
  • Matching CASD’s maximum generation budget (64 rollouts/problem), we added two label-free baselines: SD i.i.d. (64 independent samples + majority vote) and CANON (64 samples, with teacher thinking both off and on).
  • Pseudo-answer accuracy: Expanding i.i.d. sampling from 16 to 64 rollouts only improves accuracy from 48.44% to 49.63%, whereas RSA reaches 53.42%.
  • Downstream student performance (Qwen3-8B, 7-benchmark avg): CASD (58.26%) substantially outperforms SD i.i.d. (52.86%), CANON teacher-off (30.08%), and CANON teacher-on (29.11%), demonstrating that RSA’s cross-stage decoupling provides clear gains over matched-budget i.i.d. sampling and consensus-based baselines.
  1. Isolating Privileged-Context Gains from Thinking-Teacher Distillation (GHBZ W2/Q2)
  • We added two no-reference controls using the exact same thinking-enabled teacher on Qwen3-4B (4-benchmark avg across AIME25/26 and HMMT25/26): removing both the reference and prompt (clean-none: 23.33%) and removing only the reference (original-empty: 31.49%).
  • CASD achieves 43.72% (+12.23 to +20.39 points over no-reference controls, and +13.67 points over ground-truth OPSD at 30.05%), confirming that the gains stem primarily from CASD’s context construction rather than teacher-mode transfer alone.
  1. Mechanistic Evidence & Generalizability Beyond RSA (JkLJ W2/Q1–Q3; ZY2Y W2–W3)
  • Why initial trajectories help: Under identical evaluation settings (Qwen3-4B), CASD students preserve higher explicit reflection density (30.01 markers/10k tokens, 29.59/response) than terminal-consensus distillation (22.44/10k, 19.13/response), without the overthinking bloat of SFT/OPSD. Matched-prefix teacher probes (Appendix D) further show that initial references increase teacher probability on reflective error-checking phrases.

  • Generalizability & Oracle control: Swapping RSA for PDR (PDR-CASD) yields 51.57% avg vs. 52.55% for RSA-CASD, confirming robustness across generation modules. An oracle answer-selection ablation (53.98%) shows label-free CASD (52.55%) comes within 1.43 points of ground-truth answer selection. We also added a cross-model diagnostic (Qwen3-4B vs. DeepSeek-V4.1-Flash) showing how module performance interacts with base-model capabilities.

  1. Baseline Transparency & Manuscript Revisions (GHBZ W3)
  • In the revised PDF, we added full hyperparameter tables for TTRL and Intuitor (Appendix B.2), training loss curves for SFT and RFT (Appendix B.3), and qualitative analyses explaining SFT’s reasoning-pattern mismatch on specific benchmarks (Appendix C).

We hope this summary is helpful for the final assessment, and we remain happy to clarify any remaining details.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.