acceptodds
Under review as a conference paper at ICLR 2027

What Should a Self-Teacher See? Privileged Context Design for On-Policy Self-Distillation

23 views

Abstract

More privileged information does not always make a better teacher. We study this tension in on-policy self-distillation (OPSD), where a self-teacher scores the student's own rollouts under privileged context, conventionally a complete reference solution that bundles the final answer with one particular reasoning path. Holding the student view and training fixed within each scale, we compare that default against three abstractions compiled offline, a named strategy, a method-independent framing, and a problem category, and against an answer-only control that keeps the destination but removes the path. In the primary runs on competition mathematics, the best intermediate contexts improve the in-domain peak mean over the full solution by 1.4 points at 4B and 1.6 at 8B, while storing an order of magnitude fewer hint tokens. Comparisons across three seeds also show positive mean gains for the framing and category contexts at both scales. Answer-only conditioning remains competitive in the primary runs, within 0.2 points of the full solution at these scales. The preferred context varies with student scale and task. Initial teacher–student KL does not order downstream performance. What a self-teacher should see is therefore not everything it could, but the level of abstraction its student can still act on.

Primary area:
foundation or frontier models, including LLMs
Keywords:
On-policy self-distillation, privileged context, mathematical reasoning, LLM post-training

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_mK4cOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The authors study the problem in on-policy self-distillation, where the teacher model is a copy of a student model with the additional privileged information. The authors study how different forms of the information access given to the teacher affects the distillation performance on the student. The authors evacuate performance of the Qwen models of sizes 1.7B, 4B, and 8B parameters. They show that different models benefit differently based on levels of information, from coarser to finer information.

Critical strengths

The topic the authors address is important for understanding how much privileged information affects the performance of the on-policy distillation.

The authors provide clear information to reproduce the results.

The results are comprehensive for the evaluated models.

Critical weaknesses

The experiments are limited, which makes it difficult to get any general conclusion. The authors only study one model family of students using a single training dataset.

The authors study the mathematics domain in the paper, the domain which benefits from reasoning. However, the student model used does not have thinking mode enabled.

Questions for authors

Why using a larger model offline compiler to create abstract packages?

Have you checked if the teacher answer is correct (as ground truths are not available)? How was that evaluated?

Why would the student not use thinking template? What is the performance when thinking is enabled explicitly?

Have you tried training the teacher model on the reference answers, or strategies? Given this setup, the teacher input is already different from the student input, so the distribution mismatch might be caused by different reasons.

What is the performance when SFT the student on the provided dataset with reference answers (L1)?

I am curious what is the behavior of the stronger models. Have you tried 32B models?

What is the reason for the brevity (explicitly asking for sub 90 tokens in L2-L4?) The context seem oversimplistic.

Why the authors focused on LORA training and not full SFT? The learning rate for LORA seems to be small.

The experiments are only studied using a single dataset on a specific domain.

What is the reason of having the two compilers for the context modification? Which one was actually used in the main paper? It seems that there are not much performance differences across compilers.

Have you tried going beyond L1 and providing more context than the original answer to enrich the context?

In the OPSD dataset used for reference answer, what teacher has been used for the reference answer?

How meaningful are such abstractions to the model? Would a student not already know such abstractions to the problem ahead? While L2-L4 responses are more fitted towards the Qwen models, L1 is not fitted at all.

5 comments
AuthorsOpenReview ↗

Response to Reviewer mK4c (Part 1)

Thank you for the detailed list. Several of these questions we can answer directly from the submission, and where your reading of the setup differs from what we ran, that tells us the paper did not make the setup visible enough. We take the soundness rating seriously, so we start with the four points it appears to rest on.

1. The student does use thinking mode at evaluation (Q3).

This is stated in the submission in two places. Appendix A.1: the training rollout uses Qwen3's non-thinking chat template, teacher scoring uses the thinking template, and evaluation enables thinking mode. Section 3.3 gives the attribution in the main text: "Following Zhao et al., the teacher scores with Qwen3's thinking template while student rollouts use the non-thinking template, an interface difference common to L1--L5." Every Avg@12 number in the paper is therefore a thinking-mode score, sampled at temperature 1.0 with a 38,912-token budget.

The non-thinking rollout template is not a choice of ours and not an oversight. It is the ablated setting of the method we intervene on: Zhao et al. (Appendix B, Table 5) measure per-token KL across all four student/teacher thinking combinations at all three scales and select student-off with teacher-on because it maximizes the KL signal on mathematics tokens. Changing it would change the method rather than the teacher's context, which is the single variable this study manipulates.

2. Ground-truth answers are available, and the design makes solution quality a shared constant (Q2).

The training pool is OPSD's released snapshot of the OpenThoughts mathematics subset (siyanzhao/Openthoughts_math_30k_opsd), 29,434 problems with source labels olympiads (21,315), math (5,351), aops_forum (2,291) and amc_aime (477). Every problem carries a reference answer. Those are the answers OPSD's own GRPO baseline verifies against, the answers our L5 condition extracts verbatim, and the answers the routing control in Appendix E checks boxed outputs against.

Two clarifications that we think bear on soundness more than the provenance does. First, the teacher never produces an answer. It is frozen at the initial checkpoint and only assigns probabilities to tokens the student generated, so there is no teacher output whose correctness could be checked. Second, and more important: reference quality cannot confound this study. All five conditions are built from the same 29,434 problem--solution pairs, L1 is the trace itself and L2--L5 are derived from that same trace, and every comparison we report is paired on problem identity. A defect in the pool is shared by all five arms.

You are right about one thing here, and we will fix it: the paper says "verified reference solution" in its own voice. That property is inherited from the source pool and described by Zhao et al., not established by us, and we will attribute it accordingly.

3. There is no 90-token instruction (Q7).

The contracts in Appendix B give per-level typical lengths as calibration, 10--30 tokens for L4, 20--60 for L3, 30--90 for L2. The binding constraints are semantic, not metric: a ban on solution-action verbs plus a verb test for L4, a "via X"-removal test that deletes a prepositional clause and checks whether the sentence still classifies the problem, a method-name test, a verb-count test that flags an operation sequence in disguise, and a transferability self-check requiring the same L3 sentence to apply unchanged to a different problem in the same L4 category. Realized lengths come out well under the guidance (38.9, 25.2 and 17.7 tokens for L2, L3 and L4; Table 6).

On "the context seems oversimplistic": brevity here is the independent variable, not a limitation of the implementation. The finding is that a 17.7-token description of the problem trains a better 4B and 8B student than the 657.3-token derivation it was written from. A context that was not this simple would not test the question we are asking.

AuthorsOpenReview ↗

Response to Reviewer mK4c (Part 2)

4. The two compilers are a control, and the small difference between them is the result (Q1, Q10).

The primary compiler is Qwen3.5-397B-A17B throughout the main tables; Qwen3-8B is a control, reported separately in Table 12. We use the larger model for the primary condition so that the abstractions are as well written as we can make them, which makes the comparison against L1 conservative rather than favourable to us.

Your observation that there is not much difference across compilers is exactly what the control was built to test. Recompiled by Qwen3-8B under the same contracts, all six abstractions still beat L1, so the effect does not need a strong writer:

StudentL1L2L3L4
4B63.4364.0763.7164.35
8B66.6767.9667.0467.13

ID peak mean, with L2--L4 recompiled by Qwen3-8B. L1 is from the original dataset, needs no compiler and is shared. These are the L1 column and the Qwen3-8B block of Table 12.

At 8B that compiler is the student's own base model, so one model writes the abstractions, teaches from them, and learns from that teaching, with no external model at any stage. The result can then be applied without access to a frontier model, which is a stronger claim than the primary result alone.

5. Teacher-side training, and whether the discrepancy has another source (Q4).

Updating the teacher is a reasonable thing to try, by refreshing it every so many steps or by keeping an EMA of the student. SDFT (Shenfeld et al., ICML 2026, cited in our related work) does the latter, so the teacher's parameterization is plainly a live design axis rather than something self-distillation fixes. We keep ours frozen because that is OPSD's setting: Zhao et al. fix the teacher to the initial policy rather than the currently updating one, and report that this stabilizes training and acts as implicit regularization against drifting too far from the initial policy. Our L1 condition is OPSD, so changing the update rule would move the baseline we are intervening on.

It is also orthogonal to what we ask. SDFT's teacher is an EMA and is still conditioned on a demonstration; OPSD's is frozen and is still conditioned on a reference solution. However the teacher's weights are maintained, it has to be shown some context, and which context that should be is the variable we vary.

On the second half: the teacher's input differs from the student's by construction, and that is where the supervision comes from. Teacher and student are the same frozen network, so if the teacher read the student's prompt the two distributions would coincide and the loss would have no gradient. The asymmetry is the mechanism rather than a side effect of it, and it is how this family of methods is defined: SDFT writes the teacher as π(.∣x,c)\pi(.|x,c) and the student as πθ(.∣x)\pi_\theta(.|x), the same network under different conditioning, and notes that learning from a teacher with privileged information requires on-policy sampling in the first place.

What does need separating is which part of the asymmetry does the work: the interface, meaning the teacher's thinking-enabled template, or the privileged content that the bridge introduces. Each has its own control. The no-hint reference scores the same fixed base-model rollouts through the thinking-enabled teacher interface with no privileged content at all, which isolates the template. Separately, the shared-bridge runs of Table 20 hold the introductory wording fixed across L1--L5, which isolates the bridge. Section 4.3 makes the qualitative point; the submission plots the means in Figure 2(a) without tabulating them, so we give them here:

Teacher context1.7B4B8B
No hint, interface only0.11750.13150.1035
L4, problem category0.16180.17570.1557
L1, full solution0.22650.22400.1975

Mean teacher-to-student KL at initialization, answer-aligned, unclipped and over the full vocabulary. These are the quantities plotted in Figure 2(a).

The interface on its own accounts for a KL of 0.10 to 0.13, and the full solution adds a further 0.092 to 0.109 on top of that, so the privileged content is not a small perturbation on a format effect. The interface is also identical across L1--L5, and so is the student prompt, so whatever it contributes is constant across the conditions we compare. Every difference we report is between two contexts that differ only in what the hint says, and the whole L1 to L4 ordering sits inside that 0.092--0.109 span, taking up 44--59 percent of it.

Zhao et al. is the OPSD paper, ICML 2026; the table numbers we quote follow arXiv:2601.18734v3. SDFT is Shenfeld et al., "Self-Distillation Enables Continual Learning", ICML 2026; its EMA teacher and its pi(.∣x,c)pi(.|x,c) formulation are in Section 3.

AuthorsOpenReview ↗

Response to Reviewer mK4c (Part 3)

6. SFT on L1 is already reported, on this dataset and these scales (Q5).

Zhao et al. Table 2 gives 1.7B averages of 37.1 for the base model, 35.8 for SFT on the same solutions, 37.7 for GRPO and 43.4 for OPSD, and attributes the SFT regression to the concise reasoning style of the OpenThoughts traces. We did not re-run it because our L1 condition is OPSD itself, so our baseline is already the stronger of the two.

7. Whether the student already knows these abstractions, and whether L2--L4 are fitted to Qwen while L1 is not (Q13).

This is the sharpest question in the review and it has two halves.

On the first: the student never sees the hint, at rollout time or at evaluation. The category is privileged teacher context. So the question is not whether the student could name the category but whether telling the teacher changes its scoring, and it measurably does. Reading the L4 and no-hint rows of the table in Part 2, the 17.7-token category raises teacher displacement to 1.38, 1.34 and 1.50 times the no-hint interface at 1.7B, 4B and 8B. If the information were already available to the model in that position, the distribution would not move.

On the second: two conditions in the study are not compiler-written at all. L5 is the final answer extracted verbatim from the reference, and prefix-10% is a literal prefix of the reference. Both match or beat L1 at 4B (+0.19 and +0.46 on the ID peak mean), so the effect cannot be an artifact of the teacher finding compiler-written prose more familiar. What remains is the residual by which the abstractions exceed prefix-10%, up to 0.93 points, and we accept that a style-fit account could still live there. The test is a compiler outside the Qwen family, and we have not run it.

8. 32B students, other model families, other training domains, and contexts richer than L1 (Q6, Q9, Q11).

These four questions all concern generalization beyond our current training setup. Our experiments use one model family and one training pool: Qwen3 at 1.7B, 4B, and 8B, trained on OPSD’s mathematics pool of 29,434 problems. We therefore limit our conclusions to this setting and leave broader validation across model families and training domains to future work.

Two points qualify the single-domain interpretation. First, the evaluation covers multiple domains, including multilingual mathematics on MT-AIME in seven languages, graduate-level science questions on GPQA-Diamond, and two families of logic puzzles, ZebraLogic and AutoLogi. Second, the effect is not uniform across model scales. Within the model sizes evaluated, the results indicate a transition between 1.7B and 4B, with the pattern persisting at 8B.

Our analysis focuses on L1 because it already provides both the derivation and the final answer, making it the most informative condition considered in our study. The existing results show that adding privileged information does not consistently improve utility. L1 produces the largest displacement, but not the highest utility, while adding the answer to the prefix condition still performs worse than all three answer-free abstractions. These findings suggest that providing more privileged information is unlikely to change our main conclusion.

What we will change. We will clarify in the data section that “verified” refers to the labels provided by the source pool and will describe the pool’s provenance and original labels. We will state in the evaluation section, rather than only in Appendix A.1, that thinking mode is enabled and include the Zhao et al. ablation citation there. We will also clarify in Appendix B that the typical lengths are used for calibration rather than as constraints, and move the no-hint results to the main text so that the interface control is presented alongside the KL analysis. Finally, to test whether the advantage of L2–L4 depends on their being generated by Qwen, we will regenerate them with a model outside the Qwen family and repeat the comparison.

Thank you again for these thoughtful comments, which will help us improve the clarity and scope of the paper.

Reviewer_4vHUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates several tiers of privileged information in OPSD setups, and shows that the form the privileged information takes can have a strong effect on the supervision, and the performance of the downstream student.

Critical strengths
  • The paper does a really nice job creating a taxonomy of privileged information, and empirically testing each tier's effects.
  • The paper identifies an under-explored angle in the literature about what form privileged information should take for the best supervision.
  • The paper points out weaknesses of existing approaches, as well as clear evaluation failures/cherry-picking which I do very much support.
Critical weaknesses
  • The paper's proposed hierarchy is a bit fragile and doesn't show a clear trend.
  • Even when there are gains over plain final answer OPSD, the gains are modest, undercutting then case for the offline generation of many of the other forms of privileged context.
  • While some aspects of the paper are novel, many other works have also identified that full solutions might not always be optimal, and that different forms/amounts of information may serve as better guidance for the teacher.
Questions for authors
  • What exactly are the claimed contributions of this work over existing work?
  • Can you identify a clear trend/solid performance gains from the different forms of supervision tested?
3 comments
AuthorsOpenReview ↗

Response to Reviewer 4vHU (Part 1)

Thank you so much for engaging closely with the paper’s central question! Your two questions help us clarify both the contribution and its relation to prior work. In particular, the second question points to an important aspect of the contribution that we did not frame clearly enough in the submission.

1. What the contribution is over existing work.

The claim is not that teacher context matters. Stated that way it would indeed be covered by prior work, and we accept that the submission is readable that way.

The claim is that the one component every OPSD variant treats as irreducible, a complete and verified derivation, is not the component doing the work, and that a roughly 18-token description of the problem can replace it. The closest prior results all keep the solution as the unit of privileged information and vary how much of it the teacher sees: ATESD truncates the reference prefix while retaining the answer, PS-OPSD recasts it as a selected path of state transitions, and Shrestha et al. omit or substitute it to test whether the gains depend on its validity. We remove the solution and keep none of it.

The 1.4 point gain is not the consequence that matters. What matters is what the teacher turns out to need. L2, L3 and L4 are answer-free by contract: all three generation specifications prohibit the final answer, and the audit finds no direct leakage in any of the 3,600 hints from either compiler. They also order what their writer has to know. Writing L1 requires a correct and complete solution. L2 requires a working method. L3 and L4 require only an understanding of the problem itself. So the finding is not that a cheaper context is slightly better. It is that the requirement on the privileged input drops from "someone has solved this problem" to "someone has understood this problem", and at 4B and 8B the student is better for it, not worse.

That is why L4 is the level we care about most even though it is not always the winner. It describes the problem rather than its solution, so it does not presuppose that the problem has been solved. We should be equally clear about what this does not yet establish: our contexts were compiled with the reference solution in view, because that is what isolates granularity from everything else that could vary. Whether an L4 context written without any solution teaches as well is the experiment this result makes worth running, and Appendix C.4 names it as the first direction it opens.

Two further results are not, as far as we know, in the literature. First, a head-to-head with ATESD's mechanism on our own setup. A 10% reference prefix plus the answer reaches an ID peak mean of 63.89 at 4B, above L1 (63.43) and above L5 (63.61), yet below all three answer-free abstractions, with L3 and L4 ahead by 0.93 and 0.74 points. It is also the only one of the four that sees the answer. Rewriting is not truncating, and the difference is explained by neither length nor answer access. Second, we refute a direction the field is currently moving in: Armandpour et al. rank distillation contexts with a training-free gradient-alignment proxy, and we show that teacher-student discrepancy, measured directly on fixed rollouts, orders the contexts opposite to utility at 4B and 8B. L1 displaces the teacher most at every scale and trains the weakest 4B and 8B student.

One more point about scope. The question is not specific to OPSD, and the literature already contains a reason our answer comes out the way it does. SDFT (Shenfeld et al., ICML 2026, cited in our related work) builds a self-teacher by conditioning the model on an expert demonstration, and uses the demonstration in full. Its validity analysis asks two things of that teacher: its samples should reach near-optimal reward, and among the policies that do, it should stay as close as possible in KL to the current policy. Measured at initialization, where the student is still the base model, the displacement D(c) of our Equation 4 is that second quantity, averaged over token positions on fixed rollouts. Our result is that the full solution maximizes it at every scale while training the weakest 4B and 8B student, and that coarser contexts move the teacher less and teach better. Granularity is therefore a lever on the deviation term that framework asks you to minimize, and in our setting the default this family of methods adopts, the full reference, is the setting that maximizes it. Nobody has varied that axis, which is what we did.

SDFT is Shenfeld et al., "Self-Distillation Enables Continual Learning", ICML 2026; the two conditions we refer to are in its Section 3.2.

AuthorsOpenReview ↗

Response to Reviewer 4vHU (Part 2)

2. A clear trend, and the size of the gains.

The trend is this: at 4B and 8B, the self-teacher does not need the derivation. At 1.7B our data does not settle it either way.

You are right that the five levels read as a ranking. They are not meant as one, and we will say so explicitly: L2, L3 and L4 swap places across scales and tasks, and our argument does not require a fixed ordering among them. The line the data draws is between contexts that carry the derivation and contexts that do not. It does not rank the three that do not.

Here is the whole grid.

Evaluation axis1.7B4B8B
In-domain peak mean-0.28+1.39+1.57
MT-AIME-7Lang+1.71+2.10+1.71
GPQA-Diamond+0.81+1.21+1.41
ZebraLogic-0.70+0.40+1.70
AutoLogi+0.19+0.25-0.50

Best answer-free abstraction (L2, L3 or L4) minus L1, in percentage points. The in-domain row is the vs. L1 row of Table 1. The transfer rows are differences of step-200 scores in Table 2, so they do not always match that table's best-trained column, which L5 or L1 can win.

Read across the 4B and 8B columns: an answer-free abstraction beats the full solution on 9 of those 10 pairs. The exception is AutoLogi at 8B, where every trained condition sits within 0.76 points of the others. Read down the 1.7B column: the in-domain gain is gone, though three of the four transfer benchmarks still favor the abstraction.

Each cell reports the best of L2, L3 and L4, so the grid answers whether an answer-free abstraction can replace the derivation. The fixed-level version of that question is answered in Table 3, where L3 and L4 are specified in advance rather than chosen per cell, and it comes out the same way: at 4B and 8B, all 16 three-seed mean differences favor the abstraction across four aggregation rules, 42 of the 48 seed-by-rule comparisons are positive, and under the selection-free mean over all checkpoints every seed favors both.

At 1.7B the evidence supports a narrower statement, not a reversal. The in-domain differences there are flat (L3 -0.12 and L4 0.00 at peak) and the three-seed spread covers zero under every rule, so what the data supports is that the 4B and 8B result does not clearly extend down to 1.7B, not that the derivation becomes necessary. Three of the four transfer benchmarks still favor the abstraction at that scale. The direction is consistent with the mechanism we propose, in which a student that cannot rebuild a procedure from a sparse cue may still gain from seeing one executed.

AuthorsOpenReview ↗

Response to Reviewer 4vHU (Part 3)

2, continued. The size of the gains, and what has to be generated.

The offline pass is an artifact of the controlled comparison, not part of what we are proposing. To vary granularity and nothing else, every level has to be derived from the same reference solution, so for L2 to L4 we rewrote solutions we already had. That is the price of the experiment, not the price of the method.

The point of the result is what has to exist before OPSD can run at all. L1 requires a correct and complete solution for every training problem. L3 and L4 require only that someone, or some model, understood the problem: the generation contracts forbid the final answer, and L4 is a one-line statement of the domain and the objects involved. At 4B and 8B that cheaper requirement trains the better student.

That changes what it takes to build training data. If you hold a pool of problems with no solutions, producing L1 means solving them and verifying the solutions. Producing L4 means reading them. The second is not a smaller version of the first; it does not need a solver, a verifier or an answer key, and the quality bar on the generator is far lower, since what it writes is checkable by inspection in a way a derivation is not. Lowering the granularity of the privileged signal is what lowers that bar, and that is the result we care about.

The size of the gain is secondary to this, and a large gain would make us more suspicious rather than less. Both conditions are built from the same problems, and L1 is not a weak baseline: on this recipe it outperforms SFT on the same solutions and matches or exceeds GRPO (Zhao et al., the OPSD paper, ICML 2026; their Table 2). What we report is that the most expensive signal is unnecessary, not that a new signal is dramatically better. If a 657-token verified derivation and a 17.7-token problem description train students within 1.5 points of each other, the derivation is not where the value is coming from.

One limit we should state plainly, since it bears directly on this. Our L2 to L4 contexts were written with the reference solution in view, because that is what isolates granularity. Whether an L4 context written from the problem alone teaches as well is the experiment this result makes worth running, and Appendix C.4 names it as the first direction it opens.

What we will change. Lead the abstract and introduction with the replacement claim rather than the framework, and state explicitly that we do not claim a monotone granularity ordering. We will also say in the main text what the offline compilation is for, so that it reads as the mechanism of the controlled comparison rather than as a proposed pipeline, and state the requirement each level places on whoever writes it where the results are presented rather than only in the method section.

Reviewer_wtFdOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies how the privileged context provided to the teacher affects performance in on-policy self-distillation (OPSD). The authors compare five levels of context, ranging from full reference solutions to abstract strategy hints, problem categories, and final answers. The main finding is that more detailed context does not necessarily lead to better student performance, and the optimal context depends on model scale and evaluation task.

Critical strengths
  • The question of how teacher context affects OPSD is interesting.

  • The paper provides a reasonably extensive empirical comparison across multiple model scales, benchmarks, and context types.

Critical weaknesses
  • Extremely poor and unclear writing. The paper is unnecessarily difficult to follow, with excessive jargon, complicated phrasing, and poorly explained terminology throughout. The abstract is difficult to read. The results section also just restates numbers from tables without offering any interesting insights.

  • Limited experimental scope: All experiments are conducted using LoRA fine-tuning on Qwen3 models, making it unclear whether the findings generalize to other model families or full fine-tuning. Additionally, the experimental coverage is inconsistent across context levels, with only L1, L3, and L4 evaluated across multiple seeds.

  • Limited novelty: The primary conclusion appears to be that the choice of teacher context affects OPSD performance. However, several existing works cited by the authors have already investigated different forms of teacher context. It is unclear what new insight this paper provides.

  • Limited significance of the findings: The improvements over the full-solution baseline are small, and the preferred context level varies across models and tasks. The paper does not provide a clear explanation of why certain context types outperform others, nor does it establish a practical principle for selecting the appropriate context.

Questions for authors
  1. Novelty: What is the main novel insight relative to prior work?

  2. Evaluation metric: The ID peak mean averages the best checkpoints independently selected for each benchmark, meaning it does not necessarily correspond to any single trained model. This seems like an impractical metric. What motivates its use?

  3. Multi-seed experiments: Why are L1, L3, and L4 evaluated across three seeds, while L2 and L5 are reported using only a single seed (Lines 219–223)?

  4. Full fine-tuning: Why was LoRA chosen for all experiments? Would the conclusions hold under full-parameter fine-tuning? Can the authors replicate some of the experiments using full fine-tuning?

  5. Practical utility: Given that different context levels perform best across models and benchmarks, how should a practitioner select the appropriate context level without performing the full set of experiments?

  6. Writing and presentation: Several formulations appear unnecessarily complicated, such as Equation 2, which essentially formalizes performance evaluation on a benchmark. Similarly, terminology such as "under peak reporting" is introduced without clear explanation. Could the authors simplify the presentation and focus more on interpreting results rather than listing numerical differences?

3 comments
AuthorsOpenReview ↗

Response to Reviewer wtFd (Part 1)

Thank you for the review. We accept the presentation criticism, and your two examples are well chosen, so we take those first. The technical questions we answer from what is already in the paper.

1. Writing, Equation 2, and terminology (Q6, W1).

You are right about Equation 2. The context-utility functional is referenced once and used nowhere, and we will remove it. "Under peak reporting" has the same problem for a different reason: it appears in the introduction, while the metric it refers to is not defined until Section 4. That goes too, along with the formalism-first framing of the abstract and introduction.

On the results section restating numbers: the paper does contain a mechanism and a test of it, and we agree it is not where a reader would find it. The account is that a verified derivation commits the teacher to one route and pulls the student back whenever its own rollout leaves that route. The test is in Table 21. At 4B, routing the student's incorrect rollouts to the full solution gives an ID peak mean of 63.06, against 64.54 for routing them to the category and 63.43 for fixed L1. Anchoring the teacher precisely where the student went wrong costs 1.48 points and falls below the fixed baseline.

The item-level evidence points the same way:

Abstraction against L1, step 2001.7B4B8BSource
Reasoning completed, percent70.0 vs 65.885.0 vs 79.286.7 vs 85.0Table 19
ZebraLogic puzzles rescued from L1-unsolved18.1%29.1%35.3%Table 18, Figure 8

The first row is the Complete column of Table 19, abstraction against L1 at each scale. The second is the New solve column of Table 18, the share of puzzles L1 never solves that the abstraction does.

2. The ID peak mean (Q2).

We agree, and we report it because it is the convention of the method we intervene on: Zhao et al. evaluate every 20 steps to 100 and report the best score per benchmark. Read under any other rule, our L1 would not reproduce published OPSD.

That is why the submission adds three further rules, two of them selection-free, rather than relying on the peak. The conclusion does not depend on it. In Table 3, all 16 three-seed mean differences at 4B and 8B favor the abstraction under all four rules. Under the selection-free mean across all checkpoints, every individual seed favors both L3 and L4. At the fixed step-200 endpoint, L2 and L4 lead L1 by 1.67 points at 4B. If the peak is discounted entirely, the result stands on the two selection-free columns. We will say that in the results section instead of leaving it to be assembled from Table 3.

AuthorsOpenReview ↗

Response to Reviewer wtFd (Part 2)

3. Seeds for L1, L3 and L4 but not L2 and L5 (Q3, W2).

The design reason is that L3 and L4 are the coarsest answer-free contexts, so replicating the L1/L3/L4 contrast tests the central claim in its strongest form: if the least informative contexts beat the derivation, the derivation is not necessary. L2 and L5 can strengthen that case but cannot establish it, and three seeds at three scales read under four rules was what the compute allowed.

The 8B headline of +1.57 is L2 at a single seed. The seed-replicated figures, which we should have led with:

ContrastPer-benchmark peakAll-checkpoint mean
4B, L3 minus L1+0.71+1.03
4B, L4 minus L1+0.96+1.01
8B, L3 minus L1+0.25+0.49
8B, L4 minus L1+0.49+0.62

Mean paired difference over seeds 42--44, from Table 3; the second column is selection-free. All eight means are positive, and in that column every individual seed is too.

We will quote these in the abstract and mark single-seed numbers as single-seed.

4. LoRA and full fine-tuning (Q4, W2).

LoRA is the published OPSD configuration rather than a choice of ours. Zhao et al. run all experiments with LoRA and give rank 64, alpha 128, learning rate 5e-6, effective batch 32, 1,024-token rollouts and sampling temperature 1.1 in their Table 6. We adopt them unchanged so that our L1 condition is OPSD and not a variant of it. Our intervention is one field of the teacher's prompt; changing the adapter or the optimizer would make every level a different method and would no longer isolate context granularity.

The effect survived everything we did vary: three seeds, four aggregation rules, two compilers with one of them the student's own base model, a bridge shared across all five levels, a doubled rollout budget, and three student scales. Under the shared bridge every intermediate context beats L1 by 1.02--2.32 points, and at a 2,048-token budget all three stay ahead by 0.92--1.94. Full-parameter fine-tuning is one axis we did not vary, and we will name it in the limitations.

AuthorsOpenReview ↗

Response to Reviewer wtFd (Part 3)

5. How a practitioner selects a level (Q5).

There is a rule, and it does not require the full set of experiments.

Above 4B, use the category or the framing, and reach for the cheaper one first, since L4 is 17.7 tokens. This is the replicated finding rather than a per-task tuning result. Across four rules and two scales, all 16 mean differences are positive, L4 leads L1 at peak in all six 4B and 8B runs, and an answer-free abstraction leads the full solution on 9 of the 10 scale-axis pairs. At 1.7B, keep the full solution.

If the level has to be chosen without training any students, Appendix E gives a proxy that needs only forward passes of the frozen base model. The hint advantage is how much more probability a hinted teacher puts on the student's own sampled tokens than a hint-free one does, on the same rollout. Measured on 1,200 training problems with four base-model rollouts each, it ranks the levels the way training does:

LevelHint advantage, no training4B ID peak mean, after training
L3-0.02064.81
L4-0.02764.63
L2-0.04264.07
L1-0.08463.43

The left column is forward passes only; the right is Table 1. The proxy also separates the levels cleanly, with a median between-level to within-level variance ratio of 4.5 against 1.6 for the absolute student-teacher gap, and a per-problem argmax that is stable across rollouts for 78 percent of problems.

A practitioner runs one offline scoring pass, not nine training runs. We agree this answer should not have been in an appendix, and we will move it into the main text.

6. Novelty (Q1, W3).

The contribution is not that context matters. It is that the solution is not the unit of privileged information. ATESD truncates the reference prefix while keeping the answer, PS-OPSD recasts it as a selected path of state transitions, and Shrestha et al. omit or substitute it. All of them keep the solution as the thing being varied. We remove it and keep nothing of it, which is what makes the required input obtainable for a problem nobody has solved. The target is also wider than one method: SDFT conditions its self-teacher on a full expert demonstration, and the same question applies there.

We tested the claim in the form that could have failed. Full-solution dominance predicts L1 is best, and it fails at 4B and 8B on 9 of the 10 scale-axis pairs. We also ran ATESD's mechanism head to head. A 10% reference prefix plus the answer reaches 63.89 at 4B, above L1 at 63.43 but below all three answer-free abstractions, which it trails by 0.19 to 0.93 points. It is the only one of the four that sees the answer. Rewriting is not truncating.

7. On the significance of the findings (W4).

This one is from your weakness list rather than your questions. The mechanism is in item 1 and the selection rule in item 5, so what is left is the size of the improvements.

What the result changes is not the score. It is what has to exist before OPSD can run at all. L1 requires a correct and verified solution for every training problem. L4 requires only that someone read the problem: it is a one-line statement of the domain and the objects involved, and the generation contract forbids the final answer. At 4B and 8B the cheaper requirement trains the better student.

For a pool of problems with no solutions, producing L1 means solving them and verifying the solutions. Producing L4 means reading them. The second needs no solver, no verifier and no answer key, and the quality bar on whoever writes it is far lower, since a one-line problem description is checkable by inspection in a way a derivation is not.

A small gain is therefore the expected shape of this result, not a weakness of it. If a 657-token verified derivation and a 17.7-token problem description train students within 1.5 points of each other, the derivation is not where the value is coming from. L1 is also not a weak baseline: on this recipe it beats SFT on the same solutions and matches or exceeds GRPO (Zhao et al., their Table 2). One limit worth stating. Our L2 to L4 contexts were written with the reference solution in view, because that is what isolates granularity. Whether an L4 context written from the problem alone teaches as well is the experiment this result makes worth running, and Appendix C.4 names it as the first direction it opens.

We hope these responses address at least some of your concerns, and we thank you again for your careful and constructive review.

Zhao et al. is the OPSD paper, ICML 2026; the table numbers we quote follow arXiv:2601.18734v3. SDFT is Shenfeld et al., "Self-Distillation Enables Continual Learning", ICML 2026.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.