acceptodds
Under review as a conference paper at ICLR 2027

Masked Self-Distillation: Letting Reasoning Models Keep Thinking

16 views

Abstract

On-policy self-distillation (OPSD) has emerged as a post-training paradigm for LLMs. However, we show that naive OPSD degrades reasoning models on tasks that demand exploration and reflection, such as competition mathematics. We trace this degradation to a context asymmetry: exploration and reflection are redundant to a solution-aware teacher, so uniform distillation causes the student to lose both abilities. Specifically, on identical student prefixes, the privileged teacher assigns far lower probability than the student to a sparse set of epistemic markers linked to exploration and reflection, such as “Wait” and “perhaps”. We propose masked self-distillation, which withholds the distillation loss at marker positions and retains it everywhere else. By letting reasoning models keep thinking, the masked objective reverses the degradation in every tested configuration and outperforms SFT and GRPO. More broadly, hindsight is a poor guide to exploration: privileged self-distillation should transfer its preferences selectively, not uniformly.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, on-policy self-distillation, reasoning, privileged information, epistemic markers

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_iLxDOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

The paper studies on-policy self-distillation with a teacher conditioned on the question and a reference reasoning trace, while the student is conditioned only on the question. When both models score the same student-generated prefixes, the teacher assigns lower probabilities to epistemic markers such as “Wait” and “perhaps.” The proposed method masks the full-vocabulary distillation loss at positions where the student samples a token from a predefined marker set. Across three checkpoints experimented on AIME24, AIME25, and MATH-500, the authors find that the proposed method prevents the degradation observed with standard uniform distillation and outperforms a matched non-marker masking control.

Critical strengths
  1. Skipping a small set of loss terms substantially improves accuracy over ordinary self-distillation across all tested model–benchmark combinations (Table 2).
  2. The matched non-marker control accounts for mask size and several token statistics, providing useful evidence that the choice of positions matters.
  3. The paper is generally well written and easy to follow.
Critical weaknesses
  1. The empirical results shown for outperforming the base model and SFT are less convincing than evidence for avoiding degradation. The 7B gains over SFT are only 0.8 to 1.8 points.
  2. Also, I find that the proposed setup does not establish that marker meaning explains the benefit. They do not match the teacher’s reduction in the sampled token’s probability, and the entropy mask also avoids much of the degradation (Fig. 4). Marker counts and answer lengths alone do not measure reflection.
Questions for authors
  1. Does masking non-marker positions with similarly large teacher-induced probability drops recover the benefit? Which token IDs account for most masked positions?
  2. Why are base-model scores below the model card’s results?
4 comments
AuthorsOpenReview ↗

Response to Reviewer iLxD (1 / 2)

We sincerely appreciate your careful analysis and your encouraging feedback! Your probing question on probability-drop matching directly inspired a new control experiment that makes our mechanistic evidence much tighter. We hope the following responses resolve all of your questions, and we truly appreciate your support of our work.

Q1 (Part 1) / W2 (Part 1): Does masking non-markers with comparable probability drops (Δp\Delta p) recover accuracy?

Response. Thank you for suggesting this control experiment! In addition to the covariates matched in Table 2, we ran a new Δp\Delta p-matched non-marker control on DeepSeek-R1-Distill-Qwen-1.5B that also matches the sampled token's probability drop Δpt=ptS(y^t)−ptT(y^t)\Delta p_t = p_t^S(\hat{y}_t) - p_t^T(\hat{y}_t):

Method (DeepSeek-R1-Distill-Qwen-1.5B)AIME24 avg@12AIME25 avg@12MATH-500 pass@1Average
Base (Table 2)27.220.869.639.2
Naive OPSD (Table 2)21.216.565.534.4
Matched non-marker mask (Table 2)22.416.266.335.0
Δp\Delta p-matched non-marker mask (new)21.916.466.034.8
Masked OPSD (ours) (Table 2)32.524.577.644.9

Masking high-Δp\Delta p non-markers remains near Naive OPSD because large probability drops on non-markers primarily occur when the teacher corrects student calculation or branching errors (Figure 3). Consequently, masking them leaves epistemic markers unprotected while removing helpful error correction. We will add this control to Table 2 and Section 5.3.

Q1 (Part 2): Which token IDs account for most masked positions?

Response. Thanks for asking for this concrete token-level breakdown! Across the masked positions in our 100-update run on DeepSeek-R1-Distill-Qwen-1.5B, the top five token IDs account for 33.1% of all masked occurrences: 14190 (Wait, 15.6%), 8365 ( perhaps, 5.9%), 7196 ( maybe, 4.0%), 13824 ( Wait, 4.0%), and 1744 ( think, 3.6%), followed by 3520 ( actually, 2.1%), 3783 ( wait, 2.0%), 4977 ( seems, 1.8%), and 88190 ( Hmm, 1.6%). In particular, variants of Wait (14190, 13824, 3783) alone account for 21.6% of all masked positions. We will include this token-level breakdown in Appendix A.3.

W2 (Part 2): Why does the student-entropy mask in Figure 4 avoid collapse while still trailing the marker mask?

Response. Thank you for spotting that in Figure 4 (Appendix B), the equal-budget highest-student-entropy mask stays near the base checkpoint (avoiding naive OPSD's collapse while trailing the marker mask). This follows directly from the token-level properties in Section 3.1. First, it avoids collapse because mid-sentence hedging and reconsideration tokens naturally carry high student uncertainty, so the top-entropy mask protects part of the model's reflective behavior. Second, it trails the marker mask because much of the entropy mask budget falls on uncertain calculation steps where teacher correction is beneficial rather than on epistemic markers (Figure 3). We will also make the summary of Figure 4 in Section 6 precise so that it clearly distinguishes the entropy mask from the random mask.

W2 (Part 3): Do marker counts measure reflection?

Response. We appreciate this careful methodological distinction! Tracking epistemic markers as indicators of reflection initiation is a common, lightweight practice in the reasoning literature (e.g., DeepSeek-R1 and Kim et al., 2026a, Appendix E.5). At the same time, we agree that lexical marker counts and response lengths are surface-level proxies rather than direct measures of substantive verification or self-correction, as an individual marker can occasionally act as a conversational pause rather than genuine reflection (Appendix G distinguishes these cases qualitatively). Following your helpful point, we will explicitly acknowledge this limitation in Section 6 and tighten our wording throughout the paper to refer precisely to "epistemic marker usage" rather than equating marker counts directly with reflection depth.

AuthorsOpenReview ↗

Response to Reviewer iLxD (2 / 2)

Q2: Why are the base scores lower than the model card's?

Response. Thank you for flagging this discrepancy with the model-card numbers! The gap arises from three concrete differences between our unified evaluation protocol (Appendix C, Table 9) and the official DeepSeek-R1 evaluation setup:

  1. Generation length cap and sampling budget: To keep evaluation tractable across all models and training ablations, we use a 16,384-token maximum generation length and avg@12 on AIME, whereas the DeepSeek-R1 model card uses a 32,768-token cap and 64 samples. As a result, longer reasoning traces that exceed 16,384 tokens are truncated before producing a final \boxed{} answer.
  2. System prompt and prefix formatting: To maintain a uniform chat template across model families (DeepSeek-R1-Distill-Qwen and OpenMath2-Llama3.1-8B), we include a standard system prompt ("You are a helpful assistant.", Appendix D) without forcing an initial <think>\n prefix, whereas the DeepSeek-R1 evaluation protocol omits system prompts and enforces <think>\n at the start of every response.
  3. Strict rule-based verification: We grade all outputs strictly using symbolic rule-based verification (Math-Verify) on extracted \boxed{} answers, without fallback heuristics or LLM-assisted grading.

Crucially, all methods in our paper (Base, SFT, GRPO, Naive OPSD, and Masked OPSD) are evaluated under the exact same prompt template, 16,384-token cap, sampling seeds, and strict Math-Verify grader, ensuring an apples-to-apples comparison. We will explicitly note these protocol differences relative to the model cards alongside Table 2 and in Appendix C.

W1: Modest gains over Base and SFT on the 7B model

Response. We appreciate your careful observation! We agree that while Masked OPSD achieves clear gains over Base and SFT on the 1.5B model, its margins over Base and SFT on the 7B model are modest. Our primary, consistent finding across all nine model–benchmark configurations is that masking reverses the severe accuracy collapse of naive OPSD.

Thanks to the authors for providing additional results and analysis, including the justification for the model card score discrepancy. As a result of the authors' response, I have revised the soundness score.

Thank you very much for taking the time to re-evaluate our response and for raising your Soundness score to 3 ("no substantive concerns"). We truly appreciate your thoughtful and constructive feedback, which helped strengthen the paper.

Please let us know if there are any remaining questions or concerns we could clarify further to support a stronger overall assessment during the discussion period.

Reviewer_3gFVOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper shows that naive OPSD degrades reasoning on tasks where exploration and reflection matter. Based on the observation that the solution-aware teacher assigns much lower probability to epistemic markers, the authors propose masking the tokens in a marker list and excluding them from the distillation loss. The resulting method outperforms SFT, GRPO, and naive OPSD on several benchmarks for the tested checkpoints.

Critical strengths
  • The paper is easy to read and clearly written.
  • The method is well motivated by diagnostic experiments and observations.
Critical weaknesses
  • The failure mode addressed here has already been pointed out in prior work (Kim et al., 2026a), and related remedies have recently appeared (Shen et al., 2026; Kim et al., 2026b). As far as I understand, the differences are that this paper uses a pre-defined marker list rather than a signal-based criterion, and removes the loss at those positions rather than reversing it. I am not fully convinced that these differences constitute a sufficiently large contribution, though I would be happy to be corrected on this point.
  • The baselines are fairly basic. Comparisons with other OPSD variants would strengthen the results.

References

  • Kim et al., 2026a. Why does self-distillation (sometimes) degrade the reasoning capability of LLMs?
  • Kim et al., 2026b. Rebellious Student: Reversing Teacher Signals for Reasoning Exploration with Self-Distilled RLVR.
  • Shen et al., 2026. Anti-Self-Distillation for Reasoning RL via Pointwise Mutual Information.
Questions for authors
  • Could the authors clarify the contribution of this paper relative to Shen et al. (2026) and Kim et al. (2026b)?
  • Are there explicit inclusion/exclusion criteria that the authors used to construct the marker list? Membership seems potentially subjective. Relatedly, what are the advantages of this pre-defined list over the signal-based selection used in related work?
7 comments
AuthorsOpenReview ↗

Response to Reviewer 3gFV (1 / 2)

Thank you for your thorough review and valuable suggestions on related work and OPSD baselines! We sincerely appreciate your recognition that our paper is clearly written and well motivated by the diagnostic experiments, as well as your openness to clarification. Your comments helped us sharpen our positioning and add direct comparisons against recent OPSD variants. We hope the responses and new experiments below address your concerns, and we would be very grateful if you could reconsider our work.

W1 / Q1: Contribution relative to Kim et al. (2026a), AntiSD (Shen et al., 2026), and RLRT (Kim et al., 2026b)

Response. Thank you for pointing out Kim et al. (2026a), RLRT (Kim et al., 2026b), and AntiSD (Shen et al., 2026), and for giving us the opportunity to clarify our contribution! While all four works study teacher–student discrepancies under a solution-aware teacher, they address different questions under fundamentally different paradigms (reward-free OPSD vs. multi-rollout GRPO):

  1. Compared with Kim et al. (2026a) (cited in §2.2): fine-grained token attribution and a working OPSD remedy. While Kim et al. (2026a) also observed that self-distillation can degrade reasoning, they did not isolate the token-level conflict between harmful epistemic suppression at reflection markers (Table 1) and beneficial step-level error correction on reasoning steps (Figure 3), nor did their global adjustments restore base accuracy in OPSD (leaving open "how to filter out non-informative content while retaining epistemic expressions", Appendix A). We provide this fine-grained attribution and validate positive recovery in standalone OPSD against a matched non-marker control (Table 2) and cross-task bounds (§5.4, Table 3).

  2. Compared with RLRT (Kim et al., 2026b): different target problem and training paradigm. RLRT targets exploration boundary collapse in RLVR rather than standalone OPSD: it "augments GRPO by reinforcing these tokens on correct rollouts" (Eq. 7 in §5) while keeping standard GRPO updates on incorrect ones.

  3. Compared with AntiSD (Shen et al., 2026): different training paradigm. AntiSD shares our motivation, but departs from standalone OPSD and relies on GRPO with multi-rollout outcome rewards (plus teacher-entropy gating) to add a sign-reversed PMI bonus to the GRPO advantage (Eq. 2 in §2, Eq. 6 in §3.2), because signal reversal is an ascent step that "is not self-terminating". In contrast, we show that pure OPSD can be repaired without outcome rewards, multi-rollout sampling, or signal reversal simply by withholding loss at epistemic marker positions while keeping standard teacher supervision on substantive reasoning steps.

WorkCore Problem & ParadigmRequires Outcome Reward / GRPO?Analysis & Intervention Mechanism
Kim et al. (2026a)Observe SD reasoning degradationObservational diagnosis (no working OPSD fix)Broad certainty-bias view, global SD adjustments fail to restore base accuracy
RLRT (Kim et al., 2026b)Mitigate exploration collapse in RLVRYes (GRPO, 8 rollouts/prompt, reweights correct rollouts)Treats divergence as an exploration signal to amplify self-driven tokens (Eq. 7 in §5)
AntiSD (Shen et al., 2026)Preserve deliberation against shortcut biasYes (GRPO, 8 rollouts/prompt + entropy gate)Adds sign-reversed PMI as an incremental token reward to GRPO advantage (Eq. 2 in §2, Eq. 6 in §3.2)
Masked OPSD (Ours)Decompose & repair hindsight suppression in OPSDNo (Pure OPSD, 1 rollout/prompt, no verifier)Isolates epistemic suppression from step-level error correction with confounder-matched controls & cross-task bounds

We will expand §2.2 with the table above to clearly distinguish our standalone OPSD analysis and intervention from Kim et al. (2026a) and GRPO-based remedies (AntiSD and RLRT).

AuthorsOpenReview ↗

Response to Reviewer 3gFV (2 / 2)

W2: How does masking compare with other OPSD variants?

Response. We appreciate this constructive suggestion! We completely agree that benchmarking directly against recent OPSD variants substantially strengthens our empirical evaluation. Following your recommendation, we expanded our evaluation on DeepSeek-R1-Distill-Qwen-1.5B under the same 100-update protocol (Table 9) to compare against four representative OPSD variants: EOPD (Jin et al., 2026, Eq. 7–10: entropy-gated on-policy distillation), AD-OPSD (Peng et al., 2026: blending the teacher distribution with the frozen base model on high-entropy tokens where ptS>ptTp_t^S > p_t^T), PW-OPSD (position-weighted OPSD that downweights early reasoning tokens), and Purified OPSD (Shen et al., 2026, Eq. 13–18: reconstructing the teacher target via centered PMI against the frozen base model):

Method (DeepSeek-R1-Distill-Qwen-1.5B)AIME24 avg@12AIME25 avg@12MATH-500 pass@1Average
Base (Table 2)27.220.869.639.2
Naive OPSD (Table 2)21.216.565.534.4
Matched non-marker mask (Table 2)22.416.266.335.0
EOPD (new)26.419.768.838.3
AD-OPSD (new)26.920.071.239.4
PW-OPSD (new)27.520.671.840.0
Purified OPSD (new)28.321.473.040.9
Masked OPSD (ours) (Table 2)32.524.577.644.9

While unconditioned OPSD variants (EOPD, AD-OPSD, PW-OPSD, Purified OPSD) mitigate naive OPSD's collapse and recover performance to near the Base model, Masked OPSD achieves the strongest improvement across all three benchmarks. This is because unconditioned positional or scalar divergence weights inevitably attenuate helpful teacher corrections alongside harmful suppression, whereas targeted marker masking preserves both reflective behavior and step-level error correction. AntiSD and RLRT are not included in this table because they require a verifiable outcome reward and 8 rollouts per prompt under GRPO, whereas Purified OPSD and AD-OPSD serve as the closest reward-free OPSD analogues of their signal-based token modulation. We will include this comparison table in Section 5.2 and Appendix C.

Q2 (Part 1): How was the epistemic marker list constructed?

Response. Thanks for raising this question! After Qwen3-32B extracted candidate phrases from 1,000 base trajectories, we retained generic expressions that initiate hesitation, self-checking, or backtracking and excluded problem-specific or mathematical terms (Appendix F). To test sensitivity to the choice of lexicon, Appendix A.3 selects a core of 41 expressions using frequency, probability drop, and student entropy. Replacing the full lexicon with this core lowers 7B accuracy by at most 0.4 percentage points across the three benchmarks (Table 7). We will state this screening criterion explicitly in Appendix F.

Q2 (Part 2): Why use a marker list instead of a scalar signal rule?

Response. Unlike GRPO methods that rely on outcome rewards to filter wrong rollouts, reward-free OPSD cannot rely on unconditioned scalar signals (such as Δpt\Delta p_t, KL, or student entropy) because they conflate reflection with calculation errors. As shown in Figure 3, uncertain calculation steps and epistemic markers both exhibit large teacher–student discrepancy and high uncertainty. Masking or downweighting by scalar signals (as in EOPD, AD-OPSD, and Purified OPSD in W2 above, or the entropy mask in Figure 4) inevitably discards beneficial teacher error correction on math steps. We will clarify this distinction in Section 5.3.

Thank you for the additional comparisons.

Since the math tasks support verifiable rewards and the paper already includes GRPO as a baseline, a direct comparison with RLRT and AntiSD would help clarify the trade-off between accuracy and training cost.

The rationale for focusing on reward-free methods would be stronger if the paper identified a setting where that restriction matters, for example, tasks where reliable outcome verification is unavailable.

I therefore maintain my original rating.

Reviewer_46KeOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper identifies a failure mode of on-policy self-distillation: privileged teachers can suppress epistemic markers associated with exploration and reflection. It proposes masking the distillation loss at these positions to mitigate reasoning degradation.

Critical strengths
  1. Important problem and thorough analysis. The same-prefix diagnostics reveal concentrated teacher–student disagreement at epistemic marker positions. Matched-mask and other controls provide substantial support for the proposed explanation.

  2. Effective marker-based intervention. The authors construct a useful epistemic marker lexicon and demonstrate that targeted masking consistently reverses OPSD degradation across the tested mathematical reasoning configurations.

Critical weaknesses
  1. Limited methodological novelty. The intervention is a simple, fixed lexicon-based loss mask. The main contribution lies in the diagnosis and empirical findings rather than the technical sophistication of the method.

  2. Limited experimental breadth. While cross-task controls are included, the strongest evidence centers on mathematical reasoning. More extensive evaluations on coding, logical reasoning, or planning would strengthen the generality of the conclusions.

Questions for authors

NA

1 comment
AuthorsOpenReview ↗

Response to Reviewer 46Ke

Thank you very much for your positive assessment and forward-looking suggestions! We truly appreciate your recognition of the importance of the problem, our same-prefix diagnostics, the confounder-matched controls, and the effectiveness of our intervention. Your question on broader applicability directly motivated new experiments across coding, logical deduction, and planning, which we are excited to share below.

W1: Does a fixed lexicon mask offer enough methodological novelty?

Response. Thank you for this thought-provoking perspective on methodological simplicity! We agree that the mask is simple, and we view this simplicity as a deliberate strength: it provides an interpretable, zero-overhead intervention that pinpoints exactly where the loss is removed and tests its causal effect. Moreover, our analysis in §3.1 (Figure 3) and §5.3 / Appendix B (Figure 4) shows why a semantic lexicon outperforms seemingly more flexible scalar rules (such as masking by teacher–student divergence Δpt\Delta p_t / KL or student entropy): unconditioned scalar signals conflate beneficial error correction at uncertain calculation steps with harmful suppression of epistemic markers. Thus, the lexicon mask is not merely a heuristic shortcut, but a clean way to decouple epistemic exploration from step-level error correction, pointing to a natural direction for future work: context-aware gating that dynamically distinguishes corrective teacher signals from hindsight-driven suppression.

W2: How far do the findings generalize beyond mathematics?

Response. Thank you for your constructive suggestion to evaluate generalization beyond mathematics! We completely agree that testing broader reasoning domains makes the paper much more compelling. Below, we reproduce our cross-task results from §5.4 (Table 3, 7B) for your convenience alongside new 100-update experiments on DeepSeek-R1-Distill-Qwen-1.5B across coding (MBPP sanitized), logical reasoning (ProofWriter), and planning (PlanBench Blocksworld), using the frozen marker lexicon without domain-specific tuning:

Domain & Task / MetricModelBaseNaive OPSDMasked OPSDMasked − Naive
Math: AIME24 / avg@12 (Table 3)7B47.8%39.5%49.0%+9.5 pp
Science: SciKnowEval L3 / aggregate (Table 3)7B41.8%39.2%42.7%+3.5 pp
Tool Use: ToolAlpaca / task success (Table 3)7B54.2%55.4%55.6%+0.2 pp
Intent: MASSIVE / accuracy (Table 3)7B83.9%84.5%84.5%0.0 pp
Coding: MBPP sanitized / pass@1 (new)1.5B43.97%40.86%48.25%+7.39 pp
Logic: ProofWriter / final-line check (new)1.5B17.33%21.67%23.33%+1.66 pp
Planning: Blocksworld / valid plan (new)1.5B0.00%2.78%5.56%+2.78 pp

Across all seven benchmarks, Masked OPSD reverses Naive OPSD's degradation and surpasses the base model on reasoning tasks where hindsight supervision suppresses self-checking (with the clearest out-of-domain gains on Coding and Science), while on format-constrained tasks (Logic, Planning, Tool Use, and Intent), masking preserves Naive OPSD's format gains and remains comparable or slightly better, consistent with weaker marker concentration on these tasks. We will incorporate these new results into Section 5.4.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global Response: Summary

Dear Area Chair and Reviewers,

We thank the AC for coordinating the review process and all reviewers for their time and thoughtful evaluations. Our reviewer-specific responses address each concern in detail. Here we summarize the cross-reviewer evidence and revisions.

1. Expanded Baselines and Positioning vs. Recent OPSD and RLVR Methods (Reviewers 3gFV & 46Ke)

Four recent OPSD variants (Table 2, Section 5.2, Appendix C): Under the same 100-update protocol on DeepSeek-R1-Distill-Qwen-1.5B, we benchmarked EOPD (38.3 avg), AD-OPSD (39.4), PW-OPSD (40.0), and Purified OPSD (40.9) across AIME24, AIME25, and MATH-500. While these signal- or position-weighted variants recover near Base (39.2), Masked OPSD (44.9 avg) achieves the strongest gains across all three benchmarks by preserving helpful teacher corrections on non-marker calculation steps. Direct RLVR (GRPO) comparisons (Reviewer 3gFV follow-up): Under identical 100-update settings on 1.5B, Masked OPSD (44.9 avg) outperforms GRPO (39.2), RLRT (40.7), and AntiSD (41.7) across all three benchmarks without outcome rewards. Positioning and verifier-unavailable settings (Section 2.2): We clarified how standalone token-level epistemic masking differs from observational diagnosis (Kim et al., 2026a), RLVR (GRPO) remedies (RLRT, AntiSD), and rollout-level routing (SRPO, Li et al., 2026)—resolving within-trajectory token conflicts and remaining applicable where trajectory-level verification is unavailable (e.g., SciKnowEval L3, privileged-context internalization, ToolAlpaca).

2. Tighter Mechanistic Controls, Lexicon Audit, and Evaluation Protocol (Reviewers iLxD & 3gFV)

Probability-drop matched control (Table 2, Section 5.3): Matching the teacher's probability drop (Δp) on non-marker positions yields 34.8 avg on 1.5B—near Naive OPSD (34.4) and the original matched control (35.0), vs. 44.9 for Masked OPSD—confirming that large probability drops on non-markers primarily reflect beneficial error correction rather than epistemic suppression. Token audit, lexicon criteria, and tightened claims (Table 7, Sections 5.3 & 6, Appendices A.3 & F): We reported the top masked token IDs (top five account for 33.1%; "Wait" variants alone account for 21.6%), explicitly documented lexicon screening criteria alongside the 41-expression core-list check (Table 7), distinguished the entropy mask from random masking in Figure 4, noted modest 7B margins over SFT, and framed marker counts strictly as surface usage proxies rather than direct measures of reflection depth. Evaluation protocol vs. model cards (Table 2, Appendix C): We documented all protocol differences relative to model cards (16,384-token cap, unified system prompt without forced <think>\n, and strict symbolic Math-Verify grading), applied identically across all methods for apples-to-apples comparisons.

3. Cross-Domain Generalization Beyond Mathematics (Reviewers 46Ke & 3gFV)

Three new out-of-domain benchmarks (Table 3, Section 5.4, Appendix A.4): Alongside our four 7B cross-task controls (AIME24 +9.5 pp, SciKnowEval L3 +3.5 pp, ToolAlpaca +0.2 pp, MASSIVE 0.0 pp), we added 100-update 1.5B evaluations across coding (MBPP sanitized: 48.25% vs. 40.86% Naive OPSD, +7.39 pp), logical deduction (ProofWriter: 23.33% vs. 21.67%, +1.66 pp), and planning (PlanBench Blocksworld: 5.56% vs. 2.78%, +2.78 pp) using the frozen marker lexicon without domain tuning. Across all seven domains, masking gains align consistently with epistemic-marker concentration.

We thank the AC and all reviewers again for their time and effort

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.