acceptodds
Under review as a conference paper at ICLR 2027

Free to Ignore: Cornerman Decoding for Distilled Reasoners

Abstract

Distilling long chain-of-thought (CoT) traces from stronger teachers offers a practical route to improving reasoning in small language models. These traces demonstrate both solution content and the organization of reasoning. Yet our analysis shows that students can produce useful intermediate reasoning without using it effectively to advance their solutions. This motivates guidance on how students develop, revise, and complete solutions during generation. We introduce Cornerman Decoding, which pairs a frozen student with a learned local adviser. The adviser observes only the latest student paragraph and either remains silent or suggests continuing, switching approaches, checking a claim, or finishing, without supplying new solution steps or answers. The student retains the full problem and reasoning history and is free to follow or ignore the advice. Across reasoning benchmarks, we evaluate the same frozen 1.5B adviser with 1.5B and 7B students fine-tuned on complete CoT traces. Compared with decoding the same students without advice, Cornerman improves macro-average accuracy by 5.5 and 1.0 percentage points, respectively, while reducing mean student-generated tokens by 45.2% and 21.1%. Paired case studies further illustrate how guidance can help students reach correct answers that do not appear in their uncoached trajectories.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, reasoning, inference-time intervention, knowledge distillation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_dnRcOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a frozen adviser that observes a student's latest reasoning paragraph and provides optional structural guidance, such as checking a claim, continuing an approach, switching strategies, or finishing.

Critical strengths

The distinction between non-binding advice and externally controlled reasoning is interesting. The experiments cover nine benchmarks, and the authors include meaningful controls and paired trajectory analyses. The 1.5B greedy-decoding accuracy improvement is substantial, particularly on MATH500 and GPQA-Diamond.

Critical weaknesses

The paper suffers from the following issues:

(1) The paper claims a 45.2% reduction in generated tokens for the 1.5B student. However, the metric excludes adviser-generated tokens, adviser prefill, and the overhead of repeatedly restarting student decoding. So, for a fair comparison, you need to include these missed generated tokens. This could be concerning for the 7B model where the token reduction is 21.1%.

(2) The strongest results come from 1.5B greedy decoding. However, the accuracy improvement decreases to approximately 1 percentage point for the 7B model and 1.4 points under sampled decoding for 1.5B model. The 7B results are also inconsistent over various datasets. These findings raise concerns about the privilege of the proposed method over the SOTA methods. In Table 3, page 7, the paper compares Cornerman with ACTS, DEER, JET, and LC-R1. However, its advantage over the strongest alternatives is relatively small. ACTS achieves nearly the same accuracy with approximately 46% fewer student-generated tokens.

Questions for authors

Please check the Critical Weaknesses section.

2 comments
AuthorsOpenReview ↗

Response to Reviewer dnRc

Thank you for recognizing the distinction between non-binding advice and external control. We address the two concerns with full-system measurements and additional budget-controlled comparisons.

1. Complete inference cost

The reported 45.2% and 21.1% reductions measure student-generated output across nine benchmarks. We have now measured end-to-end cost on MATH500, including student and adviser prefill and decoding, and student-context recomputation after advice replacement.

Measurements use one A800-80GB with vLLM 0.7.3 and prefix caching. Batch time covers all 500 problems; single-query latency averages the same 50 questions at concurrency 1. Estimated FLOPs sum 2 × model parameters × uncached processed tokens over both models, excluding attention operations.

StudentMethodTFLOPs/problemBatch minLatency s
1.5BFull-CoT29.672.367.8
1.5BCornerman39.126.233.1
7BFull-CoT60.718.548.2
7BCornerman116.925.838.0

Adviser generation averages 387 and 338 tokens per problem for the 1.5B and 7B students, respectively; these are included in the accounting above. After including adviser computation and student-resumption overhead, 1.5B Cornerman completes all 500 problems in 26.2 rather than 72.3 minutes, a 2.8× speedup. Mean single-query latency falls from 67.8 to 33.1 seconds, a 51.2% reduction, with 1.32× the estimated FLOPs. For 7B, single-query latency falls from 48.2 to 38.0 seconds (21.2%), while batch runtime and estimated FLOPs increase. The full-system measurements therefore show speedups in both batch processing and single-query response at 1.5B; the 7B runtime benefit is confined to single-query response.

2. Gains across settings and comparison with alternatives

7B and sampled decoding. The gains are smaller than for 1.5B greedy decoding, but accuracy improves alongside shorter output in both settings. With the same frozen 1.5B adviser, the 7B student gains approximately 1 percentage point in macro accuracy while generating 21.1% fewer student tokens. The effect is benchmark-dependent, with GPQA-Diamond improving from 40.4% to 46.5%.

Under sampled decoding, Cornerman raises macro accuracy from 48.4% to 49.8% while reducing mean student output from 9,115 to 6,383 tokens (30.0%). The smaller accuracy gain is therefore accompanied by substantially shorter reasoning, with the student frozen and no new solution steps supplied by the adviser.

ACTS, JET, and s1. ACTS does produce approximately 46% fewer student tokens in Table 3. This comparison excludes its controller computation, just as it excludes our adviser's computation. We therefore compared costs including both models on MATH500 and evaluated Cornerman with 8k and 16k student budgets. We also applied s1-style budget forcing (Muennighoff et al., 2025) to the same Full-CoT student:

MethodAcc. (%)TFLOPs/problemBatch min
ACTS81.036.3N/A
JET80.818.635.3
s1-style, 8k81.613.810.0
Cornerman, 8k83.427.89.9
Cornerman, 16k84.234.817.9

The paper's standard 32k Cornerman result remains 84.4%; the 8k/16k Cornerman rows are additional experiments. JET's row reports a new evaluation of its released checkpoint with the same final-answer scorer. ACTS costs include its 4B controller; its runtime was not measured on this card. The other runtimes use the hardware above.

At 16k, Cornerman exceeds ACTS by 3.2 percentage points at comparable estimated compute (34.8 versus 36.3 TFLOPs/problem). JET uses fewer estimated FLOPs, while Cornerman at 8k achieves higher accuracy and shorter batch runtime. At the same 8k student budget, Cornerman also improves on s1-style budget forcing by 1.8 points with similar batch runtime (9.9 versus 10.0 minutes), though higher estimated FLOPs. Unlike JET's reasoner post-training, Cornerman trains a separate adviser shared across frozen students.

Additional task coverage. On 1,000 MMLU-Pro test questions sampled across all 14 subjects, the same adviser improves the 1.5B Full-CoT student's accuracy from 22.5% to 27.2%, while reducing mean student output from 18,221 to 5,649 tokens (69.0%). Accuracy improves in 12 of the 14 subjects, including business, law, and psychology.

Together, these results support the value of a shared local adviser for frozen students: gains persist under sampling and at 7B, and the budget-controlled evaluation shows higher MATH500 accuracy than ACTS at comparable estimated compute.

While the end-to-end measurements and budget-controlled comparisons address some of my concerns, they also reveal substantially higher computational costs, particularly for the 7B model. Although the 1.5B latency and MATH500 results are encouraging, the limited accuracy gains and inconsistent efficiency improvements across settings leave my overall concerns only partially resolved. I therefore maintain my rating.

Reviewer_J4fZOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Cornerman Decoding, an inference-time method that pairs a frozen distilled reasoning student (DeepSeek-R1-Distill-Qwen 1.5B/7B, further SFT’d on tagged LIMO traces) with a separately trained 1.5B adviser. At each paragraph boundary, the adviser sees only the last ≤800 characters of the latest student paragraph and outputs one of five verdicts (PASS/STEP/ALT/CHECK/END). Each non-PASS verdict is inserted into the student’s context as a fixed first-person reminder, and CHECK also quotes the claim to re-examine; the student is free to follow or ignore it. The adviser is trained by SFT on ~2.4k DeepSeek-annotated student paragraphs. Under single-run greedy decoding on nine benchmarks, the method reports +5.5 pp (1.5B) and +1.0 pp (7B) macro accuracy, with 45% and 21% fewer student tokens. The paper also includes control conditions, comparisons with ACTS/DEER/JET/LC-R1, and adherence and replay analyses.

Critical strengths
  1. The audit of incorrect MATH500 trajectories identifies two concrete failure modes: a correct answer is reached but not submitted in 49.1% of cases, and repetition loops occur in 61.3%. These observations motivate interventions that help students develop, revise, and complete their reasoning.

  2. The separation between a local adviser and a student with full reasoning context is appealing. The limited advice interface encourages structural changes without supplying solution steps or directly controlling termination.

  3. The paper reports frequency-weighted immediate adherence of 58–60%, with END advice followed by immediate answer submission in only 30–37% of audited cases. A selected-prefix replay also finds lower observed accuracy when the first suggestion is retained than when it is removed (51.5% vs. 53.1%). These analyses appropriately distinguish non-binding advice from reliable control, while leaving the mechanism behind the full method's gains unresolved.

Critical weaknesses
  1. The reported token reductions exclude adviser prefill and decoding as well as the overhead of resuming student generation. The 1.5B adviser is consulted at every paragraph (up to 400 segments), and replacing earlier advice edits the middle of the context, which may force the student to re-process later tokens. Fewer student-generated tokens therefore do not imply lower end-to-end latency or total compute, and the efficiency comparison with ACTS and JET (Table 3) is based on student tokens alone.

  2. Table 4 (MATH500, 1.5B, a single greedy run) shows the trained adviser outperforming silent, untrained, and sham controls, but speak rates differ widely (0%, 68%, and 79% for the controls vs. 33.5%), so gains cannot be cleanly attributed to content, timing, or frequency. A control that inserts the real advice messages with shuffled or random verdicts at a matched speak rate is missing, and the sham sentences are unrelated text, which makes it a weak control for content. Moreover, the first-suggestion replay shows no benefit from an isolated suggestion (51.5% vs. 53.1%), and the paper does not identify which interaction patterns (e.g., repeated END, CHECK, or loop breaking) drive the full method’s improvement.

3.Both students belong to the DeepSeek-R1-Distill-Qwen family and are fine-tuned on the same LIMO traces, with paragraph tags from that training used as consultation boundaries, so transfer to independently trained reasoning models is unclear. Apart from CSQA (Appendix B.3), the paper only states disjointness of adviser-training problems from MATH500, and it does not quantify the extent or agreement of the manual review of DeepSeek-generated labels.

Questions for authors
  1. Can you report end-to-end latency, total token consumption (including adviser prefill and decoding), or FLOPs, including student resumption overhead? How does Cornerman compare with ACTS and JET under a matched inference-resource budget?

  2. Are the adviser-training problems disjoint from all benchmarks other than CSQA? What proportion of the 2,367 annotated examples was manually reviewed, and can you report quantitative annotation-quality measures (e.g., agreement rate, held-out adviser accuracy)?

  3. Can you add (a) a matched-speak-rate control with shuffled verdicts, (b) a self-reminder baseline in which the student selects its own structural reminder at paragraph boundaries, and (c) a 7B adviser to test whether the learned policy benefits from greater capacity?

1 comment
AuthorsOpenReview ↗

Response to Reviewer J4fZ

Thank you for the constructive review. We added the following experiments and analyses.

Q1. End-to-end cost and ACTS/JET

We count student and adviser prefill and decoding, including student-context recomputation after advice replacement. Estimated FLOPs use 2 × model parameters × uncached processed tokens, summed over both models, excluding attention operations. Timing uses one A800-80GB with vLLM 0.7.3. Batch time covers all 500 MATH500 problems; latency averages the same 50 questions at concurrency 1.

StudentMethodTFLOPs/problemBatch minLatency s
1.5BFull-CoT29.672.367.8
1.5BCornerman39.126.233.1
7BFull-CoT60.718.548.2
7BCornerman116.925.838.0

Cornerman lowers single-query latency at both scales. Batch runtime falls for 1.5B but rises for 7B; estimated FLOPs increase at both scales.

The paper's standard 32k result is 84.4%. We also tested 8k/16k student budgets, using a common final-answer scorer:

MethodAcc. (%)TFLOPs/problemBatch min
ACTS81.036.3N/A
JET80.818.635.3
Cornerman, 8k83.427.89.9
Cornerman, 16k84.234.817.9

ACTS includes controller costs but was not timed; other runtimes use the GPU above. Cornerman at 16k exceeds ACTS by 3.2 points with 4.1% lower estimated FLOPs. JET uses fewer FLOPs, while Cornerman at 8k has higher accuracy and shorter batch runtime. Unlike JET's reasoner post-training, Cornerman trains a separate adviser shared across frozen students.

Q2. Data separation and annotation

Adviser paragraphs come from the 1.5B Full-CoT student's generations on MATH training problems. Normalized exact matching found zero overlaps between MATH-train or LIMO and all seven mathematics benchmarks.

Of 2,367 paragraphs, 200 were author-labeled and 2,167 DeepSeek API-labeled. Author blindly re-labeled 217 randomly sampled API examples (10%); agreement was 97%, and disagreements were corrected before training. Human labeling/review thus covered 417 examples (17.6%). All labels passed format and CHECK-quotation validation.

Q3(a). Random advice controls

We retained the real templates but randomized interventions, using reference-adviser speaking probabilities per problem or globally (28.8%). Types followed its empirical distribution; CHECK quoted a random paragraph span. We also tested an 80% probability. Controls use the 1.5B student on MATH500, greedy decoding, a 32k budget, and three seeds (mean ± sample SD).

MethodAcc. (%)Student tokens
Cornerman84.44,128
Random, per-problem82.5 ± 0.86,068
Random, global81.1 ± 1.36,126
Random, 80%76.9 ± 1.86,926

The first two controls realized speaking rates of 26.0% and 28.5%. Cornerman exceeds them by 1.9–3.3 points with about 32% fewer student tokens. Raising the random speaking probability to 80% reduces accuracy. The original replay instead tests only the first reminder, with no later advice in either branch.

Q3(b). Self-reminder

The frozen 1.5B student selected verdicts from its latest paragraph using eight demonstrations and constrained five-way decoding, with the same reminders and random CHECK quotations. On MATH500 it achieved 62.0% with 12,588 student tokens. It never chose PASS, and 97.4% of choices were STEP, repeatedly encouraging continuation rather than reproducing Cornerman's selective interventions.

Q3(c). Larger adviser

We trained a 7B adviser on the same data as the 1.5B adviser.

BenchmarkFull-CoTCornerman (1.5B)Cornerman (7B)
1.5B student
Minerva26.132.733.8
AMC2365.065.072.5
AIME2426.733.336.7
AIME2510.023.323.3
OlympiadBench43.948.749.0
7B student
Minerva39.042.639.3
AMC2387.587.587.5
AIME2453.356.756.7
AIME2540.036.743.3
OlympiadBench63.661.762.8

The larger adviser improves four of five benchmarks for the 1.5B student. For 7B, AIME25 and OlympiadBench improve, Minerva decline.

Cross-family transfer

We also explored applying this pipeline to models from other families such as Gemma-3-1B-it. For this model, MATH500 accuracy fell from 43.8% to 2.2% after LIMO SFT, with 474/500 outputs reaching the generation limit. This degradation occurred before adding any adviser, so we did not proceed to adviser evaluation on this model. Evaluating transfer to Gemma therefore requires adapting the Full-CoT student-training setup first.

Reviewer_wxtHOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposed Cornerman Decoding to improve the test-time reasoning capability of LLMs. Specifically, an adviser model is trained to observe the latest student paragraph and either remain silent or suggest continuing, switching approaches, checking a claim, or finishing, without supplying new solution steps or answers. Empirical experiments show that Cornerman Decoding can effectively improve the reasoning capability and also reduce the generated tokens for models of different sizes.

Critical strengths
  1. The proposed method is intuitive and effective. Empirical results show that Cornerman Decoding can improve reasoning capability and reduce reasoning tokens at the same time.

  2. The writing is clear and easy to follow.

Critical weaknesses
  1. The evaluated benchmarks are not extensive enough. Only math reasoning and QA benchmarks are used. It is unclear whether the proposed method can generalize to other tasks, such as code generation. Also, the overhead introduced by the auxiliary adviser model is not discussed.
Questions for authors
  1. For the adviser model, are the output verdicts special tokens or normal text tokens? And will it output the following detailed instructions in addition to the verdicts? Or does it only output the verdicts and then insert hard-coded instructions according to each verdict?

  2. As another simple and effective method for reasoning capability improvement, how is s1 [1] compared with Cornerman decoding?

  3. Although Cornerman Decoding helps reduce the generated tokens, the adviser model also introduces extra computation. How is this overhead measured? Specifically, could the authors provide tokens per second (TPS) as an additional metric?

  4. From Table 1, it seems that the improvement shrinks as the student model scales up. If using a larger adviser model, can the reasoning improvement also be further scaled up?

  5. Do the 1.5B and 7B students share the same adviser model? Or do they have their own adviser model? How is the adviser model trained (more specifically, on which student model's trajectories)?

  6. Could the author provide stats on different inserted advice types during inference?

Reference:

[1] Muennighoff, Niklas, et al. "s1: Simple test-time scaling." EMNLP 2025.

4 comments
AuthorsOpenReview ↗

Response to Reviewer wxtH

Thank you for recognizing the method's effectiveness and the paper's clarity.

Q1. Adviser output

Verdicts are ordinary text, not special tokens. Except for PASS, the adviser generates a verdict and a fixed reminder (Table 1). The decoder validates and inserts the reminder; only CHECK's verbatim quotation from the student paragraph is variable.

Q2. Comparison with s1

We tested s1-style budget forcing on the same 1.5B Full-CoT student on MATH500: 4k/8k thinking caps and two "Wait" continuations.

MethodAcc. (%)Mean student tokens
Full-CoT78.87,822
s1 (4k)79.22,663
s1 (8k)81.63,797
s1 (Wait x2)68.018,982
Cornerman (8k)83.42,693
Cornerman84.44,128

Cornerman reaches 84.4% accuracy. With an 8k student-generation budget, it achieves 83.4%, exceeding s1's 8k thinking-cap result by 1.8 points with 29.1% fewer student-generated tokens.

Q3. End-to-end cost and TPS

New measurements include adviser calls and student-resumption overhead, on one A800-80GB with vLLM 0.7.3.

StudentMethodEst. TFLOPs/problemTime (min)TPSMean latency (s)
1.5BFull-CoT29.672.395167.8
1.5BCornerman39.126.21,26133.1
7BFull-CoT60.718.51,75648.2
7BCornerman116.925.81,13538.0

Time and TPS use the full MATH500 batch; TPS = student-generated tokens / end-to-end time. Mean latency uses the same 50 questions at concurrency 1. FLOPs count both models' uncached prefill and decoding, including student recomputation, at 2P/token (P: model parameters; attention excluded).

For 1.5B, batch runtime falls from 72.3 to 26.2 minutes and single-query latency nearly halves, despite higher estimated FLOPs. For 7B, single-query latency improves, while batch runtime and estimated FLOPs increase.

Q4. Adviser size

We trained a 7B adviser on the same data as the paper's Cornerman adviser.

BenchmarkFull-CoTCornerman (1.5B)Cornerman (7B)
1.5B student
Minerva26.132.733.8
AMC2365.065.072.5
AIME2426.733.336.7
AIME2510.023.323.3
OlympiadBench43.948.749.0
7B student
Minerva39.042.639.3
AMC2387.587.587.5
AIME2453.356.756.7
AIME2540.036.743.3
OlympiadBench63.661.762.8

The larger adviser improves four of five benchmarks for the 1.5B student. For 7B, AIME25 and OlympiadBench improve, but Minerva decline. Adviser scaling thus yields further gains on several benchmarks, particularly for the 1.5B student.

Q5. Shared adviser and training data

Both students share the same frozen 1.5B adviser, obtained by full-parameter SFT of Qwen2.5-1.5B-Instruct on about 2,400 paragraphs. All paragraphs come from the 1.5B Full-CoT student solving MATH problems disjoint from MATH500. The DeepSeek API assisted annotation, with a sample of its labels reviewed by the authors.

Q6. Advice-type statistics

We counted the advice inserted during the greedy evaluations on MATH500., the shared 1.5B adviser inserted 6,228 and 5,594 reminders for the 1.5B and 7B students. Type percentages use all insertions as the denominator; speak rate = insertions / consultations.

StudentSTEPALTCHECKENDSpeak rate
1.5B31.3%33.7%18.4%16.6%30.3%
7B29.6%25.0%13.2%32.2%28.8%

Task generalization

We also tested the unchanged adviser with the 1.5B student on HumanEval+ and subsets of MMLU-Pro and LiveCodeBench, without task-specific training. Scores are accuracy (%) for MMLU-Pro and pass@1 (%) for code.

BenchmarkFull-CoTCornerman
MMLU-Pro22.527.2
HumanEval+54.950.6
LiveCodeBench22.516.5

Cornerman improves accuracy in 12 of 14 MMLU-Pro categories, including business, law and psychology: +4.7 points overall with 69.0% fewer student tokens. Neither code benchmark improves. An explanation is that natural language multi-step reasoning shares more cues with the adviser's training paragraphs, whereas repetition and revision during program construction or debugging can have different meanings.

Thank you for providing detailed responses and clarifications. Could the author further explain why Cornerman reduced the time and increased the TPS for the 1.5B model in the Table of End-to-end cost and TPS?

Thank you for asking. TPS in this table is the total number of student-generated tokens divided by the wall-clock time to finish all 500 MATH500 problems. That time includes adviser calls and resuming student generation.

For the 1.5B student, one notable difference was how often generation ran to the 32,768-token limit: 69 of 500 problems with Full-CoT, versus 7 with Cornerman. Average student output fell from 8,257 to 3,961 tokens per problem, while the adviser generated only 387 tokens per problem on average. Even with the adviser overhead, the batch finished in 26.2 rather than 72.3 minutes. Wall time therefore fell more than student output did (63.8% vs. 52.0%), which is why throughput rose from 951 to 1,261 student tokens/s.

Thank you for the further clarification on TPS numbers. I maintain my positive assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.