acceptodds
Under review as a conference paper at ICLR 2027

CoRD: Conjecture-and-Refutation Decoding for Evidence-Gated Overrides in Collaborative Reasoning

Abstract

Token-level collaboration enables an LLM to assist an SLM at selected reasoning states, but existing methods often rely on Teacher preference to rank local candidates. Such preference does not establish whether replacing the SLM's current action improves its own subsequent reasoning. We propose CoRD(Conjecture-and-Refutation Decoding), which reformulates local collaboration as intervention verification. At each triggered state, CoRD forms an incumbent–challenger pair, rolls out short candidate-specific Student futures, and asks an LLM verifier to determine whether the incumbent exposes a reasoning error and the challenger avoids it. An override is made only under this asymmetric evidence, followed by a short verified suffix to stabilize the correction. Across six mathematical reasoning benchmarks and two Teacher scales, CoRD outperforms Teacher takeover and S2T in 11 of 12 settings; with a 32B Teacher it raises Math Avg. from 24.81% to 40.50% over Student-only decoding while keeping generation under Student control. Frozen-state counterfactual analysis further shows that accepted interventions achieve 27.27 percentage points of downstream utility, compared with only 1.20 percentage points for rejected interventions under a 16-token verified suffix. Suffix commitment also eliminates the observed token-only damage. These results support moving token-level collaboration from candidate preference toward evidence-gated correction.

Primary area:
foundation or frontier models, including LLMs
Keywords:
collaborative decoding, LLM reasoning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_VcwMOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes CoRD, an inference-time collaboration method where a large teacher model helps a small student model correct reasoning mistakes during generation. It evaluates different intervention candidates through the future trajectories induced under the student’s own reasoning dynamics. Moreover, it implements a temporally structured counterfactual window and an asymmetric verification rule to improve the model's reasoning performance.

Critical strengths
  1. The average performance on math is promising. With a 32B teacher and a 7B teacher, CoRD achieves evidently higher average accuracy compared with baselines.
  2. The ablation studies and analysis are useful. The paper examines the effectiveness of different components, such as historical reconstruction, future rollouts, and asymmetric verification. They are helpful to find out whether accepted interventions actually improve downstream answer correctness.
  3. This method is inference-based, so it doesn't need any training weights and is more cost-effective.
Critical weaknesses
  1. Why specifically choose Qwen2.5 series models, rather than the latest Qwen3 models? Is this because of computing resources, cost, or because the latest models have already saturated the benchmarks you test?
  2. The evidence for the core contribution is not very sufficient. In Table 2, removing student-conditioned counterfactual rollout decreases accuracy by only 0.28%. Since counterfactual future verification is the central contribution, its actual benefit remains unclear.
  3. CoRD uses a fixed rewind window of 8 tokens without identifying the actual error location. Although Table 3 measures results with different R, this heuristic may be questionable when reasoning errors occur much earlier than the detected disagreement. Or, for different reasoning trails, the R value should be dynamic rather than fixed.
  4. It would be better to see the paper include some case studies and discussions of generated trajectories.
Questions for authors
  1. If the Collaborative (Student + Teacher) sometimes can't surpass the Teacher-only performance, why do we still want the collaborative mode? In this way, the Teacher-only may have a lower computing cost compared to using both student and teacher, but with better performance.
  2. Table 6 seems to appear among the references.
  3. The paper uses some words, like "incumbent" and "conjecture-and-refutation", to denote specific steps. This makes it hard for readers to follow at first.
4 comments

We sincerely thank the reviewer for the constructive feedback and for recognizing the potential of our approach. Your comments have helped us clarify the model selection, disentangle the component analysis, and make the correction process more accessible.

Part1/4-W1: Why use Qwen2.5 rather than newer models such as Qwen3?

We initially selected Qwen2.5 to follow the model family used by S2T, allowing us to study collaborative correction using comparable Student–Teacher configurations. The 1.5B Student paired with 7B and 32B Teachers also supports examining two Teacher scales within the same family. The motivation was experimental comparability, rather than an assumption that Qwen3 had saturated the benchmarks.

The underlying question remains relevant for newer models: a Teacher-preferred local choice may not be effectively developed by the Student into a correct continuation. We therefore extended the evaluation to a Qwen3-1.7B Student paired with Qwen3-8B and Qwen3-32B Teachers.

BenchmarkStudent / TeacherStudent-onlyTeacher-onlyTakeoverS2TCoRD
MATHQwen3-1.7B / Qwen3-32B68.9376.7978.9377.1476.79
MATHQwen3-1.7B / Qwen3-8B68.9376.0774.6474.6476.07
HumanEvalQwen3-1.7B / Qwen3-32B60.9887.8078.6676.8376.22
HumanEvalQwen3-1.7B / Qwen3-8B60.9882.3269.5168.2967.68

MATH covers 280 problems with a maximum of 1,024 newly generated tokens. HumanEval covers all 164 problems with a maximum of 512. Both use non-thinking mode and greedy live decoding. Qwen3 MATH CoRD uses seed 3407, while the MATH baselines use seed 42, so those columns do not share one seed. HumanEval is execution-based Pass@1 at seed 42. The Qwen3 MATH CoRD and S2T cells match Table 20 MeanAcc₃. The other Qwen3 cells are the additional measurements named above.

On MATH, CoRD is 1.43 above S2T with the 8B Teacher (76.07 versus 74.64) and 0.35 below S2T with the 32B Teacher (76.79 versus 77.14). On HumanEval Pass@1, CoRD exceeds Student-only by 15.24 with the 32B Teacher (76.22 versus 60.98, 125 versus 100 of 164) and by 6.70 with the 8B Teacher (67.68 versus 60.98). With the 32B Teacher it passes one fewer HumanEval problem than S2T (125 versus 126). These experiments extend the study to newer models and executable code generation, with the relative benefit varying across configurations.

Part2/4-W2: Why does removing counterfactual rollout produce only a small accuracy change?

This observation motivates separating future evidence, continuation generation, and suffix commitment. Counterfactual future evaluation and the asymmetric verifier gate are operationally coupled: Student-generated continuations provide the evidence on which the gate decides whether to replace the incumbent. Removing rollout can therefore alter both verifier input and the continuation subsequently committed.

CoRD can modify the reasoning trajectory even when no challenger is accepted. The retained incumbent continues from the reconstructed prefix, and its already-checked continuation can also be committed. Therefore, the number of accepted challenger replacements captures only part of the intervention process.

We conducted matched comparisons on the same 280 MATH problems using seeds 42, 123, and 3407. The metric is the average cumulative success rate across Pass@1, Pass@2, and Pass@3, evaluated in this fixed seed order.

ConfigurationMeanAcc₃
S2T single-token selection79.29
Historical reconstruction only78.21
+ Incumbent multi-token suffix commitment79.52
+ Challenger replacement, future evidence hidden79.76
+ Future-guided verification (Full CoRD)80.48

Adding incumbent suffix commitment improves performance by 1.31 percentage points. With the full continuation generation and commitment procedure retained, providing future evidence to the verifier yields a further 0.72-percentage-point improvement over hiding future evidence. In addition, full CoRD improves over historical reconstruction with incumbent suffix commitment but without challenger acceptance by 0.96 percentage points.

Counterfactual future evaluation and asymmetric gating are operationally coupled: the continuation provides the evidence needed by the gate to make its decision. Removing rollouts may change both the evidence available during candidate selection and the tokens subsequently committed. The hidden-future-evidence control separates these effects by retaining generation and commitment while changing only the verifier's access to future information.

These results further illustrate the roles of different mechanisms in terms of end-to-end accuracy: historical reconstruction revises the context, incumbent suffix commitment preserves already-checked continuations, and future evidence provides additional information for candidate evaluation. Full CoRD achieves 80.48, outperforming the controlled configurations.

The complete execution logs from the original three-seed runs further record the frequency of individual operations. These statistics are computed from the full tokenwise CoRD executions on 280 MATH problems with seeds 42, 123, and 3407, totaling 840 problem runs. They are separate from the frozen diagnostic cohort in Appendix D, which contains 466 comparisons across MATH and GSM8K, including 28 accepted comparisons.

EventCount across 840 problem runs
KL-triggered events5,204
Events with eligible candidate comparisons500
Candidate comparisons508
Comparisons accepting a challenger38
Events with actual historical modifications110
Historical modifications without challenger acceptance102
Events committing a window without accepting a challenger462

Among the 110 events involving actual historical modifications, 102 did not accept a challenger. This directly explains why counting only accepted candidate replacements cannot capture all changes introduced by the algorithm. Even when the incumbent is retained, historical reconstruction and continuation commitment can still modify the trajectory.

Together, these analyses distinguish the roles of three operations: historical reconstruction changes the context, commitment preserves already-checked continuations, and future evidence informs candidate replacement. Event statistics characterize how frequently these operations occur, while controlled comparisons quantify their end-to-end effects. The 80.48 result above corresponds to the specified three-seed supplementary experiment and is consistent with the corresponding main-table result.

Part3/4-W3: Why use a fixed rewind window of eight tokens rather than dynamically locating earlier errors?

The fixed window provides a bounded reconstruction budget that balances historical error coverage against the amount of context examined. Section 5.5.2 and Appendix D.3 study this trade-off through window-size experiments and the localization analysis in Figure 3.

Rewind length RCoverage of localized errorsMean verification-window length (tokens)
420.7311.90
856.4315.90
1262.6019.90
1670.1023.90

Increasing R from 4 to 8 raises coverage by 35.7 percentage points. Increasing R from 8 to 16 adds 13.7 percentage points while adding another eight tokens to the mean verification-window length. The lower average coverage gain per additional token beyond R = 8 supports it as a practical operating point balancing local coverage and verification input size.

The end-to-end window analysis gives a complementary view:

RFMATH performance
4480.48
8880.48
161678.81
242477.86

Larger windows examine a broader range but do not consistently improve final-answer performance. Together, these analyses motivate bounded reconstruction: it covers a substantial portion of the localized errors while limiting additional generation and verification work. Coverage measures whether a localized error falls within the examined range, whereas end-to-end accuracy measures the outcome of the full procedure.

Earlier errors can remain outside a fixed window. As future work, we plan to investigate adaptive rewind lengths based on online uncertainty or verifier signals and evaluate their coverage and final-answer performance under comparable computational budgets. Thank you again for this helpful suggestion.

W4: Can the paper include concrete examples of generated trajectories?

We have added two real trajectory cases to Appendix D.5, "Successful Local Corrections in Generated Trajectories." Both come from recorded GSM8K runs of tokenwise CoRD with a Qwen2.5-1.5B Student and a Qwen2.5-32B Teacher.

The cases show the problem, context before comparison, incumbent and challenger, candidate-conditioned continuations, verifier decision, committed window, and completed answer. Readers can therefore inspect the local reasoning error exposed by a continuation, the selected alternative, and how generation proceeds to the correct answer.

These examples complement the aggregate evaluation by making the interaction between future evidence, candidate selection, and commitment visible in actual generated trajectories.

Part4/4-Q1: If the Collaborative mode sometimes can't surpass Teacher-only, why not just use the Teacher with lower cost and better performance?

Thank you for raising this question. We agree that Teacher-only is preferable when it is available for every query and offers both higher accuracy and lower cost. In our co-located experiment, Teacher-only has lower latency and peak memory than CoRD.

1. When collaboration is useful. Our setting assumes that the Student handles routine generation and that access to the Teacher is limited. We use a calibrated KL-divergence trigger to selectively activate Teacher assistance. Specifically, the Student and Teacher distributions are normalized over the shared Student top-64 support, and collaboration is triggered when their KL divergence exceeds a threshold calibrated on held-out examples. This threshold is frozen before evaluation. The Student therefore generates independently between interventions and queries the Teacher only when assistance is triggered. This gives a concrete setting in which to ask how much selective Teacher guidance can improve Student-generated answers.

2. What the cost measurements show. We report inference measurements on 280 MATH problems (seed 42, NVIDIA H200). Both models are co-located on the same GPU in this research prototype. In a production architecture, the Student runs locally at high throughput and consults the Teacher as a shared remote service, so the per-request footprint is dominated by the 1.5B model alone.

MethodMean time per problemMean peak GPU memory
Teacher-only14.90 s61.22 GiB
S2T20.30 s64.12 GiB
CoRD20.70 s64.25 GiB

Two key observations:

  • CoRD vs. S2T: CoRD's correction mechanism adds only 0.40 s and 0.13 GiB over the existing collaboration baseline. The paired accuracy on this seed-42 timing run is 77.14 for S2T and 80.83 for CoRD, a single-run gain of 3.69 pp. That run is separate from Table 1 MeanAcc₃, where S2T is 79.29 and CoRD is 80.48.
  • Why Teacher-only appears "cheaper": In this co-located setup, Teacher-only loads a single 32B model and thus reports lower absolute latency and memory. However, this conflates single-request latency with system-level serving cost. At deployment scale, the 32B model's per-GPU throughput is far lower than a 1.5B model's, making Teacher-only the most expensive option per query served—not the cheapest. The collaborative mode keeps the lightweight Student as the workhorse and invokes the Teacher only for targeted guidance, which is a fundamentally more scalable architecture.

3. What CoRD contributes. Our central objective is to investigate how Teacher guidance can better align with the Student's own reasoning process, rather than simply adopting the Teacher's preferred next token. Because the two models differ in capacity and continuation distributions, a locally Teacher-preferred token may not lead to a correct trajectory when the Student continues from it. CoRD addresses this through reconstruction (accounting for the preceding context) and candidate-conditioned continuations with asymmetric verifier gating (accounting for future consequences). The cases where collaboration does not surpass Teacher-only reflect the inherent capacity gap between a 1.5B and a 32B model. The contribution of CoRD is substantially narrowing that gap while retaining the Student's deployment advantages. Speculative decoding addresses a complementary objective: accelerating generation while preserving the Teacher's target distribution, and operates on different assumptions.

Q2: Why does Table 6 appear among the references?

Thank you for pointing this out. We have moved Table 6 to the supplementary material so that it no longer interrupts the reference list.

Q3: Can the terminology be made easier to understand?

We appreciate this suggestion. We will explain the terms through the operations they describe at their first relevant occurrence and use them consistently throughout the paper.

The incumbent is the candidate currently selected by the collaboration procedure. A challenger is an alternative candidate. In CoRD, the initial incumbent is the Teacher-preferred token within the Student's top-K candidate set.

A conjecture takes the form of a candidate-conditioned Student continuation, exposing how the Student would develop that local choice. Refutation is the Teacher's examination of the resulting local trajectory for an explicit reasoning error. Together, these operations connect candidate selection to its downstream consequences in the Student's reasoning.

We introduce this mechanism-based explanation in the Introduction and retain the precise definitions in the problem formulation and method sections.

We thank the reviewer again for the thoughtful feedback, which has helped us sharpen the presentation of CoRD's motivation, mechanisms, and empirical scope.

Reviewer_yzY7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CoRD combines historical reconstruction, student-conditioned lookahead, asymmetric verification, and suffix commitment for small-large model collaboration. It asks a useful question: whether a teacher-preferred intervention actually helps the student. The reported gains are promising, but important inconsistencies and validation gaps weaken the conclusions.

Critical strengths

S1. The research question is meaningful. Distinguishing teacher preference from downstream student utility offers a useful perspective on collaborative reasoning.

S2. The proposed pipeline is explicit and interpretable, with separate mechanisms for reconstructing context, comparing alternatives, and preserving selected continuations.

S3. The evaluation includes two teacher scales, mathematical benchmarks, and frozen-state diagnostics. The appendix also acknowledges verifier uncertainty and limitations of its diagnostic implementation.

Critical weaknesses

W1. The main results are not clearly reconciled with the completed experimental records. Table 1 reports 80.48% MATH accuracy, whereas Appendix B lists two completed tokenwise configurations at 74.52%. Figure 3 gives 56.4% error coverage at R=8, while Table 20 and the accompanying text give 42.26%. Such discrepancies should be reconciled before drawing conclusions about the temporal window. Their relationship and the provenance of the main results and ablations require clarification.

W2. The conservative gate does not protect every modification made by the algorithm. Its incumbent is teacher-preferred rather than necessarily the original student token, and retaining it does not undo historical edits. Thus, the gate bounds challenger acceptance, not the safety of the full correction procedure.

Questions for authors

See Weaknesses

1 comment

We sincerely thank the reviewer for the careful comparison of the reported figures and for the precise reading of the asymmetric gate. These comments helped us separate results that come from different configurations, correct the coverage discussion, and state explicitly what the gate does and does not guarantee. We address each concern below.

W1: Provenance and consistency of reported accuracy and coverage figures

Thank you for pointing out these inconsistencies. We agree that the relationship between the main results, the completed experimental records, and the localization analysis was not sufficiently clear. We have revised the relevant sections so that each reported number is tied to its configuration.

MATH accuracy. The 80.48 in Table 1 is the final default CoRD configuration, R = 8, F = 8, and C = 16. In the previous version, two 74.52 results were reported for completed tokenwise runs with different windows, R = 7, F = 1 and R = 11, F = 5. Those runs are not the default R = 8, F = 8 configuration. The revised Appendix B no longer contains those two entries.

Localization coverage. We re-examined the coverage results and corrected the accompanying discussion. Figure 3 shows that coverage increases from 20.73 at R = 4 to 56.43 at R = 8, then by 6.17 percentage points at R = 12 and 7.48 percentage points at R = 16. The previous Table 24 contained inconsistent coverage values and has been removed. The revised discussion refers directly to Figure 3 and describes rewind length, historical-error coverage, and computational cost together.

What the two measurements are. End-to-end accuracy evaluates final-answer performance. The localization analysis measures coverage of Teacher-localized historical errors in a separate diagnostic replay. We now state this distinction so that localization coverage is not read as end-to-end accuracy, and so that R = 8 is not treated as a universally optimal rewind length.

W2: Scope of the asymmetric gate's guarantee

Thank you for this precise observation. We agree: the gate bounds challenger acceptance, not the safety of the full correction procedure. We have revised the manuscript so that this scope is explicit.

What the gate controls. At each correction step, the asymmetric gate decides whether to replace the current incumbent with a challenger, given a shared reconstructed prefix. The incumbent is the Teacher-preferred token inside the Student's top-K set, not the original Student proposal. Equation (13) bounds the conditional probability of accepting an incorrect challenger when the incumbent is correct. This is a local, one-step guarantee relative to that incumbent, not an end-to-end safety claim.

What the gate does not control. Two operations lie outside the gate. Historical reconstruction rewrites the prefix before the gate is invoked. Suffix commitment then appends inspected tokens, subject to the commitment cap and the output budget. Retaining the incumbent keeps the Teacher-preferred choice at that step, but it does not undo edits made during reconstruction. Section 4.3 and Proposition 4.1 now state these boundaries, with further discussion in Appendix A.2 and Appendix F.2.

Why the bounded guarantee remains useful. The gate is the only decision point at which a verified incumbent can be replaced by a challenger. Equation (13) bounds this transition without an independence assumption. The product form requires the stated conditional-independence assumption, and Equation (15) gives a positive expected gain only under its stated inequality. Reconstruction and commitment are governed separately by the rewind length R, the forward horizon F, and the commitment cap C. Their effects are reported in the ablation, not by the gate's bound.

These revisions align the formal statement with the algorithm. The gate is one component of the correction pipeline, not a blanket safety guarantee. We thank the reviewer again for these comments. We would be glad to know whether the responses above address the concerns, and we are happy to clarify any remaining questions.

Reviewer_8a4ROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies token-level collaboration between a small student model and a large teacher model for math reasoning. When the two models disagree, CoRD rewinds a few tokens, lets the teacher repair this short history, and then lets the student continue briefly from each candidate token. The teacher checks whether each short continuation contains an error, and the current token is replaced only when its continuation has an error and the alternative's does not. Experiments with a Qwen2.5-1.5B student and 7B/32B teachers on six math benchmarks show higher accuracy than teacher takeover and S2T.

Critical strengths
  1. The main idea is to check a local intervention through the student's own short continuation, instead of trusting the teacher's local preference. This student-aware, local view seems important for small-large collaboration, although related insights appear in prior work. The design of when to intervene and how to replace a token is also interesting. Replacing a token only when the current choice is judged wrong and the alternative is judged fine is a simple and conservative way to avoid harmful changes.

  2. CoRD improves over the collaborative baselines in most settings. With the 32B teacher, it raises the average accuracy from 24.81% to 40.50% over student-only decoding.

Critical weaknesses
  1. My main concern is why one would not simply use the teacher's answer. CoRD does not exceed teacher-only decoding on any benchmark. The KL trigger also needs the teacher's distribution at every step, so the teacher already runs on every token. If speed is the reason to keep the student, speculative decoding is a natural alternative: the student drafts tokens, the teacher checks them in one pass, and the output matches teacher-only quality while running faster. The paper should explain what is gained by keeping the student in control, and compare cost and accuracy with teacher-only and speculative decoding.

  2. It is unclear whether this per-query overhead is better than training. CoRD adds teacher calls, student rollouts, and verification to every query, and its trigger must be recalibrated for each benchmark. Distillation pays its cost only once. A comparison with a distilled student under a similar total compute budget would show whether inference-time correction is worth this cost.

Questions for authors
  1. How much of the gain over S2T comes from the accepted overrides? At each trigger, CoRD starts from the same token that S2T would choose, so it can only differ from S2T when a challenger is accepted. In the diagnostic runs, only a small fraction of triggers lead to an accepted override, and very few of those turn a wrong answer into a correct one. Yet CoRD solves noticeably more problems than S2T, and removing the counterfactual rollout barely changes accuracy. Please break down the gain into overrides, historical reconstruction, and suffix commitment, and report how often each happens in the final method.

  2. How were R and F chosen? The window sizes in Tables 3 and 4 seem to be compared on the same MATH problems used in Table 1, and the default is the best setting there. The 20 held-out examples only set the KL threshold. If the window was selected on the test set, results with a separate validation split are needed.

  3. Please report variance for the end-to-end results. The per-seed results in the appendix suggest that accuracy varies by several points across seeds, and most reported differences are within this range.

  4. Only one student model is evaluated. On the teacher side, the ablations and analyses appear to use only the 32B teacher, even though the teacher also acts as the verifier. All models are from Qwen2.5, and the method requires a shared vocabulary. Results with other student sizes, a weaker verifier, and a second model family, as S2T did with Llama, would show how general the method is.

  5. All six benchmarks are math. Judging whether a short continuation contains an error may be easier for math than for other tasks. Results on code or general reasoning, as in S2T, would show whether the approach extends beyond math.

5 comments

We sincerely thank the reviewer for the detailed and constructive comments. Your suggestions have helped us clarify the motivation behind Student-controlled collaboration, strengthen the analysis of individual mechanisms, and extend our evaluation to additional model scales, model families, and tasks. We address each concern in detail below.

Part1/5-W1: Why preserve Student control? What are the benefits and costs of collaboration?

Thank you for raising this important question. We address it from three perspectives: the deployment baseline, the cost structure, and the research motivation.

1. When collaboration is useful. Our setting assumes that the Student handles routine generation and that access to the Teacher is limited. We use a calibrated KL-divergence trigger to selectively activate Teacher assistance. Specifically, the Student and Teacher distributions are normalized over the shared Student top-64 support, and collaboration is triggered when their KL divergence exceeds a threshold calibrated on held-out examples. The threshold is frozen before evaluation. With this trigger, the Student generates independently between interventions and queries the Teacher when assistance is requested. This gives a concrete setting in which to ask how much selective Teacher guidance can improve Student-generated answers. In real-world serving scenarios with latency, throughput, or device-memory constraints, the primary generator must be the smaller model. A 1.5B Student can serve orders-of-magnitude more concurrent requests per GPU than a 32B Teacher. The practical question is therefore: given that the Student is the deployed model, how much can Teacher guidance improve its output? On MATH, Table 1 reports MeanAcc₃ of 51.67 for Student-only and 80.48 for CoRD, a gain of 28.81 pp. Relative to S2T at 79.29, the gain is 1.19 pp. This is the core value of collaboration, not replacing the Teacher, but elevating the Student.

2. What the cost measurements show. We report inference measurements on 280 MATH problems (seed 42, NVIDIA H200). Both models are co-located on the same GPU in this research prototype. In a production architecture, the Student runs locally at high throughput and consults the Teacher as a shared remote service, so the per-request footprint is dominated by the 1.5B model alone.

MethodMean time per problemMean peak GPU memory
Teacher-only14.90 s61.22 GiB
S2T20.30 s64.12 GiB
CoRD20.70 s64.25 GiB

Two key observations:

  • CoRD vs. S2T: CoRD vs. S2T: CoRD's correction mechanism adds only 0.40 s and 0.13 GiB over the existing collaboration baseline. The paired accuracy on this seed-42 timing run is 74.64 for S2T and 75.71 for CoRD, a single-run gain of 1.07. That run is separate from Table 1 MeanAcc₃, where S2T is 79.29 and CoRD is 80.48.
  • Why Teacher-only appears "cheaper": In this co-located setup, Teacher-only loads a single 32B model and thus reports lower absolute latency and memory. However, this conflates single-request latency with system-level serving cost. At deployment scale, the 32B model's per-GPU throughput is far lower than a 1.5B model's, making Teacher-only the most expensive option per query served—not the cheapest. The collaborative mode keeps the lightweight Student as the workhorse and invokes the Teacher only for targeted guidance, which is a fundamentally more scalable architecture.

3. What CoRD contributes. Our central objective is to investigate how Teacher guidance can better align with the Student's own reasoning process, rather than simply adopting the Teacher's preferred next token. Because the two models differ in capacity and continuation distributions, a locally Teacher-preferred token may not lead to a correct trajectory when the Student continues from it. CoRD addresses this through reconstruction (accounting for the preceding context) and candidate-conditioned continuations with asymmetric verifier gating (accounting for future consequences). The cases where collaboration does not surpass Teacher-only reflect the inherent capacity gap between a 1.5B and a 32B model. The contribution of CoRD is substantially narrowing that gap while retaining the Student's deployment advantages. Speculative decoding addresses a complementary objective: accelerating generation while preserving the Teacher's target distribution, and operates on different assumptions.

Part2/5-W2: Why use inference-time correction rather than distillation?

Distillation and CoRD exploit Teacher knowledge at different stages. Distillation transfers Teacher knowledge into the Student's parameters through offline supervision and parameter updates. It produces a permanently improved Student but requires curating supervision data, running a training pipeline, and repeating this process when models or tasks change. In contrast, CoRD instead uses evidence from the current query to guide localized correction while keeping model parameters unchanged. This distinction is particularly relevant when the Teacher's preferred choice is not well aligned with the Student's continuation capability. CoRD examines reconstructed contexts and candidate-conditioned Student trajectories, allowing guidance to respond to the local reasoning consequences of a particular decision.

We quantify the computational investment at different stages using our S2T-Local reproduction and complete CoRD execution logs:

MeasurementScopeRecorded time
S2T-Local supervision data collection2,000 training trajectories and 114,224 candidate-supervision pairs13.16 hours
Complete CoRD inference280 MATH problems, seed 4220.70 s per problem on average

Both measurements were obtained on an NVIDIA H200 GPU. The supervision data collection time excludes subsequent selector training, trigger-feature collection, and trigger training. CoRD inference time includes prompt processing, prefill, online scoring, historical reconstruction, future continuation, verification, and output decoding, but excludes model loading, offline calibration, and answer scoring.

These measurements quantify preparation-stage and inference-stage costs, but do not constitute a complete quality comparison against a distilled Student under an identical budget. CoRD requires neither additional supervised training data nor parameter optimization for collaboration, and can directly reuse existing Student and Teacher checkpoints.

Q1: How much of the improvement comes from candidate replacement, historical reconstruction, and suffix commitment?

Thank you for asking us to quantify these effects. On the reported Table 4 entries, Full CoRD is 80.48. That is 1.19 above S2T single-token selection at 79.29. From incumbent suffix commitment at 79.52, Full CoRD is a further 0.96 higher. These are differences of the displayed entries. The ablation uses the same 280 MATH problems across three seeds. We report the mean of cumulative Pass@1, Pass@2, and Pass@3 in a fixed seed order, where cumulative Pass@k is the fraction of problems solved at least once in the first k runs.

ConfigurationMeanAcc₃
S2T single-token selection79.29
Historical reconstruction only78.21
+ Incumbent multi-token suffix commitment79.52
+ Challenger replacement, future evidence hidden79.76
+ Future-guided verification (Full CoRD)80.48

The same table places the hidden-future row at 79.76, which is 0.72 below Full CoRD at 80.48. That gap is the displayed difference when future evidence is withheld from the verifier. Removing rollouts can also change the continuations available to commit. Rejecting a challenger retains the incumbent token at the current position. An edit to the preceding context can still be retained, so the same token as S2T can follow a different prefix and lead to a different continuation. CoRD can also commit the checked continuation of the incumbent branch. Whether commitment changes which tokens appear in that window depends on what stepwise decoding would have produced there.

These statistics are computed from the full tokenwise CoRD executions on 280 MATH problems with seeds 42, 123, and 3407, totaling 840 problem runs. They are separate from the frozen diagnostic cohort in Appendix D, which contains 466 comparisons across MATH and GSM8K, including 28 accepted comparisons. These runs contain 5,204 KL-triggered events. Of these, 500 events reached an eligible candidate comparison, yielding 508 comparisons, and 38 comparisons accepted a challenger. Actual historical modifications occurred in 110 events, and 102 of those 110 did not accept a challenger. Separately, 462 events committed a window without accepting a challenger.

Part3/5-Q2: Were R and F selected using the evaluation set?

We distinguish two separate calibration processes: KL-threshold calibration and window configuration analysis.

KL thresholds are determined using 20 held-out calibration problems per dataset and frozen before evaluation, with no further adjustment. Window sizes R and F were analyzed through the end-to-end and coverage results shown in Tables 5–6 in Section 5.5.2 and Figure 3 in Appendix D.3. We acknowledge that this analysis was conducted on the same MATH evaluation problems used in Table 1.

We select R = 8 and F = 8 by jointly considering three factors: (1) end-to-end performance, both R = 4, F = 4 and R = 8, F = 8 achieve 80.48, while larger balanced windows degrade. (2) historical coverage, increasing R from 4 to 8 raises Teacher-localized error coverage from 20.73 to 56.43, with diminishing per-token gains beyond R = 8 (Figure 3). (3) verification cost, the mean verification window increases with R.

Cross-dataset consistency. Importantly, R = 8 and F = 8 is used uniformly across all evaluation datasets (MATH, GSM8K, OlympiadBench, AIME) without per-dataset adjustment. Under this fixed configuration, CoRD improves over Student-only by 25.10 pp on GSM8K (93.94) and by 25.79 pp on OlympiadBench (44.14) with the 32B Teacher. With the 7B Teacher, it achieves a Math Avg. of 37.50, above both Takeover and S2T at 32.40. This cross-dataset stability provides indirect evidence that the selected configuration is not overfit to a single evaluation set.

Q3: How much do end-to-end results vary across random seeds?

We report complete per-seed results on MATH, GSM8K, and OlympiadBench. Configuration is identical across seeds. Only the base random seed varies.

DatasetSeed 42Seed 123Seed 3407Mean ± Std
MATH75.7172.8673.5774.05 ± 1.49 pp
GSM8K91.9991.5390.3891.30 ± 0.83 pp
OlympiadBench36.8538.2334.8636.65 ± 1.69 pp

Sample standard deviations range from 0.83 to 1.69 pp, consistent with typical seed-level variability in autoregressive generation-based evaluation.

To address the concern that reported improvements may fall within this variance range, we compare CoRD and S2T on the same seeds:

DatasetSeedS2TCoRDΔ
MATH4271.0775.71+4.64
MATH12370.3672.86+2.50
MATH340771.4373.57+2.14

CoRD outperforms S2T in every individual seed, with per-seed margins ranging from +2.14 to +4.64 pp, all exceeding the observed standard deviation of 1.49 pp. This indicates a consistent improvement rather than one driven by a single favorable run.

Comments on the paper

1 comment

Dear Reviewers,

We are sincerely grateful for the time and care you devoted to our submission, and for the thoughtful comments that helped us clarify the paper. We have posted a point-by-point response to each review and uploaded the corresponding revised paper.

The author–reviewer discussion closes on November 18, 2026 (Anywhere on Earth). If anything in the responses or the revision is still unclear, we would be very glad to clarify it before the discussion ends.

We thank you again for your constructive comments and we remain very grateful for your time.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.