acceptodds
Under review as a conference paper at ICLR 2027

SPIRE: Let the Student Choose Its Own Supervision for Implicit Reasoning

Abstract

Implicit chain-of-thought reasoning compresses explicit reasoning traces into a short sequence of latent states. Recent methods enrich latent supervision with teacher key–value (KV) states, but identifying informative teacher states does not determine which states best supervise each student step. We investigate this allocation problem and find that teacher-state utility depends on the receiving student step, while preserving the teacher’s reasoning order improves performance. We introduce SPIRE (Student-Selected Supervision for Implicit Reasoning), a framework in which the student chooses its own supervision. Given a compact pool of teacher states, SPIRE uses the student’s current representations to jointly select an ordered, non-reusing supervision path, assigning a distinct teacher target to each active student step. We solve this selection exactly with dynamic programming and characterize its stability under score noise. SPIRE introduces no additional trainable routing parameters and leaves the latent inference procedure unchanged. Experiments on three arithmetic reasoning benchmarks and three model backbones demonstrate performance gains over implicit CoT baselines.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Implicit Reasoning, Knowledge Distillation, Latent Chain-of-Thought

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_GcF5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces SPIRE, a framework for student-conditioned supervision allocation in implicit chain-of-thought reasoning. Rather than assigning teacher key–value (KV) states solely according to teacher-side importance scores, SPIRE uses the student's attention queries to evaluate the compatibility of candidate teacher states with individual latent reasoning steps. It then employs dynamic programming to select an ordered, non-reusing path of teacher states and uses the selected KV states as training targets. The selection mechanism introduces no additional trainable routing parameters and is removed at inference time. The paper evaluates SPIRE on three arithmetic reasoning benchmarks and three model backbones. The reported results show improvements over existing implicit reasoning baselines, and the ablation studies provide evidence for student-conditioned selection and non-reusing supervision paths.

Critical strengths
  1. A useful and clearly formulated problem. The paper distinguishes selecting informative teacher states from deciding which states are useful for particular student steps. This is a relevant question for latent reasoning, where many teacher states must supervise a much shorter student trajectory.
  2. A lightweight training-time mechanism with consistent reported results. SPIRE reuses existing representations, adds no trainable routing parameters, and does not change inference. The reported average accuracy exceeds the strongest implicit baseline on each of the three backbones.
Critical weaknesses
  1. The matching criterion and its contribution are not sufficiently validated. Section 3.2 measures supervision utility by the change in answer cross-entropy after injecting teacher KV into a student step. SPIRE instead maximizes query–key cosine similarity, but does not directly show that higher similarity predicts greater utility. Attribution is also confounded: Teacher-only uses 12 teacher-selected states, while SPIRE selects 12 from a pool of 18, changing both the candidate pool and the allocation rule.
  2. Performance is highly sensitive to candidate-pool size. Increasing the pool from M=18M=18 to M=24M=24 drops average accuracy from 33.4% to 7.1%. The paper does not explain this sharp decline, leaving the method’s robustness and the behavior of its selection mechanism unclear.
  3. The evaluation scope is narrow. All three benchmarks concern arithmetic reasoning, and GSM8K and GSM-Hard are closely related. The results support the reported settings but do not establish generalization to other reasoning domains or task types; the claims should be scoped accordingly.
Questions for authors
  1. Can the authors show whether the query–key matching score used by SPIRE is associated with the intervention-based utility from Section 3.2? In particular, does selecting higher-scoring candidates tend to reduce answer loss more than selecting other candidates for the same student step?
  2. Figure 6 reports 7.1% average accuracy at candidate-pool size M=24M=24, while M=18M=18 gives 33.4%. Can the authors clarify what causes this drop and whether it reflects training behavior, candidate selection, or a reporting issue?
Reviewer_fbMeOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces SPIRE, a new latent-reasoning method. To provide dense supervision for a latent reasoning trace, the previous method, KaVa, prunes the teacher's chain-of-thought trace to match the length of the student trace. This pruning is independent of the student's latent reasoning trace. SPIRE, on the other hand, selects teacher steps based on how well they match the student. The authors identify three desired properties of the matching: selection should be performed for each student step, selections should not be independent across student steps in order to avoid overlap, and the matching should preserve the order of the original teacher steps.

Specifically, the method first creates a "supervision pool" from the teacher trace using the previous approach. The student then scores candidate teacher states using cosine similarity between student queries and teacher keys. It uses dynamic programming to find an ordered matching between student and teacher steps and applies an MSE loss to the selected KVs.

The experiments train on GSM8K-Aug, use a supervision pool of 18, and evaluate three backbones on GSM8K, GSM-Hard, and SVAMP.

Critical strengths

The paper asks a useful question: how should the supervision signal be allocated across latent steps? The student does not necessarily need to reason in exactly the same way as the teacher, so a fixed allocation may not be optimal.

Critical weaknesses
  1. The Section 3 preliminary study is the main motivation for the method, but I do not find it convincing:

    • For RQ1 and RQ2, the paper uses 12 teacher states and 12 student steps, even though the utility matrix U does not need to be square. It is not clear why the analysis does not use a larger set of teacher states or the full teacher trace.

    • To compute the utility matrix, the paper mixes teacher KV from token j into student KV at token i and measures the change in cross-entropy. I do not understand why it uses a linear combination of the KVs rather than swapping one student step completely.

    • Selecting teacher states by their average utility at the other student steps gives 0.01334 higher cross-entropy than selecting the best state for the receiving student step. It is not clear whether this is a large change without variation or uncertainty for this experiment. The paper also needs to make clearer how the teacher steps are allocated to student steps after this preliminary study.

    • The Spearman correlation result might be misleading. Many student steps could have the same top-k teacher states while having different rankings for all remaining teacher states simply because they are equally bad. This would give a low Spearman correlation but would not show that every student step needs to select a different target.

  2. The experimental evaluation is not thorough enough.

    • The training data are only GSM8K-Aug, which has short, equation-only reasoning traces. I would like to see at least the NL version and, more importantly, a setting with longer traces such as MetaMath; these results would affect my score.

    • I think it is importnat to show how the performance changes when we change the pool size from M = K (which is close to KaVa setting) and gradually increase it to M equal to the full teacher trace, without R-KV scoring. I think this ablation is important, and a longer-trace dataset is essential for it.

Questions for authors

1 Clarification and analysis: Why does the Section 3 study use a 12-by-12 utility matrix and a linear mixture of teacher and student KVs rather than a larger or full teacher-state pool and a complete KV replacement? Could the authors also report uncertainty for the 0.01334 cross-entropy difference and top-k teacher-state overlap across student steps? These results would help determine whether the study actually supports student-specific target selection.

  1. Requested experiment: How does SPIRE perform when trained on GSM8K-Aug-NL and MetaMathQA? I would like to see GSM8K-Aug-NL because it is used by previous latent-reasoning methods [1, 2, 3]. I would also like to see MetaMathQA [4] because it is used by previous methods [3, 5]. Results on these natural-language and longer reasoning traces would affect my assessment.

  2. Requested experiment: How does performance change as the candidate-pool size varies from M = K, which is close to the KaVa [3] setting, to M equal to the full teacher trace? This ablation would show whether the larger supervision pool and the R-KV pool-selection stage are important.

[1] Shen, Zhenyi, et al. "Codi: Compressing chain-of-thought into continuous space via self-distillation." Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

[2] Wu, Haoyi, Zhihao Teng, and Kewei Tu. "Parallel continuous chain-of-thought with jacobi iteration." Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

[3] Kuzina, Anna, Maciej Pióro, and Babak Ehteshami Bejnordi. "Kava: Latent reasoning via compressed KV-cache distillation." International Conference on Learning Representations. Vol. 2026. 2026.

[4] Yu, Longhui, et al. "Metamath: Bootstrap your own mathematical questions for large language models." International Conference on Learning Representations. Vol. 2024. 2024.

[5] Su, DiJia, et al. "Token assorted: Mixing latent and text tokens for improved language model reasoning." arXiv preprint arXiv:2502.03275 (2025).

Reviewer_9pDGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Latent COT uses continuous reasoning tokens rather than discrete tokens. These models are typically trained using a discrete teacher COT model and some form of alignment between the teacher and a student model that generates continuous tokens. This provides supervision for the continuous tokens beyond the cross entropy loss on the final answer.

This paper proposes a way to improve this alignment by computing, for each layer and each KV attention head, the cosine similarity between the queries of latent tokens and the keys of teacher tokens. It then uses dynamic programming to choose a discrete alignment matrix and minimizes the squared L2 loss between the aligned student and teacher keys and values.

Critical strengths
  • The paper studies an important and relevant problem.
  • The proposed method is intuitive.
  • The proposed method does not add additional trainable parameters and only affects training, so it does not increase inference cost.
  • The paper is well written and well motivated. The preliminary experiments help explain why the alignment should depend on the student position and preserve the teacher’s reasoning order.
Critical weaknesses
  • The method builds on an existing KV distillation method and mainly changes how teacher states are selected and assigned as supervision. As such, its methodological contribution seems rather incremental.

  • The benefit of student-dependent selection is not fully established. In the Qwen 0.5B ablation, the improvement over teacher-only selection is marginal. The student-dependent variant selects 12 states from 18 candidates, whereas the teacher-only variant directly uses the top 12. Comparing against other ways of selecting 12 states from the same 18 candidates would help establish the benefit of student-dependent scoring more clearly. Since this is one of the paper’s core contributions, stronger evidence for its benefit is needed.

  • The method shows strong sensitivity to the candidate-pool size. Increasing it from 18 to 24 reduces average accuracy from 33.4% to 7.1% on Qwen 0.5B. This substantial drop is not sufficiently explained.

  • No standard deviations across training seeds are reported. Given the sometimes marginal improvements, I would strongly encourage the authors to report these if computationally feasible.

Questions for authors
  • How would soft alignment compare with hard selection when preserving the same ordering and no-reuse constraints?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.