acceptodds
Under review as a conference paper at ICLR 2027

SUPERVISION MECHANISMS IN IMPLICIT CHAIN-OF-THOUGHT: TARGETS, ALLOCATION, AND SPACE

Abstract

We study supervision mechanisms in implicit chain-of-thought models along three dimensions: the quantities to align (Targets), the updates to supervise directly (Allocation), and the dimension and directions of the alignment space (Space).Mathematical analysis reveals how different supervision targets organize reasoning errors: local update targets constrain discrepancies in individual state changes,whereas cumulative update targets aggregate these discrepancies across steps.Controlled comparisons in a shared recurrent student show that, with local update targets, extending direct supervision to intermediate updates in the answer subspace further improves reasoning accuracy. The choice of supervision space also matters: the answer subspace outperforms full-space alignment and rank-matched random subspaces, highlighting the role of direction selection beyond dimensionality reduction. Based on these findings, we introduce Answer-guided Local Update Supervision (ALUS), which aligns teacher and student changes between consecutive hidden states at every valid update. Alignment uses a fixed subspace derived from output-weight rows for training-answer tokens and adds no inference-time modules.ALUS reaches 41.17 ± 0.20% on GPT-2/GSM8K,8.31 percentage points above no alignment and 4.88 above the strongest adapted supervision objective within the shared student. The benefits of all-valid supervision and the answer subspace persist with six updates; experiments with Pythia-1B and nonnumeric ProsQA provide further evidence for the effectiveness of ALUS.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Implicit Chain-of-Thought, Latent Reasoning, Supervision Mechanisms, Knowledge Distillation, Representation Alignment

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_H4rVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies implicit chain-of-thought distillation through supervision targets, positions, and alignment spaces. It proposes ALUS, which matches teacher and student local state updates at all valid positions in a fixed low-dimensional subspace constructed from training-answer token output weights. The paper analyzes error aggregation and fixed linear readouts, and reports 41.17% accuracy on GPT-2/GSM8K, an 8.31-point gain over no alignment, with additional evaluations on Pythia-1B, ProsQA, and numerical transfer tasks.

Critical strengths
  1. The comparisons keep the student architecture fixed, preserve per-example aggregation weights, and calibrate initial alignment-gradient scales. Rank-matched random, teacher-PCA, and vocabulary-source controls help isolate the effect of supervision-space selection.
  2. ALUS improves accuracy across three seeds, with additional evidence from another backbone and a nonnumeric task. Loss ablations support the contribution of alignment beyond reference-text training, and the method adds no inference-time module.
Critical weaknesses
  1. The subspace is constructed from training-answer tokens, but their coverage of test answers and performance at different coverage levels are not reported. How to choose the subspace for other tasks, how that choice affects generalization, and how a rank-matched full-vocabulary basis performs remain untested.
  2. The K=3/6 comparison changes both student computation and teacher segmentation, without analyzing performance by difficulty. It remains unclear how K should scale with reasoning complexity or whether increasing K helps harder problems.
  3. With K=6, Local does not outperform State/Cumulative; on ProsQA, the answer subspace outperforms the random subspace in only one of three seeds. The conditions under which these choices help remain unexplained.
Questions for authors

Please address the concerns raised in W1–W3 above.

2 comments
AuthorsOpenReview ↗

Response to Reviewer H4rV (1/2)

Response to W1

Thank you for your question. With space-prefixed GPT-2 tokenization, our GSM8K coverage analysis found all 1,462 test-answer token occurrences covered across all 1,319 questions at full retention. In our rebuttal experiments, retaining even half the vocabulary beats the non-answer control in every seed. The rank-32 basis built from all GPT-2 tokens scores 35.38% on GSM8K, compared with 41.17% for the training-answer basis.

Coverage and accuracy. During the rebuttal period we trained students with nested subsets of the training-answer vocabulary at rank 32. The 50% and 75% subsets covered 47.47% and 71.27% of test-answer token occurrences, respectively. Complete-answer coverage rose from 44.96% to 69.37%, while mean accuracy stayed near 40%. Even the 50% group exceeded the non-answer control in all three seeds. For the 50% subset, mean accuracy across three seeds was 42.66% on the 593 questions whose answers were fully covered and 37.92% on the other 726. These descriptive groups may differ in answer length and difficulty.

Construction on another task. For a new task, we collect token types from its training answers, center their pretrained output-weight rows, and freeze the top 32 right singular vectors before student training. We applied this rule to symbolic reasoning on ProsQA. Its training vocabulary covers all 38 test answer concepts across 500 questions. ALUS achieves 89.53% accuracy, an 8.07 percentage-point gain over the student trained without alignment across three paired seeds.

Full-vocabulary basis. When a task-specific answer vocabulary is difficult to specify, the same construction can use all 50,257 pretrained GPT-2 tokens. In the GSM8K experiment conducted during the rebuttal, this rank-32 basis yields 460, 486 and 454 correct answers out of 1,319 for seeds 11, 12 and 13 (35.38±1.29%). The submitted rank-32 answer basis scores 41.17±0.20% and leads in all three seeds, by 5.79 percentage points on average. On ProsQA, in a separate rebuttal experiment, this full-vocabulary basis achieves 88.33±0.76% at rank 32, compared with 89.53±1.33% for the answer basis. The full-vocabulary run used a different loss aggregation and validation frequency from the submitted answer-basis run. Under their respective training protocols, the answer basis led in all three seeds, by 1.20 percentage points on average.

Transfer without retraining. The GSM8K-trained model also transfers to new numeric question distributions without further training. ALUS improves over the no-alignment student by 7.87 points on SVAMP, 6.88 on ASDiv, and 22.03 on MultiArith (revised Table D9).

Conclusion. On GSM8K, even 50% answer-type retention yields higher mean accuracy than either the non-answer control or the rank-matched full-vocabulary basis. For a new task, the ProsQA results support using training-answer tokens when available and show that a rank-matched full-vocabulary basis is a viable alternative. The numeric transfer tests further show gains on new question distributions without retraining.

GSM8K test accuracy by training-answer token retention (mean ± SD over three seeds).

Training-answer types retainedAccuracy (%)
0% (non-answer control, Table C3)37.53±0.89
50%40.06±0.61
75%39.95±0.47
100%41.17±0.20

ProsQA vocabulary comparison (Table D8).

SeedAnswer basisFull-vocabulary basisDifference
11455/500 (91.00%)446/500 (89.20%)+1.80 pp
12446/500 (89.20%)440/500 (88.00%)+1.20 pp
13442/500 (88.40%)439/500 (87.80%)+0.60 pp
Mean ± SD89.53 ± 1.33%88.33 ± 0.76%+1.20 ± 0.60 pp
AuthorsOpenReview ↗

Response to Reviewer H4rV (2/2)

Response to W2

Thank you for pointing this out. We capped the number of teacher spans at three and supervised only nonempty spans. Under this limit, doubling student updates from three to six changed mean GSM8K accuracy from 41.17±0.20% to 41.14±0.74%. The 4–5-reference-step questions show why the distinction matters. Relative to the same three-update student under the three-span limit, six updates gained 1.98 points under the six-span limit and changed accuracy by −1.20 points under the three-span limit. Extra student updates alone therefore do not explain the original subgroup gain.

Three-span limit. During the rebuttal period, we trained the six-update student to take two updates per span and align their combined change to one teacher target, keeping one supervision point per nonempty span. Across three paired seeds, the mean change over all 1,319 questions was −0.03 percentage points. Table D3 gives the overall and length-stratified accuracies.

Reference-solution length. At fixed segmentation, the changes for questions with 1–3, 4–5 and at least six reference steps were +1.10, −1.20 and −1.56 points. On the 150 questions with the longest reference solutions, accuracy fell from 11.33% to 9.78%. Longer reference solutions thus gave no observed advantage to the larger student budget in this control.

Conclusion. Under the three-span limit on GSM8K, K=3 is the more economical choice. For a new task, we would compare candidate K values on representative validation questions at fixed segmentation before assigning extra updates to longer problems.

Accuracy by reference-solution length (Table D3). Each configuration is labeled as student updates / maximum teacher spans. Brackets give the change from 3/3 in percentage points, computed before rounding.

Reference stepsQuestions3/3 (%)6/6 (%) [Δ pp]6/3 (%) [Δ pp]
1–369758.9258.35 [−0.57]60.02 [+1.10]
4–547224.4426.41 [+1.98]23.23 [−1.20]
≥615011.3311.11 [−0.22]9.78 [−1.56]

Response to W3

Thank you for your question. The controlled comparisons point to two different conditions. The benefit of a local update target depends on the step configuration; the benefit of answer directions on GSM8K depends strongly on whether supervision reaches intermediate steps.

Local targets. On three-update GSM8K, Local exceeds State and Cumulative by 2.20 and 2.00 points. Shifting the same teacher updates to different steps costs 7.38 points, which shows the value of assigning each change to its corresponding span. With six updates and finer segmentation, the three targets are within 0.43 points. In our rebuttal diagnostic (seed 11, 200 updates), the Local alignment and answer gradients approach orthogonality at K=3 (cosine −0.127 to +0.001), but become more opposed at K=6 (−0.042 to −0.095). We then tested whether removing the conflicting gradient component helped at K=6. After 50 further updates, answer cross-entropy at step 250 was 2.5306 with the conflict removed and 2.5297 with the original gradient. Removing the conflict gave no short-run improvement on this trajectory.

Answer directions. With Local targets and rank 32 fixed on GSM8K, switching from random to answer directions gains 5.89 points at K=3 and 7.81 at K=6 when all valid updates are supervised. With supervision only at the last valid update, the differences are −0.99 and −0.10. On ProsQA, where each question supplies two candidates, the answer basis averages 89.53% versus 88.40% for random and is ahead in one of three seeds.

Conclusion. The clearest evidence for Local comes from ordered, three-step supervision. For answer directions, the numerical comparisons at both K values point to a different condition: the selected directions are useful when they guide intermediate computations throughout the solution. On ProsQA the incremental contribution of direction selection is smaller.

Reviewer_qsWPOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies supervision targets, allocation, and alignment spaces for implicit chain-of-thought within a shared recurrent student, and proposes ALUS to align all valid local updates in an answer subspace. Experiments cover GPT-2, Pythia-1B, mathematical reasoning, and ProsQA, with additional six-update evaluations, loss ablations, and inference-cost measurements.

Critical strengths

The shared student architecture, initial-gradient calibration, and rank-matched random, teacher-PCA, and frequency-matched subspace controls provide reasonably convincing evidence for the roles of answer-related directions and direct supervision placement. ALUS exceeds the strongest adapted supervision objective by 4.88 percentage points on GPT-2/GSM8K, provides useful gains across settings, and introduces no inference-time alignment modules.

Critical weaknesses
  1. The explanation for the advantage of local updates remains limited: the error-aggregation analysis establishes differences between objectives but does not connect them to observed optimization and accuracy. With six updates, Local, State, and Cumulative achieve 41.55%, 41.98%, and 41.70%, respectively, so the evidence more strongly supports the answer subspace and all-valid supervision than a general advantage of local updates themselves.
  2. The strongest comparative results involve substantially simplified objective adaptations: for example, LoRi’s decomposition is replaced, and several original SPOT and KaVa components are removed. Initial-gradient norm matching also does not replace validation-based tuning for each objective, so the 4.88-percentage-point advantage does not yet establish competitiveness against complete existing methods.
  3. The answer subspace always uses rank 32, without dimensionality sensitivity or evaluation with more open-ended answer vocabularies; on ProsQA, its mean gain over the rank-matched random subspace is only 1.13 percentage points, with no improvement in two of three seeds. Inference timing is reported, but training overhead from teacher computation, the reference-text branch, and calibration is not quantified, limiting assessment of applicability and overall cost.
Questions for authors
  1. Can training-time cross-step error correlations or gradient analyses explain why local targets help with three updates but no longer outperform state and cumulative targets with six?
  2. Does ALUS retain its advantage when the strongest adapted baselines receive comparable validation-based tuning budgets? Could at least one complete existing method be included?
  3. How sensitive are the answer-subspace gains to rank and answer-vocabulary coverage? Including the teacher, reference-text branch, and calibration, how much do training time and peak memory increase over the no-alignment student?
2 comments
AuthorsOpenReview ↗

Response to Reviewer qsWP (1/2)

Response to W1 / Q1

Thank you for your question. The accuracy advantage of Local at three updates narrows with six updates and finer teacher segmentation. Our rebuttal experiments identify a difference in early optimization, although the short gradient intervention leaves its contribution to final accuracy unresolved. Local leads State and Cumulative by 2.20 and 2.00 points on three-update GSM8K; the three means are within 0.43 points in the six-update setting.

Do errors accumulate differently across steps? Local gives each transition its own loss, whereas a cumulative target combines errors across transitions. At warmup update 200, the mean cosine between adjacent Local direction-error vectors was −0.041 at K=3 and −0.049 at K=6. The late K=6 step pairs had only 4, 2 and 0 valid examples for estimating across-example error correlations. These measurements give little support for a change in error cancellation as the explanation for the accuracy pattern.

Does local alignment compete with answer prediction? We tracked six seed-11 trajectories on 32 fixed probe examples at updates 0, 25, 100 and 200, matching initialization, sample order and initial alignment-gradient norms within each K. For Local, the alignment–answer gradient cosine moved toward zero at K=3 (−0.127 to +0.001) and became more negative at K=6 (−0.042 to −0.095). We tested the latter observation by continuing the K=6 Local run for 50 updates. Endpoint answer cross-entropy was 2.5297 with the original gradient, 2.5306 after removing the conflicting alignment component and 2.5300 in the norm-matched control.

Conclusion. The gradients show that local supervision interacts differently with answer prediction early in the two configurations. Removing the conflict gave no short-run improvement, so its contribution to final accuracy remains unresolved. Our performance conclusion is configuration-specific: Local has a measured advantage at three updates. At six updates, the clearer gains come from supervising all valid positions and using answer directions.

Response to W2 / Q2

Thank you for your suggestion. In Table 4, ALUS and each of the five adapted baselines received a validation-tuning budget of three candidate configurations. For the comparison with the strongest baseline on the shared student, we tested three learning rates for ALUS and three decoder learning rates for SIM-CoT before evaluating the selected configurations on seeds 11–13. Across three training seeds, ALUS achieved 41.17% on GSM8K versus 36.29% for SIM-CoT, a gain of 4.88 percentage points.

Validation tuning. Each adapted baseline in Table 4 was tuned separately on validation data. Appendices A.4–A.5 clarify which weights were calibrated. LoRi used a calibrated overall weight of 0.052562. SPOT and KaVa each used a loss weight of 1. For SIM-CoT’s fully trainable 124M-parameter decoder, learning rates of 1e-4 and 3e-4 achieved 68.02% and 67.58% validation accuracy on seed 11, respectively. The third candidate, 3e-3, failed the stability criterion in Appendix A.5. We therefore used 1e-4 for seeds 11–13.

What the shared-student comparison preserves. LoRi matches low-rank trajectory statistics, SPOT aligns states to spans through a frozen output head, and KaVa matches teacher keys and values. For the specific concern about LoRi’s decomposition, our rebuttal experiment found a normalized subspace overlap of 1.0000 between rank-32 PCA and the Tucker hidden-mode factor on identical centered inputs from 128 examples. Thus Table 4 tests the methods’ distinctive supervision mechanisms in the same student, and the LoRi check verifies that the two calculations select the same hidden directions under matched inputs.

Complete-method comparison. During the rebuttal, we added the original Coconut as a complete-method baseline. Its GSM8K accuracy is 34.1±1.5% with a 50-epoch curriculum and six continuous thoughts. ALUS achieves 41.17±0.20% with 40 epochs and three updates.

Conclusion. Table 4 therefore provides a fair comparison between ALUS and the five adapted baselines on the shared student. Each baseline retains its defining supervision mechanism and can select its own hyperparameters with the same validation budget as ALUS. Under these conditions, ALUS leads SIM-CoT by 4.88 points on GSM8K. Coconut provides the requested complete-method reference under its original training protocol.

AuthorsOpenReview ↗

Response to Reviewer qsWP (2/2)

Response to W3 / Q3

Thank you for your question. On GSM8K, intermediate ranks outperform the extremes in our sweep and retaining half the training-answer vocabulary gives accuracy within about one point of the full reference. Relative to a student with the same reference-text loss, alignment adds 13.9% to time per update and 4.7% to peak allocated memory.

Rank. During the rebuttal period, we ran a rank-sensitivity experiment with three seeds and 40 training epochs at q=8, 16, 64 and 128, yielding 34.17±0.56%, 39.10±0.24%, 39.68±0.81% and 37.63±0.46%. The intermediate ranks perform better than the two extremes in this sweep. The submitted q=32 configuration achieves 41.17±0.20%; Table C5 lists it alongside the four ranks tested in the sweep and identifies the training protocol.

Vocabulary coverage. On GSM8K, retaining 50% or 75% of the training-answer vocabulary for the rank-32 basis gives 40.06±0.61% and 39.95±0.47% accuracy, respectively, across three seeds in the experiments we ran during the rebuttal period. Both are about one percentage point below the 100% retention reference (41.17±0.20%) and above the 0% retention control (37.53±0.89%, Table C3). In the 0% control, all training-answer token types used to build the basis are replaced by an equal number of frequency-matched non-answer token types.

Full-vocabulary alternative. On GSM8K, a rank-32 basis built from all 50,257 GPT-2 tokens yielded 35.38±1.29% versus 41.17±0.20% for the training-answer basis. Both used 40 epochs and validation-accuracy checkpoint selection. On ProsQA, the training vocabulary covers all 38 test answer concepts. The answer, random and full-vocabulary bases scored 89.53%, 88.40% and 88.33%, respectively, with the answer basis ahead of random in one of three seeds. Tables C4 and D8 report the vocabulary comparisons.

Training overhead. During the rebuttal period, we benchmarked training cost. The measured increment is 13.9% in update time and 4.7% in peak allocated memory over the no-alignment student, which already includes reference-text CE. On one RTX 5090, ALUS takes 2.636 s per update and 12.16 GiB, versus 2.314 s and 11.61 GiB without alignment. The answer-generation-only reference takes 1.615 s and 7.54 GiB. These matched measurements use the same GSM8K examples, effective batch 256, BF16 and gradient checkpointing, with two warmup and eight timed updates. They include the frozen teacher; its forward pass takes 0.133 s per ALUS update. Four 128-example calibration groups add a one-time 4.91 s. Table D10 gives the measured costs.

Conclusion. These measurements support selecting rank on validation data and using training-answer tokens when available. For the measured GSM8K setup, the incremental cost of alignment is 13.9% per update and 4.7% in peak allocated memory, with calibration paid once.

Reviewer_2iL2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates supervision mechanisms for implicit chain-of-thought reasoning along three dimensions: supervision targets, allocation, and representation space. Based on mathematical analysis and empirical findings, the authors propose Answer-guided Local Update Supervision, which aligns teacher and student hidden-state updates at each valid reasoning step within an answer-related subspace derived from output weights, without introducing additional inference-time modules. Experiments on GSM8K demonstrate improvements over several adapted supervision baselines, while additional evaluations across different reasoning lengths, model backbones, and reasoning tasks further assess its effectiveness. Ablation studies examine the contributions of different supervision choices and loss components.

Critical strengths
  1. The paper provides a structured analysis of implicit CoT supervision from three complementary dimensions: Targets, Allocation, and Space. The combination of mathematical analysis and controlled experiments offers useful insights into how supervision choices influence learning, making the contribution valuable beyond the proposed method itself.

  2. ALUS demonstrates consistent improvements over no-alignment training across different backbones and reasoning tasks. The shared student architecture and extensive comparisons of supervision strategies help isolate the effects of supervision design, providing meaningful empirical support for the proposed approach.

Critical weaknesses
  1. Although the paper provides mathematical analyses of local-update supervision and answer-subspace construction (Sections 3.1–3.3 and Appendix B), these analyses do not establish why the proposed choices should consistently improve reasoning performance. Moreover, prior studies have already explored intermediate supervision and output-related representation alignment, making the novelty and theoretical significance of ALUS less convincing.

  2. The empirical advantages of ALUS are not fully consistent across settings. For example, local-update supervision does not outperform state or cumulative supervision when K=6K=6 (Tables D1–D2), raising concerns about the robustness of its core design choice. Furthermore, several baseline methods are substantially modified for comparison (Appendix A.5), potentially weakening their performance. These issues limit the extent to which the experimental results establish the superiority and generalizability of ALUS.

Questions for authors
  1. In Section 1, the authors motivate the study by emphasizing how supervision design affects learning signals. However, is there empirical evidence that inappropriate supervision targets, allocation, or spaces are major bottlenecks in existing implicit CoT methods, rather than other factors such as student capacity or teacher quality?
  2. Section 3.1 and Appendix B.2 show that local and cumulative supervision have equivalent exact matching conditions but different error aggregation properties. What theoretical or empirical evidence establishes that these differences should favor local supervision under the actual direction/log-norm objective?
  3. In Sections 3 and 3.4, ALUS aligns teacher updates derived from textual reasoning spans with student updates generated through continuous recurrence. Given their different computational processes, what evidence supports the assumption that these updates are semantically corresponding and should be directly aligned?
  4. Section 3.3 (Eq. 5) constructs the answer subspace from pretrained output weights, while Appendix B.4 establishes its approximation properties under a fixed linear readout. How does this theoretical result justify improved reasoning ability rather than merely better alignment of answer-token scores? Could the authors provide evidence connecting the selected directions to actual reasoning information?
  5. Table 4 compares ALUS with adapted versions of existing methods. However, Appendix A.5 indicates that several adaptations omit important components of the original approaches (e.g., LoRi's Tucker decomposition and SPOT's hybrid architecture). Could these modifications disadvantage the baselines, and how can the authors establish the fairness of these comparisons?
  6. Sections 4.4–4.5 report improvements across different settings, but all results are based on only three seeds. Could the authors provide stronger statistical evidence, particularly for smaller improvements such as the 1.13-point difference between answer and random subspaces on ProsQA (Table D4), to support the claimed effectiveness and generalizability?
  7. Could the authors investigate sensitivity to the subspace rank qq, reasoning length KK, and teacher-span segmentation strategy? Experiments with stronger backbones, more diverse reasoning tasks, and direct measurements of intermediate reasoning quality would further strengthen the conclusions (Sections 3.3 and 4.5).
  8. Tables 1–3 primarily examine individual supervision choices, while Table 2 additionally considers the interaction between Allocation and Space. Could the authors investigate interactions involving Targets, particularly Targets × Allocation and Targets × Space, to establish whether the three design choices contribute consistently and independently?
6 comments
AuthorsOpenReview ↗

Response to Reviewer 2iL2 (1/6)

Response to W1

Thank you for your comment. Prior work has used intermediate teacher representations to supervise latent reasoning. With the teacher and student architecture held fixed, we study how the learning signal changes when the student matches the teacher’s state, its cumulative change from the start, or its change over a single reasoning span. We then test which student updates benefit from direct supervision and along which directions the updates should be matched. ALUS combines these findings by aligning each valid student update with the teacher’s change over the corresponding span in a subspace derived from the output weights of training-answer tokens.

How ALUS differs from prior work. SPOT uses frozen vocabulary weights to compare a student state with the content of a teacher span. LoRi uses teacher-derived low-rank directions to match trajectory statistics and anchors the answer boundary. ALUS instead pairs each student update with the teacher’s change across the corresponding span and matches the two changes along directions derived from training-answer output weights. The projection acts only in the training loss.

Why these choices work together. Two successive update errors can offset each other even though the intervening state, which feeds the next computation, is already wrong. Supervising each update exposes that error at the step where it occurs (Section 3.1). The answer basis determines which part of the error receives supervision: at fixed rank, it minimizes the worst-case distortion of a unit update under the frozen, centered answer readout (Appendix B.4).

What the controls show. On GPT-2/GSM8K, cyclically shifting the same teacher updates across student steps lowers three-step accuracy from 41.17% to 33.79% (Section 4.1). Holding the local target and rank 32 fixed, the answer basis gains 5.89 and 7.81 points over a random basis at K=3 and K=6 when all valid updates are supervised. With direct supervision only at the last valid update, the corresponding differences are −0.99 and −0.10 points.

Conclusion. ALUS directly supervises the change at each valid step, so an error there receives a training signal even if a later update offsets it. The answer readout determines which components of that change enter the loss. The shift control shows that the ordering of the teacher updates matters, and the three- and six-update comparisons show when the answer directions help most. Their advantage appears when they guide the intermediate updates as well as the last one.

Response to W2

Thank you for your comment. ALUS improves over the matched no-alignment student at both tested reasoning lengths and over the strongest validation-tuned baseline at three updates. The advantage of Local over alternative targets is clearest at three updates.

Reasoning length. At K=3, Local outperforms State and Cumulative on GPT-2/GSM8K (Table 1). At K=6, their accuracies are close (41.55%, 41.98%, and 41.70%), while ALUS still gains 9.50 points over the matched no-alignment student (Table 5). The latter comparison tests the combined objective at six updates.

Baseline fairness. Each of the five adapted baselines in Table 4 retains its core mechanism and is tuned separately with the same budget as ALUS: three candidate configurations per method. SIM-CoT retains its fully trainable 124M-parameter decoder, which reconstructs each span, propagates gradients to the student and is removed at inference. With the same teacher, data and update budget, ALUS reaches 41.17±0.20% and SIM-CoT 36.29±0.92%, a 4.88-point difference. Q5 details the LoRi and SPOT implementations.

Within-student controls. Adding ALUS alignment at K=3 gains 8.31 points with the other losses fixed, and selecting answer directions over rank-matched random directions gains 5.89 points with the targets and positions fixed.

Conclusion. At three updates, Table 4 provides a fair comparison. The five baselines retain their core supervision mechanisms, use the same student and training data as ALUS, and are tuned independently within the same validation budget. ALUS leads the strongest adapted baseline by 4.88 points. At six updates, ALUS gains 9.50 points over no alignment, while the three target choices have similar mean accuracy.

AuthorsOpenReview ↗

Response to Reviewer 2iL2 (2/6)

Response to Q1

Thank you for your question. Section 1 asks “how does the design of supervision affect learning signals and reasoning performance?” Ranking the principal bottlenecks of existing implicit CoT systems would require varying capacity and teacher quality as well. Our study asks what can be gained for a given teacher–student pair.

Experimental design. The motivation is that an explicit rationale supplies the teacher with a sequence of textual steps, while the student has no observed target for what each latent update should represent. Even after choosing a student and a teacher, one must still decide what to match, which updates to supervise directly, and which representation directions to use. We therefore hold the student architecture and frozen teacher checkpoint fixed within each comparison and vary one supervision choice at a time.

Matched comparisons. On GPT-2/GSM8K at K=3, switching from state targets to local updates improves accuracy by 2.20 percentage points. Extending supervision from the last to all valid updates in the answer subspace improves it by 8.14 points. Replacing a rank-matched random subspace with the answer subspace under all-valid local supervision improves it by 5.89 points. Each contrast favors the same choice in all three paired seeds.

Conclusion. With the student and teacher held fixed, changing the supervision design produces measurable accuracy gains. Section 1 makes no claim that it is a larger bottleneck than student capacity or teacher quality.

Response to Q2

Thank you for your question. Under the implemented direction/log-norm loss, Local gives each update a direct correction for its own error. Controlled three-update comparisons on GPT-2 and Pythia-1B favor this choice over State and Cumulative targets.

Learning signal. For the implemented stabilized cosine and log-norm SmoothL1 loss, hold the teacher, projection and position weights fixed. Differentiating with respect to one projected update gives a single direct pair term under Local. Under Cumulative, every supervised prefix containing that update contributes. Both retain recurrent backpropagation. Appendix B.5 relates these discrepancies to projected vector error.

Controlled result. On three-update GPT-2/GSM8K, all-valid supervision and the answer subspace are fixed. Local achieves 41.17±0.20%, State 38.97±0.67% and Cumulative 39.17±0.88% (Table C1). On Pythia-1B, Local gains 2.22 and 1.97 points over State and Cumulative (Appendix D.2). Both contrasts favor Local in every seed on both backbones. Appendix B.2 links this derivation to the comparisons in Section 4.1.

Conclusion. The derivative identifies which error each step is trained to correct. Under that same loss, the matched experiments on two backbones favor direct correction of the local update in the three-step setting.

Response to Q3

Thank you for your question. The strongest evidence for the proposed pairing is that moving the same teacher changes to different student steps lowers accuracy. A complementary probe identifies an intermediate arithmetic result that the trained student encodes.

Does the pairing matter? Within each rationale, we keep the teacher update vectors and ALUS loss coefficients fixed (4.0 for direction and 0.8 for log norm) and cyclically shift which student step receives each vector. Ordered supervision achieves 41.17±0.20% on GPT-2/GSM8K, compared with 33.79±0.64% after the shift (Table C1). Every seed favors the ordered assignment.

What content is accessible? During the rebuttal period, we trained linear probes on frozen student states to predict the first operation’s result in two-operation arithmetic. On 630 held-out problems, accuracy averaged over three update positions and three training seeds was 91.59% for ALUS and 23.26% without alignment (Q7).

Conclusion. Teacher span boundaries define the training pairs. Shifting the targets shows that their order matters, and the probe shows that the student encodes an intermediate arithmetic result. A span can split or combine steps in a written solution, so these findings support the training correspondence while leaving the precise operation at each latent update open.

AuthorsOpenReview ↗

Response to Reviewer 2iL2 (3/6)

Response to Q4

Thank you for your question. The fixed-readout result explains how to select the directions. We test whether transferring the answer-subspace component of an intermediate state can influence the next operation in a two-operation arithmetic problem.

Subspace component transfer. Appendix B.4 shows that, at fixed rank, the answer basis minimizes the worst error in the frozen, centered answer readout for a unit update. Consider a recipient solving (3+4)×2 and a donor solving (5+6)×3. Each student state is computed from its question alone. At the second latent update, we replace only the recipient’s rank-32 answer-subspace component with the donor’s and let the recipient finish. If the transferred information allows the recipient to use the donor’s intermediate sum, 11, in its own multiplication by 2, the predicted answer is 22. Copying either problem’s final answer would give 14 or 33. Only 22 counts as success for this pair.

Intervention result. Across 128 fixed arithmetic pairs, the recipient produced the specified combined answer in 17.97% of cases, averaged over three ALUS training seeds. On the same models and pairs, repeating the intervention in each of three independent rank-32 random subspaces yielded 0%. Keeping the answer-subspace replacement fixed, students trained with random-subspace supervision achieved only 0.78% hybrid-answer accuracy, compared with 17.97% for students trained with answer-subspace supervision. Table E2 reports both controls.

Implication for training. With local targets, all-valid supervision and rank 32 fixed, GSM8K accuracy is 41.17% for the answer basis, 35.28% for random directions and 34.39% for teacher PCA (Table 3).

Conclusion. The GSM8K comparison shows the training benefit of supervising these directions. On the two-operation diagnostic, transferring the answer-space component can make the recipient produce the answer predicted by combining donor-side information with the recipient’s downstream operation.

Response to Q5

Thank you for your comment. The five adapted baselines in Table 4 retain their core supervision mechanisms and share the same three-update GPT-2 student as ALUS. We allocated three candidate configurations per method and selected hyperparameters separately on validation data. Under those conditions ALUS retains its advantage. The teacher checkpoint, training data, answer and reference-text losses, optimizer update count and checkpoint-selection rule are shared.

LoRi. We retain trajectory-statistic matching and the answer-boundary anchor. We keep both selected layers and estimate a shared rank-32 basis from teacher updates. On identical centered inputs, PCA selects the same hidden directions as LoRi’s Tucker hidden-mode factor. During the rebuttal period, we verified this on 128 training examples (normalized subspace overlap 1.0000). This establishes equivalence of the hidden-direction calculation under matched inputs. LoRi’s original basis also depends on its rationale-token states and layer factorization.

SPOT. We retain frozen-head state-to-span alignment. With one latent state per span and uniform token weights, the transport cost reduces to the mean tokenwise cost plus a constant. Restoring hybrid text/pause inputs would change the intermediate text available to the student, and rejection-sampled fine-tuning would introduce another training stage. Keeping these conditions fixed lets us evaluate SPOT’s alignment mechanism on the same latent reasoning task.

Strongest baseline. SIM-CoT, the strongest baseline here, retains its fully trainable 124M-parameter step-reconstruction decoder and uses the learning rate chosen on validation data. It reaches 36.29±0.92% on GSM8K, compared with 41.17±0.20% for ALUS.

Conclusion. We preserved each baseline’s core supervision mechanism and gave every method three candidate configurations for validation tuning. In this fair comparison on the same three-update student, ALUS leads the strongest adapted baseline, SIM-CoT, by 4.88 percentage points on GSM8K.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of our response and manuscript revisions

We thank reviewers 2iL2, qsWP, and H4rV for their careful reading and constructive feedback.

We will address each question in its review thread. The revised manuscript incorporates the following results and clarifications.Changes in the revised manuscript are highlighted in yellow.

Contribution of ALUS (2iL2, H4rV). We compare supervision targets, positions, and subspaces within the same recurrent student. ALUS brings these choices together by matching each valid student update to the teacher’s change over the corresponding text span, along directions derived from the output weights of training-answer tokens. On GPT-2/GSM8K at both three and six updates, these directions outperform rank-matched random directions on average when all valid updates receive supervision. The advantage disappears when only the last valid update receives supervision (Sections 4.2 and 5, Table C2).

Intermediate reasoning (2iL2). On held-out two-operation arithmetic problems, a linear probe recovered the first operation’s result from ALUS states with 91.59% accuracy, compared with 23.26% without alignment, averaged over three update positions and three seeds. In a separate test, transferring the answer-space component at the second update produced the answer predicted by combining the donor’s intermediate result with the recipient’s final operation in 17.97% of cases. Each of three rank-matched random subspaces gave 0% on the same models (Tables E1 and E2).

Baseline fairness (qsWP, 2iL2). Table 4 provides a fair comparison of ALUS and the five adapted baselines on the same student. Each baseline retains its core supervision mechanism, and each method is tuned independently with a budget of three candidate configurations per method. Appendix A.5 details the adaptations. Table 4 also lists Coconut under its original training protocol.

Reasoning length and teacher segmentation (H4rV, qsWP, 2iL2). On GSM8K, ALUS gains 8.31 percentage points at three updates and 9.50 at six over a student trained without alignment at the same update count. To isolate the effect of additional student updates, we also capped the teacher at three spans and supervised each nonempty span once. Increasing student updates from three to six then changed mean accuracy from 41.17% to 41.14% (Sections 4.4–4.5 and Appendix D.1).

Answer-vocabulary coverage (H4rV, qsWP). In Appendices C.3 and D.3, we report GSM8K test-answer coverage and accuracy at partial vocabulary retention, together with rank-matched full-vocabulary results on GSM8K and ProsQA.

We believe these revisions better address the main concerns raised by the reviewers. We hope the revised manuscript meets your expectations and conveys the value of our work.

Dear Reviewers,

Thank you for your time and effort in reviewing this paper. The authors have posted responses to your reviews. If you have not yet done so, please read their responses and leave a follow-up comment visible to the authors, indicating which concerns have been addressed and which remain unresolved.

Please also update your review and rating if your assessment has changed. Your feedback will help the authors address any remaining questions during the discussion period.

Thank you!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.