- Rating
- 2
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Computation Aligned Distillation (CAD), which aims to align the internal computation of CoT reasoning steps and final-answer generation. The authors define computation alignment through circuit overlap and introduce a perturbation-sensitivity proxy for training. Experiments across different distillation objectives, teacher/student models, and reasoning datasets show improvements over the corresponding baselines.
Critical strengths
-
The paper defines computation alignment through the overlap between reasoning and answer circuits, and proposes a perturbation-based proxy that can be optimized without expensive circuit tracing during training. The reported correlation between the proxy and actual circuit overlap provides initial support for this approach.
-
CAD improves several existing distillation methods. For example, DFT improves from 56.0% to 63.0% on SVAMP, while Classic CoT improves from 52.78% to 87.20% on MultiArith. The comparison with typo augmentation also shows that the gains cannot be explained by simple input augmentation alone.
Critical weaknesses
-
The paper defines computation alignment as circuit overlap in Eq. 9, but CAD actually optimizes the ratio of hidden-state perturbation sensitivities in Eqs. 10–12. Similar sensitivity magnitudes do not necessarily imply overlapping computational circuits. Although Figure 3 reports a strong correlation on Llama-3.1-8B-Instruct, the paper does not demonstrate that CAD training increases actual circuit overlap in the student models. It also does not rule out more general representation regularization effects. Therefore, the experiments show performance improvements but do not sufficiently establish that these improvements come from the proposed computation alignment mechanism. This is a major concern given the paper's central claim.
-
In Table 1, Classic CoT and GEM achieve only 18.94% and 19.55% on GSM8K, both below the undistilled model's 24.73%. CAD substantially improves these weak baselines, but the gains over the stronger DFT baseline are much smaller. The paper uses common training hyperparameters across different objectives and reports only one training seed. Without better-tuned baselines and multi-seed results, it is difficult to determine how reliable the smaller improvements are.
Questions for authors
-
Can the authors measure the actual circuit overlap defined in Eq. 9 before and after CAD training on Qwen2.5-0.5B and Gemma-2-2B? Does increased circuit overlap correlate with improved reasoning accuracy? This is essential for validating the central claim.
-
Figure 3 evaluates the relationship between sensitivity discrepancy and circuit overlap on Llama-3.1-8B-Instruct. Does this correlation hold for the student models used in distillation, both before and after training? Since Eq. 11 only compares response magnitudes, how can the authors distinguish shared computational pathways from different circuits with similar sensitivities?
-
How does CAD compare with general hidden-state consistency regularization or sensitivity matching using randomly selected intermediate tokens? Figure 6 rules out simple typo augmentation, but does not establish that the improvement specifically comes from reasoning–answer circuit alignment.
-
Why do Classic CoT and GEM perform worse than the undistilled model on GSM8K? Were the different distillation objectives independently tuned? Can the authors report results across multiple training seeds, particularly for the smaller improvements over DFT and other strong baselines?
-
CAD requires additional perturbed forward passes and hyperparameter search. What is the end-to-end computational overhead, and how does CAD compare under a matched compute budget? Also, the GSM8K DFT+CAD score differs between Tables 1 and 2–3 (42.47 vs. 42.27), and the improvement reported in Table 1 does not match its displayed scores.
4 comments
2/2
To major W2 and Q4:
We evaluate two tuning protocols over three seeds (42, 456, 6), reporting mean ± std.
- Tuned training hyperparameters & fixed method hyperparameters (Table F). Baseline and CAD independently tune learning rates {2e-5, 3e-5} and epochs {3, 4, 5}, with CAD's fixed as shown below. CAD improves all three distillation methods.
- Fixed training hyperparameters & tuned method hyperparameters (Table G, H and I).
- Consistent gains (Table G). Under identical training settings, CAD improves most configurations. Classic CoT falling below the undistilled student is a known effect of long-CoT supervision on small models (L363-366).
- Robustness to Selection (Table H and I). Over 97% (70/72) of DFT + CAD configurations on GSM8K and over 91% (66/72) on BoolExp exceed the baseline’s validation accuracy.
Table F. Accuracy (%) under matched tuning budgets in the Basic Distillation Setting
| Distillation Method | Base | + CAD | ||||
|---|---|---|---|---|---|---|
| Classic CoT | 0.5 | 0.4 | 6 | 28.23 ± 3.91 | 41.46 ± 3.59 | +13.23 ± 6.31 |
| DFT | 0.1 | 0.1 | 4 | 38.23 ± 1.98 | 40.30 ± 2.23 | +2.07 ± 0.87 |
| GEM | 0.1 | 0.2 | 6 | 25.15 ± 3.21 | 38.89 ± 0.53 | +13.74 ± 2.73 |
Table G. Accuracy (%) under tuned method hyperparameters in the Basic Distillation Setting (mean ± std).
| Distillation Method | GSM8K | SVAMP | MultiArith | MAWPS | ASDiv |
|---|---|---|---|---|---|
| Classic CoT | 20.05±1.14 | 40.33±6.75 | 42.04±9.45 | 46.20±3.61 | 44.82±4.99 |
| Classic +CAD | 40.31±1.81 | 55.50±4.09 | 88.51±4.32 | 76.38±5.72 | 62.95±5.14 |
| Δ | +20.26±1.63 | +15.17±10.77 | +46.47±11.96 | +30.18±9.33 | +18.13±7.77 |
| DFT | 38.23±1.16 | 57.33±3.21 | 83.89±2.55 | 76.82±1.13 | 64.19±0.96 |
| DFT +CAD | 40.61±1.93 | 57.17±5.20 | 87.41±1.28 | 78.86±1.54 | 64.90±0.57 |
| Δ | +2.37±2.72 | −0.17±7.52 | +3.52±3.61 | +2.03±1.71 | +0.71±0.59 |
| GEM | 18.59±1.53 | 37.67±2.84 | 42.04±7.25 | 52.20±2.46 | 46.22±1.15 |
| GEM +CAD | 37.37±0.98 | 57.00±2.18 | 86.30±1.16 | 74.47±2.76 | 63.59±0.92 |
| Δ | +18.79±0.76 | +19.33±3.33 | +44.26±7.40 | +22.27±4.81 | +17.37±1.08 |
Table H. Validation accuracy (%) across the full DFT+CAD hyperparameter grid on BoolExp (baseline: 62.2%)
| 0.001 | 0.1 | 80.00 | 79.40 | 90.00 | 71.40 |
| 0.001 | 0.2 | 70.60 | 86.80 | 45.60 | 68.80 |
| 0.001 | 0.4 | 80.40 | 68.00 | 81.20 | 90.60 |
| 0.003 | 0.1 | 84.20 | 67.40 | 42.80 | 94.80 |
| 0.003 | 0.2 | 80.80 | 92.00 | 93.40 | 91.40 |
| 0.003 | 0.4 | 83.00 | 86.40 | 87.00 | 43.60 |
| 0.010 | 0.1 | 76.60 | 52.40 | 88.80 | 94.40 |
| 0.010 | 0.2 | 77.60 | 88.20 | 88.60 | 88.40 |
| 0.010 | 0.4 | 74.80 | 79.80 | 91.40 | 93.00 |
| 0.030 | 0.1 | 83.40 | 63.80 | 85.80 | 91.80 |
| 0.030 | 0.2 | 86.40 | 90.00 | 93.20 | 91.20 |
| 0.030 | 0.4 | 90.40 | 92.00 | 85.40 | 75.40 |
| 0.100 | 0.1 | 90.80 | 88.00 | 94.80 | 93.60 |
| 0.100 | 0.2 | 88.60 | 61.60 | 93.60 | 79.80 |
| 0.100 | 0.4 | 79.20 | 87.40 | 92.60 | 88.20 |
| 0.300 | 0.1 | 75.80 | 82.00 | 91.40 | 92.60 |
| 0.300 | 0.2 | 88.00 | 92.60 | 91.80 | 93.40 |
| 0.300 | 0.4 | 66.60 | 93.60 | 15.40 | 89.80 |
Table I. Validation accuracy (%) across the full DFT+CAD hyperparameter grid on GSM8K (baseline: 39.45%)
| 0.001 | 0.1 | 40.21 | 40.67 | 40.67 | 40.97 |
| 0.001 | 0.2 | 41.88 | 41.43 | 40.52 | 41.12 |
| 0.001 | 0.4 | 41.88 | 41.73 | 40.52 | 41.58 |
| 0.003 | 0.1 | 40.52 | 40.67 | 42.03 | 42.49 |
| 0.003 | 0.2 | 40.67 | 42.19 | 41.88 | 40.21 |
| 0.003 | 0.4 | 41.43 | 40.36 | 41.43 | 41.27 |
| 0.010 | 0.1 | 42.64 | 42.19 | 41.12 | 42.03 |
| 0.010 | 0.2 | 40.21 | 42.34 | 42.19 | 40.82 |
| 0.010 | 0.4 | 41.73 | 40.06 | 42.03 | 40.52 |
| 0.030 | 0.1 | 42.03 | 40.82 | 41.73 | 42.19 |
| 0.030 | 0.2 | 41.88 | 40.36 | 41.88 | 40.52 |
| 0.030 | 0.4 | 39.91 | 40.97 | 39.76 | 39.76 |
| 0.100 | 0.1 | 41.27 | 41.12 | 41.27 | 44.31 |
| 0.100 | 0.2 | 40.67 | 40.67 | 42.19 | 42.64 |
| 0.100 | 0.4 | 41.58 | 41.73 | 40.67 | 38.54 |
| 0.300 | 0.1 | 40.06 | 41.43 | 42.64 | 40.21 |
| 0.300 | 0.2 | 40.82 | 42.34 | 38.85 | 41.27 |
| 0.300 | 0.4 | 42.03 | 40.52 | 40.67 | 40.67 |
Thank you for the detailed response and additional experiments, which have addressed several of my concerns. One remaining question is whether the authors could report circuit overlap for the answer-only consistency and random CoT token baselines in Table J, to further clarify whether CAD’s improvements specifically arise from increased computation alignment rather than other general effects.
Thank you for the follow-up. The circuit overlap is 7.06% for Answer-only consistency and 7.70% for Random CoT tokens, both lower than CAD. We hope our responses and additional experiments have addressed your concerns. Please let us know if any issues remain that we could clarify.
1/2
We sincerely thank the reviewer for the constructive feedback, which has helped improve the quality and clarity of our manuscript. We will upload the revised manuscript shortly.
To major W1, Q1 and Q2:
We trace each student before and after training on 100 GSM8K test examples (seed 42; Nmax=2048, τnode=95%, τedge=98%) and examine two relationships.
Table A. Sensitivity discrepancy and circuit overlap before → after CAD training.
Table B. Circuit-level validation under the Basic Distillation Setting
Table C. Circuit-level validation with GPT-4.1-mini as the teacher
Table D. Circuit-level validation with Grok-3-mini as the teacher
Table E. Circuit-level validation with Gemma 2-2B as Student Model
To Q3:
We report ablations of answer-only consistency regularization and random CoT-token matching, further supporting computation alignment as the source of CAD's performance gains:
Table J. Ablation study of the sensitivity-discrepancy signal on Basic Distillation Setting
To Q5:
Table K. Necessary extra costs of CAD under Basic Distillation Setting
Table L. Accuracy (%) under controlled computational settings.