- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies how different CoT compression methods affect training outcomes in SFT and subsequent RL. It compares uncompressed trajectories with Chain-of-Draft and CRISP trajectories of similar average length. Chain-of-Draft achieves higher SFT and post-RL accuracy than CRISP, particularly on problems that the base model solves inconsistently, and the paper further analyzes differences in the reasoning content retained by the two methods.
Critical strengths
-
CoD outperforms CRISP by 12.78 percentage points on AIME after SFT with Qwen3-8B, despite the two datasets having similar average response lengths.
-
The paper evaluates the effects of compressed CoT data through both SFT and subsequent RL, and CoD maintains higher post-RL accuracy than CRISP on Qwen3-8B and Qwen3-4B.
Critical weaknesses
The CoD and CRISP datasets are not fully matched. They differ in the generating model, candidate-selection procedure, target correctness, and final dataset membership. Consequently, these factors may also contribute to the observed performance differences.
Questions for authors
-
CoD and CRISP differ in their generating models, candidate-selection procedures, target correctness, and final dataset membership. How can the authors determine how much of the performance gap is caused specifically by the reasoning content retained during compression?
-
The main SFT and RL experiments use only one training seed. Are the reported differences between CoD and CRISP stable across multiple training runs, particularly the 2.08-point post-RL gap on Qwen3-8B?
-
All SFT variants underperform the corresponding base models, and the SFT-initialized RL models underperform Direct RL in the reported Qwen experiments. How should the practical benefit of compressed-CoT SFT be interpreted under these results?
3 comments
Response (Part 2/2)
2. Robustness across random seeds
We use one training seed and do not yet have multi-seed RL results. Repeating the full pipeline is costly, so we prioritized comparisons across models and SFT training depths under limited sources.
Although we do not use multiple training seeds, we broadened the evaluation along several dimensions to examine consistency and generality. The mathematical SFT experiments cover Qwen3-8B, Qwen3-4B, and Olmo-3-7B-Think. CoD outperforms CRISP by 12.78, 8.06, and 7.22 percentage points, respectively. Both Qwen models retain this ordering after RL, as do all four tested SFT initialization depths on Qwen3-8B. These provide consistency checks across models and initializations. Eight responses per AIME problem also reduce single-generation variability across the 90 problems.
3. Practical benefit relative to Base and Direct RL
The current results do not establish that these SFT-to-RL configurations outperform Direct RL. Our main goal is to analyze how different compressed datasets affect SFT and subsequent RL. Their benefits should be interpreted in terms of capability preservation, response length, and final training outcomes.
One possible explanation is that offline SFT trajectories do not fully match the strong base models' existing reasoning behavior or the evaluation demands. Fitting these trajectories may alter that behavior. Subsequent RL starts from different policies, and a shared budget need not remove the effects of SFT initialization. These explanations require targeted validation.
CoD nevertheless preserves capabilities better than Raw supervision in this setting. On Qwen3-8B, it reduces the accuracy drop relative to Base from 18.20 percentage points with Raw-SFT to 4.03 points. Thus, while CoD does not exceed Base, it substantially mitigates the decline seen with Raw supervision under the current setup.
The results also show a trade-off between accuracy and response length. As reported in Appendix Table 9, Qwen3-8B CoD-SFT generates an average of 12,618 tokens on the evaluation set, compared with 15,672 for Base. It uses fewer output tokens while retaining accuracy relatively close to Base. This offers a potential inference-efficiency benefit, despite the modest loss in accuracy.
Thank you again for your suggestions. We will strengthen the controlled comparisons and stability check in future work.
Thanks for the detailed response. The clarification on dataset construction and the deletion-position ablation address most of my concern about the CoD–CRISP comparison. I still think multi-seed RL results would be useful, but the consistent trends across models and SFT depths provide some reassurance. The practical role of compressed-CoT SFT is also clearer now. I will raise my rating.
Response (Part 1/2)
Thank you for your careful review and comments. We address your questions below.
1. Differences between the CoD and CRISP datasets
Both methods start from Qwen3-8B. CoD uses concise prompting, whereas CRISP first updates the model through OPSD and then generates SFT data. The change in weights is part of CRISP, not a difference in the initial backbone.
Rejection sampling with answer verification retains 59,101 verifier-approved, non-truncated CRISP responses. Since the CRISP data have already been filtered for correctness, we construct CoD data on these same questions. The goal is to match target correctness as closely as possible while preserving common question coverage. In each round, we retain correct, non-truncated responses. Only questions without an accepted response proceed to the next round. Generation stops for a question as soon as an accepted response is obtained.
For questions still without a verifier-approved response after four rounds, we retain the shortest non-truncated response as a fallback. This avoids changing the question set by removing these questions. Only 224 questions with all candidates truncated are excluded (0.38%). CoD retains 99.62% of the common question set, keeping dataset sizes closely matched.
CoD has a lower proportion of verifier-approved targets but better mathematical SFT and subsequent RL results. Its advantage therefore cannot be explained simply by a higher proportion of verified targets.
The common initial backbone and core training settings, together with closely matched question coverage and dataset sizes, aim to reduce influences other than reasoning content.
Section 5.2 and Appendix B.3 provide additional evidence through a deletion-position ablation (Figure 4b, Table 8). Both conditions use the same 4,096 Raw responses, initial model, and SFT settings, with matched deletion amounts per example. Removing different parts still yields different training outcomes, supporting the importance of retained content.