- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes CATER to shorten reasoning while maintaining accuracy. It samples several responses per problem and uses their correctness to choose the training objective. All-correct groups receive concise self-distillation, mixed-outcome groups receive correctness-based reinforcement learning, and all-wrong groups are skipped. The method also periodically updates the teacher and balances the two training gradients. Experiments report average reasoning-token reductions of 39.1% on Qwen3-4B and 24.3% on Qwen3-1.7B, with overall accuracy close to the base models.
Critical strengths
-
The problem is useful and the motivation is reasonable.
Reasoning models often produce unnecessarily long answers. Compressing problems the model already solves reliably, while focusing on correctness for less reliable problems, is a sensible approach. The two training objectives fit together naturally. Mixed-outcome groups provide correct and incorrect responses for RL to learn from. All-correct groups provide little correctness-learning signal, but can instead be used to learn shorter reasoning. Online routing allows these assignments to change as the model improves.
-
The experiments support the method’s practical value.
The method reduces token use across several tasks and model sizes. The ablations show benefits from online routing and combining RL with distillation. The paper also studies several important hyperparameters, including rollout count, distillation strength, and teacher refresh interval.
Critical weaknesses
-
The contribution over closely related work needs stronger support.
CEEH and PACE already study how to compress easier problems while preserving reasoning on harder ones. CRISP already uses a conciseness-conditioned self-teacher. CATER adds an explicit rule for choosing between distillation and RL based on online correctness. The routing ablations support this addition, but comparisons with CEEH and PACE are missing. It is therefore difficult to judge whether CATER improves on existing difficulty-aware compression methods.
-
The paper shows that routing helps, but provides less evidence for its explanation of why it helps.
The central idea is that reliably solved problems can be compressed with less risk. However, the paper does not directly measure how compression affects problems at different success rates. Figure 6 shows that more groups become all-correct during training, but does not show whether those problems subsequently become shorter without losing accuracy. Such an analysis would strengthen the main explanation. The current theory provides motivation, but does not directly establish this behavior for the proposed training method.
Questions for authors
None
3 comments
Thank you for your clarification and additional results. I have no more concerns.
We thank the reviewer for the constructive feedback and are glad that the clarifications and additional results have addressed your concerns. Your feedback has been helpful in improving our paper.
We thank the reviewer for the constructive suggestions.
Q1. How does CATER compare with closely related difficulty-aware compression methods such as CEEH and PACE?
A. CATER combines explicit competence-based objective allocation with dense token-level compression guidance. Verified student outcomes determine which questions receive concise self-distillation, while correctness learning strengthens mixed-outcome questions.
We added CEEH and PACE. All methods use Qwen3-4B-Thinking-2507, rank-64 LoRA, and 32 questions × 6 responses per update, and are evaluated at the same training step. For CEEH, we initialize the reference cache and compute the difficulty threshold over initialized entries, corresponding to the warm-active variant. For PACE, we implement the paper’s token-prefix formulation and reward function with the K40 schedule.
Table 8. Comparison with CEEH and PACE.
As shown in Table 8, CATER achieves 1.25pt higher AIME25 accuracy with shorter reasoning than CEEH. Compared with PACE, it retains 10.42pt higher AIME25 accuracy and 4.04pt higher GPQA-D accuracy, while using longer reasoning. These comparisons demonstrate CATER’s favorable balance between accuracy and reasoning efficiency, supporting the value of coordinating dense concise supervision with correctness learning.
Q2. Do reliably solved problems become shorter without losing accuracy, and how does compression affect problems with different initial success rates?
A. We evaluated the same 128 training questions before and after training, retaining their initial grouping. The set was fixed before inspecting post-training results. Among them, 72 questions had six initially correct rollouts. Each model then generated eight fresh responses per question under a 32,768-token output budget.
Table 9. Reliable-question compression under a 32,768-token output budget.
As shown in Table 9, initially reliable questions receive 44.97% shorter reasoning while retaining 100.00% observed accuracy, across 576 responses per model. Truncation is zero for both models in this group. These measurements directly show the behavior motivating CATER: reasoning on already reliable questions becomes substantially more concise while retaining observed correctness.
We also evaluated the same questions under the 10,000-token training budget, with 16 fresh responses per question and model.
Table 10. Before/after diagnostics under a 10,000-token output budget.
The high-truncation group is defined by a pre-training truncation rate of at least 25% and overlaps the other groups. Under the training budget, Table 10 shows that initially all-correct questions receive 40.65% shorter complete outputs with higher observed accuracy. In the high-truncation group, shorter outputs enable more correct answers within the same budget. The long-budget analysis separately demonstrates compression with zero truncation in the reliable group. Together, these before/after diagnostics complement the routing proportions in Figure 6 of the paper with direct measurements of accuracy and length.