- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper extends AThought, a method that shortens chain-of-thought by directly pruning reasoning steps. The V2 version keeps some reasoning steps in text form, while turning every other step into one continuous latent token. It uses a geometric criterion to pick which steps to preserve. By default, a frozen small model (Qwen3.5 0.8B) creates embeddings for the question, each reasoning step and the final solution. PCA reduces these embeddings to three dimensions, and the angle of each step’s movement is measured against the overall vector from question to solution. Steps with angles at or below the threshold (60 or 90 degrees) remain text; steps exceeding the threshold get compressed. Two training losses enable learning for these latent tokens. Embedding forcing takes the mean embedding of tokens from a compressed step and uses it as the latent input. Label forcing trains each latent position to predict the average one-hot target of tokens in that step. During inference, the model first outputs a start tag. It then feeds its own previous final-layer hidden state as the next input, repeating this for a capped number of latent positions, until it outputs an end tag. For experiments, the authors fine-tuned Qwen3.5 9B and Qwen3.6 27B for 3 epochs on 3k OpenR1 math traces. They tested on Math500, AIME 2024, 2025, 2026, ARC Challenge and GPQA Diamond, with four sampled runs each. Key findings: up to 2.6 points higher accuracy than vanilla SFT trained on the same dataset, around 16% shorter generated outputs, 2.29× better accuracy per computation unit (ACU), and 94.6% less preprocessing time than AThought.
Critical strengths
First, the main table shows a consistent gain over the right control. On both backbones the 90 degree variant beats same data SFT on average accuracy (91.5 vs 89.0 on 9B, 93.9 vs 92.9 on 27B) with shorter outputs (about 16 percent fewer tokens on both), and it beats SwiReasoning and CopT on average accuracy and ACU. Getting both accuracy and length to move the right way at once is what makes efficient reasoning methods useful.
Secondly, the ablation in Table 2 separates the pieces. Reversed selection (compress the aligned steps, keep the deviating ones) and random angle selection both do worse at nearly the same compression rate (48.09, 48.16 and 47.15 percent), and dropping label forcing drops AIME accuracy on 27B from 94.5 to 72.5. That is evidence that the loss design matters and that the ordering of what gets compressed is not arbitrary.
Critical weaknesses
First, the training and inference inputs to the latent positions do not match, and the paper does not examine it. During training a latent position receives the mean of the step's token embeddings (Eq. 3 and Algorithm 1). During inference it receives the previous final layer hidden state (Eq. 11). These live in different spaces, and nothing shows that the model's hidden states at latent positions stay close to the pooled embeddings it was trained on. The ablation where removing embedding forcing barely hurts (92.8 vs 94.5) and removing label forcing hurts a lot (72.5) suggests the latent interface is brittle. The paper also never decodes or probes the latent states to show they preserve the content of the compressed steps. The only evidence is that latent hidden states cluster separately from text hidden states (Figure 4) and have high entropy (Figure 5), which the paper itself says mostly reflects the soft target.
Secondly, the eval has noise and selection problems that make the headline numbers hard to trust. The ACU of 2.29x compares against the untuned Qwen3.6 27B backbone (0.35 to 0.80). Against same data SFT the 27B ACU goes from 0.66 to 0.80, about 1.2x, and accuracy goes up 1.0 point. That backbone baseline also looks off: it scores 70.9 on Math500, below the smaller Qwen3.5 9B at 86.6, which points to a truncation or answer extraction problem in the baseline that makes both the ACU and accuracy gains look larger. AIME has 30 problems per year, and with four runs the table gives no standard deviations, so differences of one or two points are less than one problem. The A*Thought baseline falls far below plain SFT (78.7 vs 89.0 average on 9B, with AIME 2024 at 70.0 vs 90.8), which is surprising for a published improvement and suggests it was reproduced in a setting that works against it. Finally, the latent weight lambda is chosen by a grid search while "monitoring latent token generation across benchmarks" (Appendix D), and the max latent length ablation (Figure 6) uses AIME 2024 and GPQA Diamond, which are also the test sets. Selecting settings on the evaluation benchmarks makes the reported improvements optimistic.
Questions for authors
1/ Please report standard deviations over the four runs in Table 1, and the plain SFT ACU next to the V2 ACU on both backbones. 2/ Why does Qwen3.6 27B score 70.9 on Math500 while Qwen3.5 9B scores 86.6? Is output truncated or extraction failing for the base model? 3/ How was the A*Thought baseline trained, and why does it fall below plain SFT on AIME?
The additional intervention experiments conducted by the authors are valuable. However, random replacement, zeroing out, and cross-sample replacement result in only relatively small decreases in average accuracy.
Furthermore, the performance of the 3D and 1024-dimensional representations is very similar, despite approximately 25% disagreement in their step-selection decisions. The paper's central hypothesis is that the geometric direction of a reasoning step can reveal its reasoning characteristics and, consequently, determine whether that step is suitable for compression. If this is the case, why do the two representations produce substantially different step-selection decisions while achieving nearly identical performance? Is the use of 3D PCA actually necessary in this work?
I do not expect the authors to conduct any additional experiments. I would simply appreciate clarification on these two remaining concerns. If the authors can provide reasonable explanations, I would be willing to reconsider my current rating.