acceptodds
Under review as a conference paper at ICLR 2027

A*-Thought-V2: Efficient Latent Reasoning via Geometric Dynamics of LLM

Abstract

Chain-of-Thought (CoT) improves the reasoning ability of Large Language Models (LLMs) but incurs substantial computation and context costs. Existing methods either lose intermediate information through hard pruning or lack a principled criterion for continuous compression. We present A*-Thought-V2, a geometric dynamics of LLM guided framework that models CoT as a hidden-state trajectory and replaces hard deletion with an explicit–implicit interleaved latent architecture. After projecting question, step, and solution representations into a 3D PCA space, it measures alignment between each local transition and global question-to-solution direction. Aligned steps remain explicit text, whereas deviating steps are compressed into continuous latent tokens. Directional angles capture both local semantics and reasoning dynamics: small angles indicate direct execution and answer formation, while large angles more frequently involve checking, correction, and branch exploration; their temporal variation reveals exploration, convergence, and refinement stages. To train this architecture, we introduce stepwise embedding forcing, which pools each redundant step into a single latent embedding, and label forcing, which supervises that latent token with a soft multi-modal vocabulary distribution instead of a hard one-hot label. Experiments on Qwen3.5-9B and Qwen3.6-27B across six in-domain and out-of-domain benchmarks show that A*-Thought-V2 improves average accuracy by up to 2.6% while reducing response length by up to half, increasing Accuracy per Computation Unit by 2.29, and reducing preprocessing and training time by 94.6% and up to 80.3%, respectively. Representation analyses suggest that latent states form a compact region distinct from textual states, while higher entropy at latent-token positions reflects broader soft targets that encourage richer step-level feature learning.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Latent Reasoning, Efficient Reasoning, CoT Compression

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_k4ZdOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper extends AThought, a method that shortens chain-of-thought by directly pruning reasoning steps. The V2 version keeps some reasoning steps in text form, while turning every other step into one continuous latent token. It uses a geometric criterion to pick which steps to preserve. By default, a frozen small model (Qwen3.5 0.8B) creates embeddings for the question, each reasoning step and the final solution. PCA reduces these embeddings to three dimensions, and the angle of each step’s movement is measured against the overall vector from question to solution. Steps with angles at or below the threshold (60 or 90 degrees) remain text; steps exceeding the threshold get compressed. Two training losses enable learning for these latent tokens. Embedding forcing takes the mean embedding of tokens from a compressed step and uses it as the latent input. Label forcing trains each latent position to predict the average one-hot target of tokens in that step. During inference, the model first outputs a start tag. It then feeds its own previous final-layer hidden state as the next input, repeating this for a capped number of latent positions, until it outputs an end tag. For experiments, the authors fine-tuned Qwen3.5 9B and Qwen3.6 27B for 3 epochs on 3k OpenR1 math traces. They tested on Math500, AIME 2024, 2025, 2026, ARC Challenge and GPQA Diamond, with four sampled runs each. Key findings: up to 2.6 points higher accuracy than vanilla SFT trained on the same dataset, around 16% shorter generated outputs, 2.29× better accuracy per computation unit (ACU), and 94.6% less preprocessing time than AThought.

Critical strengths

First, the main table shows a consistent gain over the right control. On both backbones the 90 degree variant beats same data SFT on average accuracy (91.5 vs 89.0 on 9B, 93.9 vs 92.9 on 27B) with shorter outputs (about 16 percent fewer tokens on both), and it beats SwiReasoning and CopT on average accuracy and ACU. Getting both accuracy and length to move the right way at once is what makes efficient reasoning methods useful.

Secondly, the ablation in Table 2 separates the pieces. Reversed selection (compress the aligned steps, keep the deviating ones) and random angle selection both do worse at nearly the same compression rate (48.09, 48.16 and 47.15 percent), and dropping label forcing drops AIME accuracy on 27B from 94.5 to 72.5. That is evidence that the loss design matters and that the ordering of what gets compressed is not arbitrary.

Critical weaknesses

First, the training and inference inputs to the latent positions do not match, and the paper does not examine it. During training a latent position receives the mean of the step's token embeddings (Eq. 3 and Algorithm 1). During inference it receives the previous final layer hidden state (Eq. 11). These live in different spaces, and nothing shows that the model's hidden states at latent positions stay close to the pooled embeddings it was trained on. The ablation where removing embedding forcing barely hurts (92.8 vs 94.5) and removing label forcing hurts a lot (72.5) suggests the latent interface is brittle. The paper also never decodes or probes the latent states to show they preserve the content of the compressed steps. The only evidence is that latent hidden states cluster separately from text hidden states (Figure 4) and have high entropy (Figure 5), which the paper itself says mostly reflects the soft target.

Secondly, the eval has noise and selection problems that make the headline numbers hard to trust. The ACU of 2.29x compares against the untuned Qwen3.6 27B backbone (0.35 to 0.80). Against same data SFT the 27B ACU goes from 0.66 to 0.80, about 1.2x, and accuracy goes up 1.0 point. That backbone baseline also looks off: it scores 70.9 on Math500, below the smaller Qwen3.5 9B at 86.6, which points to a truncation or answer extraction problem in the baseline that makes both the ACU and accuracy gains look larger. AIME has 30 problems per year, and with four runs the table gives no standard deviations, so differences of one or two points are less than one problem. The A*Thought baseline falls far below plain SFT (78.7 vs 89.0 average on 9B, with AIME 2024 at 70.0 vs 90.8), which is surprising for a published improvement and suggests it was reproduced in a setting that works against it. Finally, the latent weight lambda is chosen by a grid search while "monitoring latent token generation across benchmarks" (Appendix D), and the max latent length ablation (Figure 6) uses AIME 2024 and GPQA Diamond, which are also the test sets. Selecting settings on the evaluation benchmarks makes the reported improvements optimistic.

Questions for authors

1/ Please report standard deviations over the four runs in Table 1, and the plain SFT ACU next to the V2 ACU on both backbones. 2/ Why does Qwen3.6 27B score 70.9 on Math500 while Qwen3.5 9B scores 86.6? Is output truncated or extraction failing for the base model? 3/ How was the A*Thought baseline trained, and why does it fall below plain SFT on AIME?

Reviewer_4wjFOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes A*-Thought-V2, a method to shorten Chain-of-Thought reasoning. It represents the CoT as a trajectory of the LLM's hidden states, projects the question, each step, and the answer into a 3D PCA space, and measures the angle between each local step direction and the global question-to-answer direction. Steps with a small angle stay as normal text. Steps with a large angle are compressed into a single continuous latent token. Two training mechanisms support this: Embedding Forcing, which averages the token embeddings of a step into one latent input, and Label Forcing, which supervises that latent token with a soft, averaged vocabulary distribution.

Critical strengths
  1. The geometric criterion is a simple, interpretable, and more principled selection rule than hard pruning.
  2. The explicit–implicit interleaved design keeps compressed steps as latent tokens instead of deleting them, and Label Forcing is a reasonable way to supervise a many-token target with one token.
  3. Efficiency reporting is detailed, and the representation analyses give some insight into what latent tokens encode.
Critical weaknesses
  1. The method depends on a frozen external extractor and on pre-segmented reasoning steps. Sensitivity to the segmenter and to the extractor's representation space is only partly studied.
  2. The six 30-degree bins and their semantic/lexical labels are hand-defined and descriptive; there is no formal guarantee that this partition generalizes beyond math.
  3. The soft label is an order-agnostic mean of one-hot vectors, so token order and position weighting are lost; a single latent token may also be too small to hold numeric details.
Questions for authors
  1. How are reasoning steps segmented? Is it manual or automatic, and how sensitive are the results to segmentation errors?
  2. How was τ selected, and how does performance change for other values? Could τ be made adaptive?
  3. Aside from the angle distribution, how much does the extractor size change final accuracy?
Reviewer_cbEmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

A*-Thought-V2 shortens chain-of-thought reasoning by replacing selected text steps with continuous vectors. A frozen extractor represents the question, reasoning steps, and reference solution. A separate three-dimensional PCA projection for each training trajectory measures how each step aligns with the question-to-solution direction. Aligned steps remain text; other steps are compressed. Training supplies the mean input embedding of each compressed step and supervises its latent position with the average token-label distribution. At inference, latent inputs instead come from the model's preceding hidden states, with boundary tags controlling the return to text generation. Experiments use two Qwen backbones and six benchmarks, alongside same-data fine-tuning controls, selection and supervision ablations, and cost analyses. The results support practical compression, while the interpretation of the latent states and the stability of the geometric criterion need stronger evidence.

Critical strengths

1.The method improves on fine-tuning with the same training data. Table 1 includes fine-tuning on the same OpenR1-Math-3k data, which helps separate the compression benefit from simply using additional training examples. The 90-degree variant improves average accuracy from 89.0% to 91.5% for the 9B model and from 92.9% to 93.9% for the 27B model, while shortening responses. The latter reduction is about 16%. Table 3 also reports preprocessing and training-time savings. These are useful practical results beyond a comparison with an untouched backbone.

2.The ablations support both the step-selection policy and soft-label supervision. Table 2 compares geometry-based selection with random-angle and reversed-selection variants at similar compression rates. Average AIME accuracy is 94.5%, compared with 92.8% and 91.4%, respectively. Removing Label Forcing lowers accuracy substantially, while removing Embedding Forcing has a smaller adverse effect. These controls show that the tested selection policy and supervision contribute to performance; the paper is not relying only on a comparison against deleting reasoning steps.

Critical weaknesses

1.The evidence does not directly establish what information the generated latent states preserve. Equation (3) uses pooled reference-step embeddings as training inputs, while Equation (11) feeds back model-generated hidden states at inference. This difference is not itself an error, and an explicit state-matching loss is not inherently required. However, the soft labels in Equations (8)-(10) represent token frequencies rather than within-step order, so fitting them does not by itself establish preservation of the compressed reasoning. The representation plots and latent-length analysis in Sections 5.2-5.4 are useful, but do not directly test that claim either. The positive task results establish empirical utility more clearly than they establish information-preserving latent compression. A targeted intervention on latent states, or a clearer analysis connecting the two input regimes, could change this assessment.

2.The geometric rule is supported as a selection heuristic more strongly than as an explanation of reasoning dynamics. Section 2 and Appendix B associate angle ranges with execution, checking, and correction. These observations do not establish that a step deviating from the solution direction is unnecessary or especially suitable for compression. Table 2 provides meaningful random/reversed controls, but the rule is applied after a separate three-dimensional projection for each trajectory. The paper does not test whether step choices and downstream gains remain stable in the original representation or at other PCA dimensions. Appendix C.1 examines extractor size, which is a different sensitivity. This limits the claimed general geometric interpretation. Together with W1, it leaves me leaning toward weak rejection despite the useful compression results.

Questions for authors

1.What evidence shows that the model-generated latent states preserve the compressed steps, beyond fitting token-frequency targets and reproducing latent-span lengths? Please clarify the prediction/target alignment at latent boundaries and provide any existing controls that distinguish carrying useful information from merely adding latent computation.

2.How stable are the selected steps and accuracy-length trade-offs when angles are computed in the original representation or at other PCA dimensions? This would clarify whether the criterion depends strongly on the particular projection.

3.Relative to same-data SFT, what are the measured end-to-end inference time and memory costs, including latent updates? The reported 2.29-fold ACU improvement uses the original backbone as reference; ACU is an accuracy-to-length ratio, so actual runtime against same-data SFT would better establish the deployment benefit.

1 comment

The additional intervention experiments conducted by the authors are valuable. However, random replacement, zeroing out, and cross-sample replacement result in only relatively small decreases in average accuracy.

Furthermore, the performance of the 3D and 1024-dimensional representations is very similar, despite approximately 25% disagreement in their step-selection decisions. The paper's central hypothesis is that the geometric direction of a reasoning step can reveal its reasoning characteristics and, consequently, determine whether that step is suitable for compression. If this is the case, why do the two representations produce substantially different step-selection decisions while achieving nearly identical performance? Is the use of 3D PCA actually necessary in this work?

I do not expect the authors to conduct any additional experiments. I would simply appreciate clarification on these two remaining concerns. If the authors can provide reasonable explanations, I would be willing to reconsider my current rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.