- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes Latent-Then-Explicit Reasoning (LaTER). A reasoning LLM first runs a short recurrent latent rollout, feeding final-layer hidden states back as input embeddings without emitting tokens. It then switches once to ordinary chain-of-thought (CoT) decoding and keeps the latent key–value cache.
The paper studies two variants. LaTER-TF is training-free: it uses the LatentMAS linear mapping and leaves the latent phase when the vocabulary-probe entropy exceeds a threshold or the probe argmax is a terminating token (Eq. 4). LaTER-SFT is supervised: it trains a two-layer MLP projector together with the backbone on Latent-Switch-69K. This corpus has 69,745 records. For each one, a teacher (DeepSeek-V3.2) takes a Dolci-Think-SFT-32B trace, extracts a “solution intuition,” and writes a shortened derivation. The intuition sets the latent budget (about positions, at most 128) and also conditions a frozen reference teacher used for KL matching. A halting objective (Eq. 9) targets a train–inference mismatch: training forces the full prescribed rollout, while inference stops at the first predicted </latent_think>.
Experiments cover seven math, science, and code benchmarks. Qwen3-8B/14B are run in both settings; DeepSeek-R1-Distill-Llama-8B and OLMo-3-32B-Think are run training-free only. The paper reports accuracy and generation length, where length counts latent steps plus explicit tokens (Tables 1–4). It also reports loss ablations (Table 5), a threshold sweep (Fig. 4), wall-clock decoding time (Table 6), and qualitative entropy and PCA diagnostics (App. C).
Critical strengths
-
Consistent efficiency gains at a realistic scale, without the accuracy collapse of prior latent methods. On both Qwen3 sizes, LaTER-SFT produces shorter generations than CoT and the same-data CoT-SFT control on every one of the seven benchmarks (Table 1). Mean relative to CoT is −23.3% (8B) and −28.9% (14B); relative to CoT-SFT the mean reduction is 19.9% and 17.8% (Sec. 4.2). The training-free variant also shortens generations on all seven benchmarks for both Qwen models (−17.8% and −23.5%; Tables 3–4). The authors measure wall-clock decoding time with latent computation included, and the token savings carry over: 19.1% faster than CoT and 14.6% faster than CoT-SFT on Qwen3-8B (Table 6; I recomputed both averages from the table). Coconut, CODI, and SIM-CoT, trained on the same corpus, trail LaTER-SFT by 16.8–23.7 points of mean accuracy on Qwen3-8B (Table 2). LaTER-SFT, in contrast, matches or exceeds CoT accuracy on every benchmark for both Qwen sizes (Table 1). These reductions are large and hold for all 14 benchmark–model pairs, so they are the most robust result in the paper. It shows that a latent prefix can be combined with modern long-CoT models without giving up explicit verification.
-
A clearly identified and ablated train–inference halting mismatch. Training always runs the prescribed latent steps, but inference exits at the first predicted stop token, and masked latent-interior CE never penalizes premature stops (Sec. 3.4, App. B.5). The paper diagnoses this and shows that it matters. Without the halting objective, inference exits after 3–5 latent steps, mean savings fall from 28.9% to 4.9%, and mean accuracy falls from 86.6% to 84.1%; the full model is better on every benchmark (Table 5, Sec. 5.2). The observation is simple, and it applies to any latent-reasoning method that learns when to stop under teacher-forced rollouts.
Critical weaknesses
-
The accuracy improvements are not statistically established, and the evaluation protocol may favor shorter outputs. The abstract (lines 24–27) and the introduction (lines 89–93) state that LaTER improves accuracy. Four issues weaken this claim.
-
One sampled run. Evaluation uses a single random seed with temperature-0.6 sampling (App. A.2, lines 678–680). Every AIME 2025 accuracy in Tables 1–5 is a multiple of , which is consistent with one sample per problem. The headline 14B gain, 70.0%
<!-- -->{=html}80.0% (Table 1), is therefore 3 of 30 problems, and the LaTER-TF gain of 70.0%<!-- -->{=html}73.3% (Table 4, Fig. 1) is a single problem. For scale, a single run at on 30 problems has a binomial standard error of about 8.4 points. Paired comparisons are tighter, but the paper gives no per-problem results, repeated runs, variances, or confidence intervals with which to assess them. On Qwen3-8B, LaTER-SFT beats the same-data CoT-SFT control by only 0.3 points of mean accuracy, winning on four benchmarks and losing on three (Table 1; lines 300–301). Most of the 8B gain over CoT (75.9$\to$<!-- -->{=html}79.4) is already reached by CoT-SFT (79.1). -
Threshold chosen on the test set. For Qwen3-14B on AIME 2025, the LaTER-TF result in Table 4 and Fig. 1 (73.3%, 10,661 tokens) is the best point of a sweep run on the same 30 test problems (Fig. 4b; Sec. 5.1). Reading the values from Fig. 4b, gives about 66.7, 63.3, 70.0, 70.0, 73.3, and 66.7%, so only the selected value beats CoT. I could not find the used for the other benchmarks and backbones in Tables 3–4.
-
Generation caps that may bind. Caps are 20,000 positions for AIME and 8,192 for GPQA (App. A.2, lines 682–685), and mean CoT lengths reach 73–79% of them: 14,672 (8B) and 15,730 (14B) on AIME, and 6,039 and 6,301 on GPQA (Table 1). Truncation rates are not reported. Under such caps, a method that shortens outputs can gain accuracy simply because more answers finish within the cap. The Qwen3-8B CoT baseline scores 53.3% on AIME 2025 and 47.0% on GPQA Diamond. These are well below the thinking-mode numbers in the Qwen3 technical report (67.3% and 62.0%). That report used caps of 38,912 and 32,768 tokens and averaged 64 and 10 samples, so the gap is not an error in itself. It is, however, consistent with a baseline running in a cap-constrained regime, though it does not prove this. The 14B baseline, by contrast, is close to the report (70.0 vs. 70.4 and 62.6 vs. 64.0). In Table 4, OLMo-3-32B-Think’s mean CoT length on GPQA Diamond is 10,460 tokens. That exceeds the stated 8,192 cap, so the protocol for that row is unclear.
-
Generality across backbones is overstated. Sec. 4.3 (lines 323–367) says that “benefits also extend beyond Qwen” and cites only GSM8K. Across all seven benchmarks in Table 4, LaTER-TF lowers mean accuracy on DeepSeek-R1-Distill-Llama-8B (66.2$\to$
<!-- -->{=html}65.5; 3 wins, 4 losses) and on OLMo-3-32B-Think (82.9$\to$<!-- -->{=html}82.2; 2 wins, 2 ties, 3 losses). It also lengthens ARC-Challenge generations on both models (by 16.5% and 11.5%).
These issues do not affect the token reductions, but as reported the accuracy gains cannot be told apart from sampling noise, test-set threshold selection, or truncation effects. The fix is straightforward but requires new runs: multi-sample evaluation (e.g., avg@16 on AIME and several seeds elsewhere) with paired confidence intervals, truncation rates per method (or results at the models’ recommended budgets), and chosen on held-out data.
-
-
No experiment isolates what the continuous latent prefix contributes, and the closest prior work is under-positioned. The paper’s central claim is that the latent prefix “prepares” a shorter explicit derivation and that latent and explicit reasoning are complementary (abstract; lines 96–102). Four gaps remain.
-
The prefix is short, and its inference length is not reported. Training budgets have a mean of 41.49 positions and a median of 40 (Table 7, App. B.3), while explicit continuations run to thousands of tokens. The claim therefore rests on the prefix changing how the explicit part is written, not on the prefix replacing text. The paper does not report the number of latent steps that LaTER-SFT or LaTER-TF actually take at inference. The only reported figure is 3–5 steps for the no-halt variant (line 424).
-
The explicit control differs from LaTER-SFT in more than one way. CoT-SFT shortens outputs much less than LaTER-SFT (−4.0% and −13.3%; Table 1), which is suggestive. But App. A.3 (lines 697–698) says only that CoT-SFT “removes the latent segment, projector, and halting loss.” It does not say whether CoT-SFT keeps the KL term to the intuition-conditioned teacher or the weighting. If CoT-SFT has no KL term, the like-for-like comparison is the no-KL variant. On Qwen3-14B that variant beats CoT-SFT by 1.1 points of mean accuracy, and it loses 3.5 points on GPQA (Table 5 vs. Table 1).
-
Key controls are missing. I would expect four: (i) a prefix of placeholder or pause tokens with the same budget and losses, analogous to Coconut’s “pause as thought” ablation (Hao et al., 2025); (ii) a variant that writes the intuition (about 80 tokens) explicitly before the shortened derivation; (iii) in the training-free setting, a prefix of the same length made of non-informative embeddings; and (iv) explicit length-reduction baselines run at LaTER’s token budget. Control (iii) matters because the paper itself observes that probe argmaxes drift toward terminating tokens such as
</think>(App. C.1). A generic “already-thought” perturbation could therefore shorten the continuation. The latent baselines in Table 2 run at a mean of −58.6% to −94.0%. They show that keeping explicit CoT preserves accuracy, but they do not show that LaTER improves the tradeoff at its own operating point (about −23%). -
Positioning against prior work. Section 6 (lines 465–466) says that training-based latent methods “focus primarily on compressing explicit reasoning into continuous representations.” Three prior works are closer to LaTER than this suggests. In Coconut’s curriculum, at stage the first reasoning steps are continuous thoughts and the rest are in language. Coconut’s analysis also runs latent thoughts and then completes the solution in language. Token Assorted (Su et al., 2025; arXiv:2502.03275) replaces the initial reasoning steps with discrete (VQ-VAE) latent tokens followed by text; the paper does not cite it. SwiReasoning (Shi et al., 2026) is training-free and switches between latent and explicit reasoning based on entropy trends, which makes it the closest training-free method. The paper discusses it only in Related Work and leaves it out of the training-free experiments. Table 3 compares against the test-time optimization methods LatentSeek and LTPO, and Table 4 compares only with CoT.
Without these controls, the gains could equally come from the shortened-derivation data, the intuition-conditioned KL, or simply the extra positions, rather than from latent reasoning. This is fixable with the controls above and a sharper novelty statement. In my view the real novelty lies in adaptive single-switch timing at scale and in the halting objective.
-
Minor points. The Conclusion (line 480) says LaTER uses “one third fewer tokens than standard CoT,” but the paper’s own mean is −23.3% (8B) and −28.9% (14B) (Table 1). Figure 1 computes its percentages from rounded values: for MATH-500 it shows −18%, whereas Table 4 gives 3,472$\to$<!-- -->{=html}2,887, i.e., −16.8%. The HumanEval+ entry of 92.2% (Table 1, 14B LaTER-SFT) does not match any integer count out of 164 problems. The PCA diagnostics (App. C.3) are qualitative and cover a few samples, with no stated selection criterion. If PC1 is computed jointly over latent and explicit states, as the shared axes suggest, a separated latent cluster alone can raise the PC1 fraction. The claim that LaTER “constrains the subsequent explicit reasoning trajectory to a lower-dimensional region” (lines 1007–1009) therefore needs a PCA over explicit states only. Typos: “doceded token” (Fig. 1), “An intuition of tokens assign” (line 195), and “Follow Equation 3” (line 855).
Questions for authors
-
[Clarification] Repeated-sample statistics. How many samples per problem were drawn, and which methods were decoded greedily? App. A.2 leaves the latter open. Please report mean s.d. over several seeds, or avg@ (e.g., on AIME 2025), with paired confidence intervals for LaTER-SFT vs. CoT-SFT and for LaTER-TF vs. CoT. Please also give the number of evaluated problems per benchmark (see the HumanEval+ point above). If the 14B gains over CoT-SFT (+2.8 mean, +6.7 on AIME) survive this, my assessment of the accuracy claims and my rating would go up. If they fall within noise, I would read the paper as an efficiency result only.
-
[Clarification] Truncation. For each method and benchmark, especially AIME and GPQA, what fraction of responses hit the generation cap? How do the results change at the models’ recommended budgets (e.g., 32k/38k tokens)? Why does the OLMo GPQA CoT mean (10,460) exceed the 8,192 cap? What explains the gap between the Qwen3-8B CoT baseline and published thinking-mode results? If the accuracy gains hold with caps that do not bind, the claim that LaTER improves reasoning, rather than just keeping answers within budget, would be much stronger.
-
[Clarification] Threshold selection. Which was used for each benchmark and backbone in Tables 3–4, and was it chosen on test data? What accuracy do two alternatives reach: the terminating-token-only rule (), and a single fixed in advance or tuned on a held-out split? What fraction of exits does each criterion trigger? Most latent entropies in Fig. 7 lie well below the swept range of 3–8, so I suspect the terminating-token criterion triggers most exits. If a fixed or held-out gives similar results, my concern about LaTER-TF would largely go away.
-
[Clarification] Isolating the latent prefix. Does CoT-SFT include the KL term to the intuition-conditioned teacher, and the weighting? Could the authors add three controls: (i) a pause or placeholder prefix with the same budget and losses, (ii) an SFT variant that writes the intuition explicitly, and (iii) a training-free prefix of non-informative embeddings with the same length? If LaTER clearly beats these at similar total cost, the claim that latent and explicit reasoning are complementary would be supported. If it does not, the gains are better attributed to the data and objectives.
-
[Clarification] Training and inference details. App. B.6 specifies a learning rate of decaying to over one epoch. Does the newly introduced projector use the same rate, and how is it initialized (randomly, or from )? What is the inference-time distribution of the number of latent steps per benchmark for LaTER-SFT and LaTER-TF, and is there a maximum latent budget at inference? What unit does Table 6 report (per problem, per batch, or per benchmark), and does LaTER use the same
model.generate()path as the baselines? If the projector barely moves from its initialization at this learning rate, the description of a “learned projector” would need revisiting. -
[Suggestion / future work] Please compare against the closest training-free methods, SwiReasoning and Soft Thinking, on the same backbones and caps. Please also compare against length-matched explicit baselines, such as budget-forced or early-exit CoT at LaTER’s token count. When stating the novelty, please discuss Coconut’s partially latent curriculum stages and Token Assorted.
7 comments
Response to Reviewer aEL3 (Part 2/4)
Both LaTER variants hit the cap less often than CoT on every benchmark. LaTER-SFT reduces the rate from 36.7% to 20.0% on AIME and from 36.9% to 15.2% on GPQA, demonstrating more frequent completion within the same budget.
LaTER-SFT's accuracy and generation advantages also persist when the cap is increased to 38,912 positions. In the three-seed Qwen3-14B evaluation in Q1, it achieves higher mean accuracy than CoT on AIME 2025 (75.5% vs. 72.2%), GPQA Diamond (63.1% vs. 62.3%), and HumanEval+ (91.5% vs. 90.9%), while reducing mean generation length by 27.0%, 40.7%, and 28.3%, respectively. The extended evaluation further shows that LaTER-SFT retains its advantages when substantially more generation capacity is available, supporting the relevance of the original evaluation setting.
For OLMo, we checked the logs and repeated the experiment. The correct average CoT length for OLMo-3-32B-Think on GPQA Diamond is 7,321. We apologize for this reporting error. We will correct the entry and all affected derived descriptions.
Q3: Threshold selection and the two exit criteria
The threshold was selected on AIME 2025 and then fixed across the other benchmarks. Specifically, we chose in the exploratory AIME switching analysis and used this setting for the remaining six benchmarks without task-specific retuning.We agree that this makes the reported AIME operating point exploratory. To examine whether switching remains effective without an entropy threshold, we evaluated a terminating-token-only variant with a generation cap of 38,912 positions, and a maximum of 64 latent steps.
Table 3. Terminating-token exits in the token-only Qwen3-14B.
| Benchmark | N | Terminating-token exits (n) (%) |
|---|---|---|
| AIME 2025 | 30 | 19 (63.3%) |
| GSM8K | 1,319 | 1,037 (78.6%) |
| MATH-500 | 500 | 376 (75.2%) |
| ARC-Challenge | 1,172 | 974 (83.1%) |
| GPQA Diamond | 198 | 175 (88.4%) |
| HumanEval+ | 164 | 153 (93.3%) |
| MBPP+ | 378 | 326 (86.2%) |
In this variant, 63.3%–93.3% of examples exit through model-native terminating-token predictions. These measurements support the usefulness of terminating-token predictions as an operational switching signal without an entropy threshold.
We further evaluated the accuracy and generation length of the token-only variant to examine the added value of the entropy threshold and its complementarity with terminating-token predictions in training-free switching. Table 4 compares these results with CoT and the full LaTER-TF policy.
Table 4. Comparison with token-only results. Each entry reports Acc. (%) / mean Tokens.
| Benchmark | CoT | LaTER-TF | Token-only |
|---|---|---|---|
| AIME 2025 | 70.0 / 15,730 | 73.3 / 10,661 | 70.0 / 12,736 |
| GSM8K | 92.7 / 879 | 93.2 / 649 | 93.1 / 746 |
| MATH-500 | 93.4 / 3,472 | 97.2 / 2,887 | 96.2 / 2,590 |
| ARC-Challenge | 96.3 / 631 | 96.5 / 533 | 96.5 / 528 |
| GPQA Diamond | 62.6 / 6,301 | 60.6 / 4,445 | 58.6 / 4,542 |
| HumanEval+ | 90.9 / 2,427 | 90.9 / 1,790 | 89.6 / 2,346 |
| MBPP+ | 82.0 / 2,144 | 79.6 / 1,760 | 80.7 / 1,918 |
Combining both exit signals yields higher accuracy and shorter generations than token-only on AIME 2025, GSM8K, GPQA Diamond, and HumanEval+. For example, full LaTER-TF achieves 73.3% accuracy with 10,661 positions on AIME 2025, compared with 70.0% and 12,736 for token-only. The entropy threshold provides a complementary exit condition that limits latent computation for samples that cross the threshold before predicting a terminating token. These results support its role in achieving a more consistent balance between accuracy and generation length.
Response to Reviewer aEL3 (Part 3/4)
Q4: Isolating the contribution of the continuous latent prefix
CoT-SFT includes the KL term to the same intuition-conditioned teacher and uses the same continuation-loss weighting as LaTER-SFT. Both training use the same shortened-derivation corpus, KL coefficient, aligned-position mask, and CoT cross-entropy weight. We will make these settings explicit in Appendix A.3.
This matched control directly addresses whether LaTER-SFT's gains arise from access to different training data or supervision. Relative to CoT-SFT, LaTER-SFT reduces generation length on every benchmark, with mean per-benchmark reductions of 19.9% on Qwen3-8B and 17.8% on Qwen3-14B. Mean accuracies are 79.4% versus 79.1% on 8B and 86.6% versus 83.7% on 14B (manuscript Table 1). These comparisons support the added benefit of the latent-prefix and transition-training components together under matched data and supervision. The no-KL variant also retains 84.9% mean accuracy and 25.5% mean generation savings on 14B (manuscript Table 5), showing that substantial savings remain without KL matching.
We agree that the proposed pause-prefix, explicit-intuition, and non-informative-prefix controls would provide valuable evidence about the contribution of the latent prefix. They would complement the matched CoT-SFT comparison by helping distinguish the effects of recurrent continuous states, additional computation positions, and an explicit textual plan. We will include these comparisons in future work to clarify the complementary roles of latent and explicit reasoning and further strengthen our conclusions.
Q5: Projector training, inference lengths, and timing
The projector is randomly initialized and optimized jointly with the backbone; it is not initialized from the analytic training-free mapping. The projector is a two-layer MLP with a Linear–GELU–Dropout–Linear structure. It uses the same AdamW learning-rate schedule as the Qwen backbone. The projector receives gradients from continuation CE and KL through the recurrent latent prefix, while the halting objective directly supervises the latent boundary. We will make the initialization, optimizer groups, and gradient paths explicit.
The maximum latent budget at inference is 64 steps for both LaTER-SFT and LaTER-TF. We count the average latent steps actually executed for benchmark in Table 5.
Table 5. Mean inference-time latent steps per problem.
| Benchmark | 8B LaTER-SFT | 8B LaTER-TF | 14B LaTER-SFT | 14B LaTER-TF |
|---|---|---|---|---|
| AIME 2025 | 44 | 61 | 49 | 56 |
| GSM8K | 42 | 63 | 43 | 54 |
| MATH-500 | 41 | 62 | 43 | 55 |
| ARC-Challenge | 46 | 63 | 45 | 52 |
| GPQA Diamond | 42 | 63 | 45 | 54 |
| HumanEval+ | 42 | 63 | 44 | 61 |
| MBPP+ | 47 | 63 | 52 | 60 |
LaTER-SFT averages 41–47 latent steps on 8B and 43–52 on 14B, compared with 3–5 steps for the no-halt variant. This supports the role of the halting objective in addressing premature inference exits. LaTER-TF's means are closer to the 64-step maximum. We will supplement these means with per-problem length distributions and exit-source counts in Q3 to quantify how often switching follows the model's stopping criteria or the maximum budget.
Regarding timing, we clarify the unit as amortized seconds per sample, a value obtained by allocating the generation time to the individual sample within the generation. CoT and CoT-SFT use Transformers' model.generate(). LaTER-SFT and LaTER-TF also build on model.generate(), with adaptations for latent reasoning. To extend the timing evaluation, we ran each method on two H100 GPUs with a per-GPU batch size of 4 and 3 seeds on Qwen3-14B.
Table 6. Qwen3-14B end-to-end inference time. Entries report mean ± sample standard deviation across three seeds.
| Benchmark | CoT | LaTER-SFT | LaTER-TF |
|---|---|---|---|
| AIME 2025 | 720.90 ± 17.72 | 540.78 ± 13.64 | 601.48 ± 87.38 |
| GPQA Diamond | 271.97 ± 48.45 | 115.96 ± 12.62 | 152.82 ± 6.29 |
| HumanEval+ | 76.51 ± 3.71 | 55.99 ± 2.90 | 64.47 ± 4.55 |
LaTER-SFT reduces mean time relative to CoT by 25.0% on AIME 2025, 57.4% on GPQA Diamond, and 26.8% on HumanEval+. The corresponding reductions for LaTER-TF are 16.6%, 43.8%, and 15.7%, respectively. Both variants therefore reduce runtime alongside generation length.
Response to Reviewer aEL3 (Part 1/4)
We thank the reviewer for the careful review and for recognizing LaTER's generation savings and its halting objective for addressing the training–inference mismatch. The concerns about statistical reliability, generation budgets, and the latent prefix help us clarify the evidence and scope of the paper. Below, we provide additional measurements, clarify the experimental controls, and outline the corresponding revisions.
Q1: Repeated-sample evaluation and the reliability of accuracy gains
We agree that the original single-seed results do not establish the statistical reliability of small accuracy differences, especially on AIME 2025. Our original generation limits follow LatentMAS [1]. To assess sampling variability and budget sensitivity, we evaluated Qwen3-14B with a 38,912-position cap and seeds 42, 43, and 44. Each problem received one sampled answer per seed, with temperature 0.6, top-p 0.95, and top-k 20.
Table 1. Repeated evaluation on Qwen3-14B. Entries report mean ± sample standard deviation across the three seeds.
LaTER-SFT has higher observed mean accuracy than CoT on all three benchmarks, with generation reductions of 27.0%, 40.7%, and 28.3%, respectively. Thus, the evidence for generation efficiency extends beyond the original single-seed results and the original benchmark-specific caps. For AIME 2025 accuracy, a two-sided paired permutation test across the matched seeds yields p=0.25.
The full evaluation across benchmarks and three seeds is estimated to require approximately 700 GPU hours. We are continuing these runs during the remaining rebuttal period and will share completed results as they become available.
[1] Zou, Jiaru, et al. "Latent Collaboration in Multi-Agent Systems." ICML, 2026.
Q2: Generation budgets, and truncation
Our inspection of the original Qwen3-8B runs confirms that truncation is material. Table 2 quantifies cap-hit rates alongside accuracy and generation length.
Table 2. Qwen3-8B cap-hit statistics. N is the number of problems, and Cap is the maximum number of generation positions. Each entry reports Acc. (%) / Tokens.