acceptodds
Under review as a conference paper at ICLR 2027

LaTER: Adaptive Latent Computation for Efficient Explicit Reasoning

Abstract

Chain-of-thought (CoT) reasoning realizes additional inference-time computation through autoregressive token generation, while continuous latent reasoning removes this dependence on discrete intermediate tokens. However, making the entire derivation latent can degrade reasoning accuracy. We propose **La**tent-**T**hen-**E**xplicit **R**easoning (**LaTER**), which uses a continuous latent prefix to prepare a shorter explicit derivation. LaTER recurrently feeds projected hidden states back as inputs, then transitions to token generation while retaining the latent key-value cache. We study both a training-free variant, which determines this transition from vocabulary-probe entropy and terminating-token predictions, and a supervised variant trained on Latent-Switch-69K, a corpus of 69,745 examples that we construct by pairing solution intuitions with shortened derivations. To learn reliable stopping behavior, we introduce a halting objective that accounts for a training-inference mismatch in which forced training rollouts can mask premature stop predictions. On Qwen3-14B, supervised LaTER improves accuracy while reducing reasoning length on all seven evaluated benchmarks relative to both standard CoT and a same-data CoT-SFT control. On AIME 2025, accuracy increases from 70.0% to 80.0%, while latent steps plus explicit tokens decrease from 15,730 to 10,575. On Qwen3-8B, wall-clock decoding time decreases by 19.1% on average across benchmarks in our implementation. These results suggest that latent and explicit reasoning are complementary, and that the transition between them is a key design choice for improving the accuracy-efficiency tradeoff. Our code, data, and model are available at https://anonymous.4open.science/r/LaTER-EF4D.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Latent Reasoning, Chain-of-Thought, Large Language Models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_aEL3OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Latent-Then-Explicit Reasoning (LaTER). A reasoning LLM first runs a short recurrent latent rollout, feeding final-layer hidden states back as input embeddings without emitting tokens. It then switches once to ordinary chain-of-thought (CoT) decoding and keeps the latent key–value cache.

The paper studies two variants. LaTER-TF is training-free: it uses the LatentMAS linear mapping and leaves the latent phase when the vocabulary-probe entropy exceeds a threshold τH\tau_H or the probe argmax is a terminating token (Eq. 4). LaTER-SFT is supervised: it trains a two-layer MLP projector together with the backbone on Latent-Switch-69K. This corpus has 69,745 records. For each one, a teacher (DeepSeek-V3.2) takes a Dolci-Think-SFT-32B trace, extracts a “solution intuition,” and writes a shortened derivation. The intuition sets the latent budget (about L/2L/2 positions, at most 128) and also conditions a frozen reference teacher used for KL matching. A halting objective (Eq. 9) targets a train–inference mismatch: training forces the full prescribed rollout, while inference stops at the first predicted </latent_think>.

Experiments cover seven math, science, and code benchmarks. Qwen3-8B/14B are run in both settings; DeepSeek-R1-Distill-Llama-8B and OLMo-3-32B-Think are run training-free only. The paper reports accuracy and generation length, where length counts latent steps plus explicit tokens (Tables 1–4). It also reports loss ablations (Table 5), a threshold sweep (Fig. 4), wall-clock decoding time (Table 6), and qualitative entropy and PCA diagnostics (App. C).

Critical strengths
  1. Consistent efficiency gains at a realistic scale, without the accuracy collapse of prior latent methods. On both Qwen3 sizes, LaTER-SFT produces shorter generations than CoT and the same-data CoT-SFT control on every one of the seven benchmarks (Table 1). Mean ΔC\Delta C relative to CoT is −23.3% (8B) and −28.9% (14B); relative to CoT-SFT the mean reduction is 19.9% and 17.8% (Sec. 4.2). The training-free variant also shortens generations on all seven benchmarks for both Qwen models (−17.8% and −23.5%; Tables 3–4). The authors measure wall-clock decoding time with latent computation included, and the token savings carry over: 19.1% faster than CoT and 14.6% faster than CoT-SFT on Qwen3-8B (Table 6; I recomputed both averages from the table). Coconut, CODI, and SIM-CoT, trained on the same corpus, trail LaTER-SFT by 16.8–23.7 points of mean accuracy on Qwen3-8B (Table 2). LaTER-SFT, in contrast, matches or exceeds CoT accuracy on every benchmark for both Qwen sizes (Table 1). These reductions are large and hold for all 14 benchmark–model pairs, so they are the most robust result in the paper. It shows that a latent prefix can be combined with modern long-CoT models without giving up explicit verification.

  2. A clearly identified and ablated train–inference halting mismatch. Training always runs the prescribed mm latent steps, but inference exits at the first predicted stop token, and masked latent-interior CE never penalizes premature stops (Sec. 3.4, App. B.5). The paper diagnoses this and shows that it matters. Without the halting objective, inference exits after 3–5 latent steps, mean savings fall from 28.9% to 4.9%, and mean accuracy falls from 86.6% to 84.1%; the full model is better on every benchmark (Table 5, Sec. 5.2). The observation is simple, and it applies to any latent-reasoning method that learns when to stop under teacher-forced rollouts.

Critical weaknesses
  1. The accuracy improvements are not statistically established, and the evaluation protocol may favor shorter outputs. The abstract (lines 24–27) and the introduction (lines 89–93) state that LaTER improves accuracy. Four issues weaken this claim.

    • One sampled run. Evaluation uses a single random seed with temperature-0.6 sampling (App. A.2, lines 678–680). Every AIME 2025 accuracy in Tables 1–5 is a multiple of 1/301/30, which is consistent with one sample per problem. The headline 14B gain, 70.0%→\to<!-- -->{=html}80.0% (Table 1), is therefore 3 of 30 problems, and the LaTER-TF gain of 70.0%→\to<!-- -->{=html}73.3% (Table 4, Fig. 1) is a single problem. For scale, a single run at p≈0.7p\approx0.7 on 30 problems has a binomial standard error of about 8.4 points. Paired comparisons are tighter, but the paper gives no per-problem results, repeated runs, variances, or confidence intervals with which to assess them. On Qwen3-8B, LaTER-SFT beats the same-data CoT-SFT control by only 0.3 points of mean accuracy, winning on four benchmarks and losing on three (Table 1; lines 300–301). Most of the 8B gain over CoT (75.9$\to$<!-- -->{=html}79.4) is already reached by CoT-SFT (79.1).

    • Threshold chosen on the test set. For Qwen3-14B on AIME 2025, the LaTER-TF result in Table 4 and Fig. 1 (73.3%, 10,661 tokens) is the best point of a τH\tau_H sweep run on the same 30 test problems (Fig. 4b; Sec. 5.1). Reading the values from Fig. 4b, τH∈{3,4,5,6,7,8}\tau_H\in\lbrace 3,4,5,6,7,8\rbrace gives about 66.7, 63.3, 70.0, 70.0, 73.3, and 66.7%, so only the selected value beats CoT. I could not find the τH\tau_H used for the other benchmarks and backbones in Tables 3–4.

    • Generation caps that may bind. Caps are 20,000 positions for AIME and 8,192 for GPQA (App. A.2, lines 682–685), and mean CoT lengths reach 73–79% of them: 14,672 (8B) and 15,730 (14B) on AIME, and 6,039 and 6,301 on GPQA (Table 1). Truncation rates are not reported. Under such caps, a method that shortens outputs can gain accuracy simply because more answers finish within the cap. The Qwen3-8B CoT baseline scores 53.3% on AIME 2025 and 47.0% on GPQA Diamond. These are well below the thinking-mode numbers in the Qwen3 technical report (67.3% and 62.0%). That report used caps of 38,912 and 32,768 tokens and averaged 64 and 10 samples, so the gap is not an error in itself. It is, however, consistent with a baseline running in a cap-constrained regime, though it does not prove this. The 14B baseline, by contrast, is close to the report (70.0 vs. 70.4 and 62.6 vs. 64.0). In Table 4, OLMo-3-32B-Think’s mean CoT length on GPQA Diamond is 10,460 tokens. That exceeds the stated 8,192 cap, so the protocol for that row is unclear.

    • Generality across backbones is overstated. Sec. 4.3 (lines 323–367) says that “benefits also extend beyond Qwen” and cites only GSM8K. Across all seven benchmarks in Table 4, LaTER-TF lowers mean accuracy on DeepSeek-R1-Distill-Llama-8B (66.2$\to$<!-- -->{=html}65.5; 3 wins, 4 losses) and on OLMo-3-32B-Think (82.9$\to$<!-- -->{=html}82.2; 2 wins, 2 ties, 3 losses). It also lengthens ARC-Challenge generations on both models (by 16.5% and 11.5%).

    These issues do not affect the token reductions, but as reported the accuracy gains cannot be told apart from sampling noise, test-set threshold selection, or truncation effects. The fix is straightforward but requires new runs: multi-sample evaluation (e.g., avg@16 on AIME and several seeds elsewhere) with paired confidence intervals, truncation rates per method (or results at the models’ recommended budgets), and τH\tau_H chosen on held-out data.

  2. No experiment isolates what the continuous latent prefix contributes, and the closest prior work is under-positioned. The paper’s central claim is that the latent prefix “prepares” a shorter explicit derivation and that latent and explicit reasoning are complementary (abstract; lines 96–102). Four gaps remain.

    • The prefix is short, and its inference length is not reported. Training budgets have a mean of 41.49 positions and a median of 40 (Table 7, App. B.3), while explicit continuations run to thousands of tokens. The claim therefore rests on the prefix changing how the explicit part is written, not on the prefix replacing text. The paper does not report the number of latent steps SS that LaTER-SFT or LaTER-TF actually take at inference. The only reported figure is 3–5 steps for the no-halt variant (line 424).

    • The explicit control differs from LaTER-SFT in more than one way. CoT-SFT shortens outputs much less than LaTER-SFT (−4.0% and −13.3%; Table 1), which is suggestive. But App. A.3 (lines 697–698) says only that CoT-SFT “removes the latent segment, projector, and halting loss.” It does not say whether CoT-SFT keeps the KL term to the intuition-conditioned teacher or the λCoT=0.5\lambda_{\mathrm{CoT}}=0.5 weighting. If CoT-SFT has no KL term, the like-for-like comparison is the no-KL variant. On Qwen3-14B that variant beats CoT-SFT by 1.1 points of mean accuracy, and it loses 3.5 points on GPQA (Table 5 vs. Table 1).

    • Key controls are missing. I would expect four: (i) a prefix of placeholder or pause tokens with the same budget and losses, analogous to Coconut’s “pause as thought” ablation (Hao et al., 2025); (ii) a variant that writes the intuition (about 80 tokens) explicitly before the shortened derivation; (iii) in the training-free setting, a prefix of the same length made of non-informative embeddings; and (iv) explicit length-reduction baselines run at LaTER’s token budget. Control (iii) matters because the paper itself observes that probe argmaxes drift toward terminating tokens such as </think> (App. C.1). A generic “already-thought” perturbation could therefore shorten the continuation. The latent baselines in Table 2 run at a mean ΔC\Delta C of −58.6% to −94.0%. They show that keeping explicit CoT preserves accuracy, but they do not show that LaTER improves the tradeoff at its own operating point (about −23%).

    • Positioning against prior work. Section 6 (lines 465–466) says that training-based latent methods “focus primarily on compressing explicit reasoning into continuous representations.” Three prior works are closer to LaTER than this suggests. In Coconut’s curriculum, at stage kk the first kk reasoning steps are continuous thoughts and the rest are in language. Coconut’s analysis also runs kk latent thoughts and then completes the solution in language. Token Assorted (Su et al., 2025; arXiv:2502.03275) replaces the initial reasoning steps with discrete (VQ-VAE) latent tokens followed by text; the paper does not cite it. SwiReasoning (Shi et al., 2026) is training-free and switches between latent and explicit reasoning based on entropy trends, which makes it the closest training-free method. The paper discusses it only in Related Work and leaves it out of the training-free experiments. Table 3 compares against the test-time optimization methods LatentSeek and LTPO, and Table 4 compares only with CoT.

    Without these controls, the gains could equally come from the shortened-derivation data, the intuition-conditioned KL, or simply the extra positions, rather than from latent reasoning. This is fixable with the controls above and a sharper novelty statement. In my view the real novelty lies in adaptive single-switch timing at scale and in the halting objective.

Minor points. The Conclusion (line 480) says LaTER uses “one third fewer tokens than standard CoT,” but the paper’s own mean ΔC\Delta C is −23.3% (8B) and −28.9% (14B) (Table 1). Figure 1 computes its percentages from rounded values: for MATH-500 it shows −18%, whereas Table 4 gives 3,472$\to$<!-- -->{=html}2,887, i.e., −16.8%. The HumanEval+ entry of 92.2% (Table 1, 14B LaTER-SFT) does not match any integer count out of 164 problems. The PCA diagnostics (App. C.3) are qualitative and cover a few samples, with no stated selection criterion. If PC1 is computed jointly over latent and explicit states, as the shared axes suggest, a separated latent cluster alone can raise the PC1 fraction. The claim that LaTER “constrains the subsequent explicit reasoning trajectory to a lower-dimensional region” (lines 1007–1009) therefore needs a PCA over explicit states only. Typos: “doceded token” (Fig. 1), “An intuition of LL tokens assign” (line 195), and “Follow Equation 3” (line 855).

Questions for authors
  1. [Clarification] Repeated-sample statistics. How many samples per problem were drawn, and which methods were decoded greedily? App. A.2 leaves the latter open. Please report mean ±\pm s.d. over several seeds, or avg@kk (e.g., k=16k=16 on AIME 2025), with paired confidence intervals for LaTER-SFT vs. CoT-SFT and for LaTER-TF vs. CoT. Please also give the number of evaluated problems per benchmark (see the HumanEval+ point above). If the 14B gains over CoT-SFT (+2.8 mean, +6.7 on AIME) survive this, my assessment of the accuracy claims and my rating would go up. If they fall within noise, I would read the paper as an efficiency result only.

  2. [Clarification] Truncation. For each method and benchmark, especially AIME and GPQA, what fraction of responses hit the generation cap? How do the results change at the models’ recommended budgets (e.g., 32k/38k tokens)? Why does the OLMo GPQA CoT mean (10,460) exceed the 8,192 cap? What explains the gap between the Qwen3-8B CoT baseline and published thinking-mode results? If the accuracy gains hold with caps that do not bind, the claim that LaTER improves reasoning, rather than just keeping answers within budget, would be much stronger.

  3. [Clarification] Threshold selection. Which τH\tau_H was used for each benchmark and backbone in Tables 3–4, and was it chosen on test data? What accuracy do two alternatives reach: the terminating-token-only rule (τH=∞\tau_H=\infty), and a single τH\tau_H fixed in advance or tuned on a held-out split? What fraction of exits does each criterion trigger? Most latent entropies in Fig. 7 lie well below the swept range of 3–8, so I suspect the terminating-token criterion triggers most exits. If a fixed or held-out τH\tau_H gives similar results, my concern about LaTER-TF would largely go away.

  4. [Clarification] Isolating the latent prefix. Does CoT-SFT include the KL term to the intuition-conditioned teacher, and the λCoT\lambda_{\mathrm{CoT}} weighting? Could the authors add three controls: (i) a pause or placeholder prefix with the same budget and losses, (ii) an SFT variant that writes the intuition explicitly, and (iii) a training-free prefix of non-informative embeddings with the same length? If LaTER clearly beats these at similar total cost, the claim that latent and explicit reasoning are complementary would be supported. If it does not, the gains are better attributed to the data and objectives.

  5. [Clarification] Training and inference details. App. B.6 specifies a learning rate of 10−710^{-7} decaying to 10−810^{-8} over one epoch. Does the newly introduced projector gϕg_\phi use the same rate, and how is it initialized (randomly, or from WaW_a)? What is the inference-time distribution of the number of latent steps SS per benchmark for LaTER-SFT and LaTER-TF, and is there a maximum latent budget at inference? What unit does Table 6 report (per problem, per batch, or per benchmark), and does LaTER use the same model.generate() path as the baselines? If the projector barely moves from its initialization at this learning rate, the description of a “learned projector” would need revisiting.

  6. [Suggestion / future work] Please compare against the closest training-free methods, SwiReasoning and Soft Thinking, on the same backbones and caps. Please also compare against length-matched explicit baselines, such as budget-forced or early-exit CoT at LaTER’s token count. When stating the novelty, please discuss Coconut’s partially latent curriculum stages and Token Assorted.

7 comments
AuthorsOpenReview ↗

Response to Reviewer aEL3 (Part 1/4)

We thank the reviewer for the careful review and for recognizing LaTER's generation savings and its halting objective for addressing the training–inference mismatch. The concerns about statistical reliability, generation budgets, and the latent prefix help us clarify the evidence and scope of the paper. Below, we provide additional measurements, clarify the experimental controls, and outline the corresponding revisions.

Q1: Repeated-sample evaluation and the reliability of accuracy gains

We agree that the original single-seed results do not establish the statistical reliability of small accuracy differences, especially on AIME 2025. Our original generation limits follow LatentMAS [1]. To assess sampling variability and budget sensitivity, we evaluated Qwen3-14B with a 38,912-position cap and seeds 42, 43, and 44. Each problem received one sampled answer per seed, with temperature 0.6, top-p 0.95, and top-k 20.

Table 1. Repeated evaluation on Qwen3-14B. Entries report mean ± sample standard deviation across the three seeds.

BenchmarkMethodAcc. (%)Tokens
AIME 2025CoT72.2 ± 3.915,391 ± 338
LaTER-SFT75.5 ± 3.911,231 ± 451
LaTER-TF71.1 ± 1.912,964 ± 775
GPQA DiamondCoT62.3 ± 1.06,341 ± 40
LaTER-SFT63.1 ± 2.33,758 ± 45
LaTER-TF59.9 ± 0.64,458 ± 46
HumanEval+CoT90.9 ± 1.62,589 ± 43
LaTER-SFT91.5 ± 0.61,856 ± 128
LaTER-TF91.3 ± 0.31,932 ± 237

LaTER-SFT has higher observed mean accuracy than CoT on all three benchmarks, with generation reductions of 27.0%, 40.7%, and 28.3%, respectively. Thus, the evidence for generation efficiency extends beyond the original single-seed results and the original benchmark-specific caps. For AIME 2025 accuracy, a two-sided paired permutation test across the matched seeds yields p=0.25.

The full evaluation across benchmarks and three seeds is estimated to require approximately 700 GPU hours. We are continuing these runs during the remaining rebuttal period and will share completed results as they become available.

[1] Zou, Jiaru, et al. "Latent Collaboration in Multi-Agent Systems." ICML, 2026.

Q2: Generation budgets, and truncation

Our inspection of the original Qwen3-8B runs confirms that truncation is material. Table 2 quantifies cap-hit rates alongside accuracy and generation length.

Table 2. Qwen3-8B cap-hit statistics. N is the number of problems, and Cap is the maximum number of generation positions. Each entry reports Acc. (%) / Tokens.

Benchmark (N)CapMethodCap-hit (n) (%)All resultsBelow cap
AIME 2025 (30)20,000CoT11 (36.7%)53.3 / 14,67284.2 / 11,587
LaTER-SFT6 (20.0%)63.3 / 11,37879.2 / 9,223
LaTER-TF6 (20.0%)56.7 / 12,77770.8 / 10,971
GSM8K (1,319)2,048CoT126 (9.6%)88.8 / 1,15398.2 / 1,058
LaTER-SFT63 (4.8%)93.6 / 87298.2 / 813
LaTER-TF56 (4.2%)91.1 / 94595.1 / 896
MATH-500 (500)8,192CoT43 (8.6%)89.2 / 4,14497.6 / 3,763
LaTER-SFT32 (6.4%)89.4 / 2,90995.5 / 2,548
LaTER-TF30 (6.0%)92.0 / 3,34297.9 / 3,032
ARC-Challenge (1,172)2,048CoT74 (6.3%)91.0 / 89297.1 / 814
LaTER-SFT23 (2.0%)91.4 / 65393.2 / 625
LaTER-TF25 (2.1%)92.2 / 69594.2 / 666
GPQA Diamond (198)8,192CoT73 (36.9%)47.0 / 6,03974.4 / 4,782
LaTER-SFT30 (15.2%)52.0 / 4,28461.3 / 3,586
LaTER-TF32 (16.2%)49.0 / 4,91258.4 / 4,280
HumanEval+ (164)4,096CoT19 (11.6%)86.6 / 2,80997.9 / 2,640
LaTER-SFT14 (8.5%)87.8 / 2,39196.0 / 2,232
LaTER-TF17 (10.4%)85.4 / 2,41695.2 / 2,222
MBPP+ (378)4,096CoT58 (15.3%)75.7 / 2,63589.4 / 2,370
LaTER-SFT30 (7.9%)78.3 / 2,22885.1 / 2,067
LaTER-TF43 (11.4%)79.6 / 2,12689.9 / 1,873
AuthorsOpenReview ↗

Response to Reviewer aEL3 (Part 2/4)

Both LaTER variants hit the cap less often than CoT on every benchmark. LaTER-SFT reduces the rate from 36.7% to 20.0% on AIME and from 36.9% to 15.2% on GPQA, demonstrating more frequent completion within the same budget.

LaTER-SFT's accuracy and generation advantages also persist when the cap is increased to 38,912 positions. In the three-seed Qwen3-14B evaluation in Q1, it achieves higher mean accuracy than CoT on AIME 2025 (75.5% vs. 72.2%), GPQA Diamond (63.1% vs. 62.3%), and HumanEval+ (91.5% vs. 90.9%), while reducing mean generation length by 27.0%, 40.7%, and 28.3%, respectively. The extended evaluation further shows that LaTER-SFT retains its advantages when substantially more generation capacity is available, supporting the relevance of the original evaluation setting.

For OLMo, we checked the logs and repeated the experiment. The correct average CoT length for OLMo-3-32B-Think on GPQA Diamond is 7,321. We apologize for this reporting error. We will correct the entry and all affected derived descriptions.

Q3: Threshold selection and the two exit criteria

The threshold was selected on AIME 2025 and then fixed across the other benchmarks. Specifically, we chose τH=7\tau_H=7 in the exploratory AIME switching analysis and used this setting for the remaining six benchmarks without task-specific retuning.We agree that this makes the reported AIME operating point exploratory. To examine whether switching remains effective without an entropy threshold, we evaluated a terminating-token-only variant with a generation cap of 38,912 positions, and a maximum of 64 latent steps.

Table 3. Terminating-token exits in the token-only Qwen3-14B.

BenchmarkNTerminating-token exits (n) (%)
AIME 20253019 (63.3%)
GSM8K1,3191,037 (78.6%)
MATH-500500376 (75.2%)
ARC-Challenge1,172974 (83.1%)
GPQA Diamond198175 (88.4%)
HumanEval+164153 (93.3%)
MBPP+378326 (86.2%)

In this variant, 63.3%–93.3% of examples exit through model-native terminating-token predictions. These measurements support the usefulness of terminating-token predictions as an operational switching signal without an entropy threshold.

We further evaluated the accuracy and generation length of the token-only variant to examine the added value of the entropy threshold and its complementarity with terminating-token predictions in training-free switching. Table 4 compares these results with CoT and the full LaTER-TF policy.

Table 4. Comparison with token-only results. Each entry reports Acc. (%) / mean Tokens.

BenchmarkCoTLaTER-TFToken-only
AIME 202570.0 / 15,73073.3 / 10,66170.0 / 12,736
GSM8K92.7 / 87993.2 / 64993.1 / 746
MATH-50093.4 / 3,47297.2 / 2,88796.2 / 2,590
ARC-Challenge96.3 / 63196.5 / 53396.5 / 528
GPQA Diamond62.6 / 6,30160.6 / 4,44558.6 / 4,542
HumanEval+90.9 / 2,42790.9 / 1,79089.6 / 2,346
MBPP+82.0 / 2,14479.6 / 1,76080.7 / 1,918

Combining both exit signals yields higher accuracy and shorter generations than token-only on AIME 2025, GSM8K, GPQA Diamond, and HumanEval+. For example, full LaTER-TF achieves 73.3% accuracy with 10,661 positions on AIME 2025, compared with 70.0% and 12,736 for token-only. The entropy threshold provides a complementary exit condition that limits latent computation for samples that cross the threshold before predicting a terminating token. These results support its role in achieving a more consistent balance between accuracy and generation length.

AuthorsOpenReview ↗

Response to Reviewer aEL3 (Part 3/4)

Q4: Isolating the contribution of the continuous latent prefix

CoT-SFT includes the KL term to the same intuition-conditioned teacher and uses the same continuation-loss weighting as LaTER-SFT. Both training use the same shortened-derivation corpus, KL coefficient, aligned-position mask, and CoT cross-entropy weight. We will make these settings explicit in Appendix A.3.

This matched control directly addresses whether LaTER-SFT's gains arise from access to different training data or supervision. Relative to CoT-SFT, LaTER-SFT reduces generation length on every benchmark, with mean per-benchmark reductions of 19.9% on Qwen3-8B and 17.8% on Qwen3-14B. Mean accuracies are 79.4% versus 79.1% on 8B and 86.6% versus 83.7% on 14B (manuscript Table 1). These comparisons support the added benefit of the latent-prefix and transition-training components together under matched data and supervision. The no-KL variant also retains 84.9% mean accuracy and 25.5% mean generation savings on 14B (manuscript Table 5), showing that substantial savings remain without KL matching.

We agree that the proposed pause-prefix, explicit-intuition, and non-informative-prefix controls would provide valuable evidence about the contribution of the latent prefix. They would complement the matched CoT-SFT comparison by helping distinguish the effects of recurrent continuous states, additional computation positions, and an explicit textual plan. We will include these comparisons in future work to clarify the complementary roles of latent and explicit reasoning and further strengthen our conclusions.

Q5: Projector training, inference lengths, and timing

The projector is randomly initialized and optimized jointly with the backbone; it is not initialized from the analytic training-free mapping. The projector is a two-layer MLP with a Linear–GELU–Dropout–Linear structure. It uses the same AdamW learning-rate schedule as the Qwen backbone. The projector receives gradients from continuation CE and KL through the recurrent latent prefix, while the halting objective directly supervises the latent boundary. We will make the initialization, optimizer groups, and gradient paths explicit.

The maximum latent budget at inference is 64 steps for both LaTER-SFT and LaTER-TF. We count the average latent steps actually executed for benchmark in Table 5.

Table 5. Mean inference-time latent steps per problem.

Benchmark8B LaTER-SFT8B LaTER-TF14B LaTER-SFT14B LaTER-TF
AIME 202544614956
GSM8K42634354
MATH-50041624355
ARC-Challenge46634552
GPQA Diamond42634554
HumanEval+42634461
MBPP+47635260

LaTER-SFT averages 41–47 latent steps on 8B and 43–52 on 14B, compared with 3–5 steps for the no-halt variant. This supports the role of the halting objective in addressing premature inference exits. LaTER-TF's means are closer to the 64-step maximum. We will supplement these means with per-problem length distributions and exit-source counts in Q3 to quantify how often switching follows the model's stopping criteria or the maximum budget.

Regarding timing, we clarify the unit as amortized seconds per sample, a value obtained by allocating the generation time to the individual sample within the generation. CoT and CoT-SFT use Transformers' model.generate(). LaTER-SFT and LaTER-TF also build on model.generate(), with adaptations for latent reasoning. To extend the timing evaluation, we ran each method on two H100 GPUs with a per-GPU batch size of 4 and 3 seeds on Qwen3-14B.

Table 6. Qwen3-14B end-to-end inference time. Entries report mean ± sample standard deviation across three seeds.

BenchmarkCoTLaTER-SFTLaTER-TF
AIME 2025720.90 ± 17.72540.78 ± 13.64601.48 ± 87.38
GPQA Diamond271.97 ± 48.45115.96 ± 12.62152.82 ± 6.29
HumanEval+76.51 ± 3.7155.99 ± 2.9064.47 ± 4.55

LaTER-SFT reduces mean time relative to CoT by 25.0% on AIME 2025, 57.4% on GPQA Diamond, and 26.8% on HumanEval+. The corresponding reductions for LaTER-TF are 16.6%, 43.8%, and 15.7%, respectively. Both variants therefore reduce runtime alongside generation length.

Reviewer_S43tOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces LaTER, a method that combines latent reasoning with explicit CoT reasoning. The model first performs several reasoning steps in a continuous latent space, then switches to generating explicit reasoning tokens. The authors show that this approach can improve accuracy while reducing the number of reasoning tokens and inference time.

Critical strengths

(1) I think that combining latent and explicit reasoning is interesting. Rather than replacing explicit reasoning entirely, the method uses latent computation to shorten it while aiming to preserve accuracy.

(2) The experiments are fairly thorough, with several benchmarks, multiple baselines, and useful ablations.

Critical weaknesses

(1) In my opinion, the ablations could better isolate the effects of the different components of this method, to really demonstrate the claims the authors are making. In particular, there does not seem to be a controlled comparison of learned adaptive switching against a fixed latent budget using the same supervised training setup.

(2) I also think that the inference efficiency evaluation is somewhat limited. Wall clock latency is only reported for Qwen3-8B, while the strongest accuracy improvements are reported for Qwen3-14B. It would be useful to see whether the same latency improvements hold for larger models as well as the smaller model.

Questions for authors

Would it be possible for the authors to compare supervised LaTER against an otherwise identical model that uses a fixed number of latent steps? This would help isolate the benefit of adaptive switching.

3 comments
AuthorsOpenReview ↗

Response to Reviewer S43t

We thank the reviewer for the constructive review and recognizing the value of combining latent computation with explicit reasoning. We address the two concerns with a matched supervised comparison and new Qwen3-14B wall-clock measurements.

Q1: Comparison with fixed latent budget using the same supervised training setup

We agree that comparing adaptive switching with fixed latent budgets under the same supervised training setup is important for assessing its contribution. Following your suggestion, we train two supervised fixed-budget controls (Fixed-40 and Fixed-50) on Qwen3-8B with 8 * H100 GPUs.

Fixed-40 and Fixed-50 use 40 and 50 latent steps, respectively, during both training and inference. Each model is trained separately with its prescribed budget. The fixed-budget models use the same training settings as Adaptive LaTER-SFT and also retain the halting objective, with the stopping target assigned to the end of the fixed budget. This keeps the supervised training framework matched while comparing fixed boundaries with adaptive boundaries.

Table 1. Supervised fixed-budget and adaptive controls on Qwen3-8B. Entries report accuracy (%) / generation positions, including latent steps.

BenchmarkCoT-SFTFixed-40Fixed-50Adaptive
AIME 202560.0 / 12,79056.7 / 12,69660.0 / 10,87763.3 / 11,378
GSM8K93.2 / 1,12490.2 / 79991.1 / 79693.6 / 872
MATH-50090.2 / 3,96691.8 / 2,76091.4 / 2,69989.4 / 2,909
ARC-Challenge94.9 / 87593.2 / 61992.9 / 62291.4 / 653
GPQA Diamond49.5 / 5,95448.5 / 4,61644.9 / 4,83352.0 / 4,284
HumanEval+89.0 / 2,71081.1 / 1,75782.3 / 1,90787.8 / 2,391
MBPP+77.2 / 2,59773.8 / 1,45176.2 / 1,66178.3 / 2,228
Mean acc. (%)79.176.577.079.4

These controls support learned switching as a useful design choice for retaining mean accuracy while shortening generation. Fixed budgets favor stronger relative compression and achieve higher accuracy on MATH-500 and ARC-Challenge. Adaptive provides the higher mean-accuracy across the benchmark suite. We will incorporate this comparison into Section 5, alongside the existing switching and loss ablations.

Q2: Wall-clock efficiency on Qwen3-14B

We agree that the wall-clock evaluation should include Qwen3-14B. We extended the timing evaluation to Qwen3-14B on AIME 2025, GPQA Diamond, and HumanEval+, covering mathematics, scientific reasoning, and code generation. Each method was evaluated with 3 seeds 42, 43, and 44 on two H100 GPUs, using a per-GPU batch size of 4 and a common generation cap of 38,912 positions. We report amortized end-to-end inference time per problem, including prompt prefill, latent recurrence, and explicit generation.

Table 2. Qwen3-14B accuracy, inference time, and generation count. Values report mean ± sample standard deviation across three seeds.

BenchmarkMetricCoTLaTER-SFTLaTER-TF
AIME 2025Acc (%)72.2 ± 3.975.5 ± 3.971.1 ± 1.9
Time (s)720.90 ± 17.72540.78 ± 13.64601.48 ± 87.38
Tokens15,391 ± 33811,231 ± 45112,964 ± 775
GPQA DiamondAcc (%)62.3 ± 1.063.1 ± 2.359.9 ± 0.6
Time (s)271.97 ± 48.45115.96 ± 12.62152.82 ± 6.29
Tokens6,341 ± 403,758 ± 454,458 ± 46
HumanEval+Acc (%)90.9 ± 1.691.5 ± 0.691.3 ± 0.3
Time (s)76.51 ± 3.7155.99 ± 2.9064.47 ± 4.55
Tokens2,589 ± 431,856 ± 1281,932 ± 237

As shown in Table 2, LaTER-SFT reduces mean inference time by 25.0% on AIME 2025, 57.4% on GPQA Diamond, and 26.8% on HumanEval+, with higher mean accuracy on all three tasks. LaTER-TF reduces mean inference time by 16.6%, 43.8%, and 15.7%, respectively. Both variants also use fewer generation positions than CoT, demonstrating that the measured wall-clock savings extend to the larger backbone. We will extend the Qwen3-14B timing evaluation to the remaining benchmarks and incorporate the results into the manuscript as they become available.

AuthorsOpenReview ↗

Follow-up response to Reviewer S43t

Following your suggestion to extend the larger-model efficiency evaluation, we have now completed 3-seed Qwen3-14B experiments on all seven benchmarks.

Table 3. Full inference efficiency evaluation on Qwen3-14B.

BenchmarkMetricCoTLaTER-SFTLaTER-TF
AIME 2025Acc (%)72.2 ± 3.975.5 ± 3.971.1 ± 1.9
Time (s)720.90 ± 17.72540.78 ± 13.64601.48 ± 87.38
Tokens15,391 ± 33811,231 ± 45112,964 ± 775
GSM8KAcc (%)93.4 ± 0.394.5 ± 0.493.2 ± 0.3
Time (s)24.12 ± 0.5616.20 ± 0.4217.97 ± 1.59
Tokens935 ± 6646 ± 5658 ± 18
MATH-500Acc (%)94.4 ± 0.596.1 ± 0.596.5 ± 0.3
Time (s)164.01 ± 6.3793.61 ± 4.76110.79 ± 3.39
Tokens3,716 ± 912,357 ± 492,616 ± 35
ARC-ChallengeAcc (%)96.4 ± 0.295.8 ± 0.796.7 ± 0.2
Time (s)17.20 ± 0.5912.45 ± 0.1214.48 ± 0.24
Tokens673 ± 3437 ± 4492 ± 2
GPQA DiamondAcc (%)62.3 ± 1.063.1 ± 2.359.9 ± 0.6
Time (s)271.97 ± 48.45115.96 ± 12.62152.82 ± 6.29
Tokens6,341 ± 403,758 ± 454,458 ± 46
HumanEval+Acc (%)90.9 ± 1.691.5 ± 0.691.3 ± 0.3
Time (s)76.51 ± 3.7155.99 ± 2.9064.47 ± 4.55
Tokens2,589 ± 431,856 ± 1281,932 ± 237
MBPP+Acc (%)81.3 ± 0.382.6 ± 0.680.4 ± 1.6
Time (s)72.42 ± 4.9852.59 ± 3.4057.74 ± 1.45
Tokens2,050 ± 401,655 ± 481,771 ± 30

Across all seven benchmarks, LaTER-SFT achieves 85.6% mean accuracy versus 84.4% for CoT, with mean per-benchmark reductions of 31.1% in generation length and 34.3% in inference time. LaTER-TF reduces generation length and inference time by 24.4% and 24.3%, respectively. Both efficiency gains hold on every task.

As the discussion period is approaching its end, we would be grateful to know whether our response has addressed your concerns. If any questions or concerns remain, we would be very happy to provide further clarification. If they address your concerns, we would greatly appreciate your reconsideration of the paper. Thank you again for participating in the review of our work. We truly appreciate your consideration and wish you all the best.

Dear Reviewer S43t,

Thank you again for engaging with our submission. With the discussion window closing soon, we would be grateful for any thoughts you may wish to share.

Following your suggestions, we compared adaptive switching with fixed budgets under the same supervised training setup on Qwen3-8B. We found that adaptive switching averaged 79.4% accuracy, compared with 76.5% for 40 latent steps and 77.0% for 50. We also measured wall-clock inference time on Qwen3-14B, where LaTER-SFT saved time on all benchmarks, averaging a 34.3% reduction relative to CoT.

We hope these comparisons are helpful for your final assessment. Should any concern remain unresolved, or any part of our response require further explanation, please feel free to let us know. We will respond as promptly and concretely as possible, and sincerely hope there may still be an opportunity for a brief exchange before the discussion closes.

With thanks and best wishes,

The Authors

Reviewer_xrHDOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LaTER, which performs continuous latent computation before switching to explicit chain-of-thought reasoning. It introduces training-free and supervised switching mechanisms, together with a dataset pairing high-level solution intuitions with shortened derivations. The goal is to reduce inference cost while preserving reasoning accuracy.

Critical strengths
  1. The paper presents a promising approach to improving reasoning efficiency. Comparisons with a same-data CoT-SFT baseline help distinguish the method’s benefits from those of supervised adaptation alone. The larger model achieves consistent accuracy improvements with shorter generations, while the smaller model mainly benefits from reduced generation length with similar mean accuracy.
  2. The efficiency evaluation accounts for latent computation and includes actual decoding-time measurements. This strengthens the evidence that the method offers practical inference savings, rather than merely reducing the amount of visible reasoning text.
Critical weaknesses

The method requires additional strong model processing for data construction and longer training than CoT-SFT. Its suitability for substantially larger training datasets and workloads remains insufficiently explored.

Questions for authors

Could the authors discuss the scalability of the data-construction and training pipeline, including potential bottlenecks when extending to substantially larger training datasets?

2 comments
AuthorsOpenReview ↗

Response to Reviewer xrHD

We thank the reviewer for the positive assessment and for raising the important concern about scalability. We agree that extending LaTER to substantially larger datasets requires reducing both the cost of constructing specialized supervision and the sequential computation required during training.

Bottlenecks at larger data scales.

Scaling the current formulation involves two sources of overhead: data construction and sequential latent computation. First, existing reasoning traces do not directly provide suitable targets for LaTER's latent phase. Our current approach obtains supervision through stronger-model generation of intuition–continuation pairs. The DeepSeek-V3.2 generation cost for LATENT-SWITCH-69K is approximately USD 440, indicating a modest per-example cost. However, each additional example still requires teacher generation and validation. Second, latent inputs are generated recurrently from the student's preceding outputs and therefore cannot be supplied in advance for parallel teacher forcing. As discussed in Coconut [1], KV caching reduces repeated computation but does not remove this sequential dependency, which limits training throughput. These considerations motivate reducing reliance on per-example teacher generation and improving training parallelism.

The central challenge of reusable latent representations and appropriate segmentation

The central research questions are how to represent multiple reasoning tokens in one latent token, which tokens should be grouped together, and where compressed reasoning should transition back to text. Addressing these questions could connect the two scalability problems: suitable latent targets could be constructed directly from existing traces, reducing specialized data generation, and then supplied as known training inputs, enabling parallel computation. Such targets must preserve the information needed for subsequent reasoning while remaining predictable from the question and preceding latent states.

Existing studies provide useful starting points. CoLaR [2] and C-MTP [3] construct teacher-forced latent inputs through scaled embedding aggregation or averaging. These operations are inexpensive, but pooling raw token embeddings discards within-group order and may obscure relationships needed for reasoning, especially in fields such as mathematical problems. Token Assorted [4] and Latent-SFT [5] instead learn discrete codes or vocabulary-space targets, offering more expressive constructions at the cost of additional representation learning. Their reuse across models also requires investigation, as continuous representations tied to one backbone's embedding geometry may need realignment for another.

Toward lower-cost data construction and faster training

We plan to investigate a reusable encoding scheme that compresses a prefix of an existing reasoning trace into latent targets. This would allow us to reuse existing reasoning corpora without separately generating an intuition and rewriting the continuation for every example. We would compare fixed-length grouping, reasoning-step boundaries, and learned segmentation, using reconstruction and continuation-prediction objectives to assess what information each latent must retain.

Once the target representation is fixed, corpus conversion could use batched encoding, and the resulting targets could be cached and supplied as shifted inputs for parallel teacher-forced training. This would replace per-example teacher rewriting with reusable encoding and remove the need for sequential student latent generation during that training stage.

[1] Hao et al. "Training Large Language Models to Reason in a Continuous Latent Space." COLM, 2025.

[2] Tan et al. "Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chains." NeurIPS, 2025.

[3] Yerram et al. "Training Continuous Chain of Thought Models: A Tale of Two Regimes." ICML Workshop on Resource-Adaptive Foundation Model Inference (AdaptFM), 2026.

[4] Su et al. "Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning." ICML, 2025.

[5] Deng et al. "Latent-SFT: LLM Latent Reasoning as Chain of Superposition." EMNLP, 2026.

Dear Reviewer xrHD,

Thank you for taking the time to review our work. We appreciate your encouraging assessment and thoughtful comments.

For LATENT-SWITCH-69K, we estimate that teacher generation cost about USD 440. With similar trace lengths and prices, we could use the same process for larger datasets with a roughly proportional increase in the generation budget. As a next step, we plan to explore deriving latent targets directly from existing reasoning traces, which could reduce the need for teacher generation and support parallel training.

We hope this clarifies how we plan to extend the approach. We would be grateful for any further thoughts you may wish to share before the discussion ends. Thank you again for your time and feedback.

With thanks and best wishes,

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary for the Area Chair

Dear Area Chair,

Thank you for your time and effort in overseeing the review of our submission. We would like to briefly summarize the experiments and analyses during the discussion to address the reviewers’ main concerns.

  1. Data-construction cost and scalability analysis (Reviewer xrHD). We report an estimated teacher-generation cost of approximately USD 440 for LATENT-SWITCH-69K and analyzed the teacher-generation and sequential latent-rollout bottlenecks when scaling the pipeline.

  2. Supervised fixed-budget controls (Reviewer S43t). We train Fixed-40 and Fixed-50 on Qwen3-8B under matched supervised training settings, retaining the halting objective with fixed stopping targets. Adaptive LaTER-SFT achieves 79.4% mean accuracy versus 76.5% and 77.0%, respectively, and outperforms both controls on five of seven benchmarks.

  3. Repeated evaluation and larger-model wall-clock measurements (Reviewers aEL3 and S43t). We report results from three-seed Qwen3-14B evaluations across all benchmarks with an expanded generation cap of 38,912 positions, including means and standard deviations for accuracy, generation length, and inference time. LaTER-SFT achieves 85.6% mean accuracy versus 84.4% for CoT, with mean per-benchmark generation-length and runtime reductions of 31.1% and 34.3%. The corresponding reductions for LaTER-TF are 24.4% and 24.3%, and both variants reduce generation length and runtime on every benchmark.

  4. Comparisons with SwiReasoning and Soft Thinking (Reviewer aEL3). We evaluate both requested baselines on Qwen3-8B. LaTER-TF achieves 78.0% mean accuracy versus 73.2% and 73.4%, respectively.

  5. Truncation analysis under the original generation budgets (Reviewer aEL3). We report cap-hit rates across Qwen3-8B benchmarks. LaTER-SFT reduces cap-hit rates from 36.7% to 20.0% on AIME and from 36.9% to 15.2% on GPQA.

  6. Terminating-token-only switching control (Reviewer aEL3). We evaluate a Qwen3-14B variant that removes the entropy criterion and uses terminating-token predictions with a maximum of 64 latent steps. We report accuracy, generation length, and exit counts. 63.3%–93.3% of examples exit through terminating-token predictions. This directly tests switching without an entropy threshold.

  7. Inference-time latent-length measurements (Reviewer aEL3). We measure the actual mean latent lengths for both LaTER variants on Qwen3-8B and Qwen3-14B. LaTER-SFT averages 41–47 and 43–52 latent steps, respectively. LaTER-TF averages 61–63 and 52–61 latent steps.

We have tried to respond directly to the concerns raised in the reviews through the additional experiments and clarifications summarized above. We would be grateful if you could consider these additions alongside the original submission and reviews when making the final assessment.

Thank you for your time and consideration. We wish you all the best.

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.