acceptodds
Under review as a conference paper at ICLR 2027

Rethinking Fixed-Horizon Latent Reasoning

Abstract

Existing latent reasoners typically run for a preset latent horizon during inference. Intuitively, a longer horizon should provide more opportunity for reasoning. However, in this work, we find that extending latent Chain-of-Thought (CoT) to the full preset horizon may in fact degrade reasoning performance. We observe that latent CoT evolves through distinct functional stages, with earlier computation forming frontier reasoning states, while later computation mainly reorganizes existing candidates and can weaken gold candidate support. These findings motivate a rethink of fixed-horizon latent inference. To address this, we characterize the reasoning benefits of latent CoT by realized and remaining constructive progress. Empirically, we find that higher realized constructive progress corresponds to higher current reasoning accuracy, while more remaining progress corresponds to larger future gains from continued latent CoT. To this end, we introduce a lightweight method that estimates the remaining reasoning progress and adaptively adjusts the latent horizon during inference. Across representative latent CoT reasoners and multiple arithmetic benchmarks, our method improves reasoning accuracy by up to percentage points while reducing latent computation by up to , without retraining the LLM.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, latent reasoning, latent chain-of-thought

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_kpNtOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper demonstrates that fixed-budget latent reasoning models can get improved accuracy with an early termination signal. The first stage of latent reasoning often promotes the gold solution, but the later stage can reverse it and promote an incorrect answer. This paper presents a method that can estimate the benefit of continuing latent reasoning and how many more latent tokens should be generated. Their method largely outperforms fixed-budgets of latent tokens.

Critical strengths

S1) Clearly demonstrates that additional latent reasoning is not always beneficial, and that accuracy can be improved by estimating the realized constructive progress and early termination of reasoning. This work sits well amongst concurrent work in making more efficient explicit reasoning models.

Critical weaknesses

W1) I think some of the gains over full-k in table 1 are due to the fact that GSM8k, SVAMP, and MultiArith generally need fewer reasoning steps than the number of latent steps. Most gold label solutions in GSM8k problems are 2-4 steps, SVAMP is 1-2 steps, and MultiArith is always 2 steps. The Coconut and CODI models are trained such that one latent step is roughly corresponds to 1 gold label step. So having 6 or 10 latent steps (depending on the model) is almost always more steps than needed, so it's not surprising to me that early termination in table 1 can beat full-k. What would the results in table 1 look like if you had a test set where the gold label number of solution steps was balanced, from 1-6 or 1-10 steps?

W2) Table 8 shows that two baselines (stability and confidence) perform about as well as your method. What is the main benefit of using your progress method over these baselines? Also, Table 8 should probably be in the main text.

Questions for authors

Q1) Frontier advancement makes sense for ProsQA since the model is searching through a DAG. But what does frontier advancement mean for math problems like those in GSM8k?

Q2) In Figure 3, inserting a discrete token should hurt the model's accuracy (and thus % gold candidates formed) since passing in a discrete token is OOD for Coconut and CODI during reasoning. Can you provide more detail on the motivation for this experiment? And how did you form the "matched control that perturbs the continuous state by a comparable magnitude" in panel (b)?

Q3) For table 8, can you add how stability, confidence, and convergence are calculated?

Reviewer_dNoTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how fixed latent reasoning horizons can waste computation and overwrite correct outcomes. It proposes three functional stages—frontier advancement, candidate organization, and answer revision—and uses a lightweight predictor of remaining constructive progress to allocate additional computation for frozen reasoners. Experiments on CODI, Coconut, and SIM-CoT variants across three arithmetic benchmarks report accuracy improvements of up to 7.2 percentage points and latent-step reductions of up to 29.0%, occurring in different configurations.

Critical strengths

S1. The problem is practically relevant. Figure 1 and Section 3 show that correct intermediate outcomes can be changed by subsequent latent computation, motivating instance-dependent compute allocation.

S2. The approach is empirically feasible. Table 1 demonstrates accuracy–compute improvements with frozen reasoners, and Table 9 provides out-of-sample evidence that the learned signal contains information about future gains.

Critical weaknesses

W1. The connection between the identified challenges and the design is incomplete. Section 4.1 notes that state changes need not represent useful progress and that future gains are unavailable online, but Appendix A does not identify concrete failure cases of prior adaptive methods or establish how the new design resolves them. More importantly, Ω_t in Eq. (2) measures improvement over the historical best, whereas the decision concerns the benefit of continuing from the current state. Appendix C.2, Eq. (9), explicitly gives ΔG_t(a)=P_t(a)+d_t−d_{t+a}, showing that signed gains also depend on the current and subsequent gaps to the attained frontier. Yet the controller in Eqs. (4)–(5) primarily allocates computation using predicted Ω_t, without explicitly modeling these gaps or the timing of gains. Proposition 4.1 is a generic property of squared-loss regression, and Theorem 4.2 assumes increasing differences; neither closes this decision-level gap. The necessity of the new target relative to existing stopping criteria, and the conditions under which it is appropriate, therefore remain unclear.

W2. The experiments do not adequately establish the incremental contribution of the specific design. Tables 1–2 primarily compare against Full-K and fixed depths, rather than existing dynamic-horizon latent CoT methods; Table 2 matches maximum budgets rather than actual average computation. Table 8 usefully compares adaptive signals under the same controller and matched average computation, but is not a full comparison with published methods. Moreover, progress outperforms confidence in only three of six settings; on MultiArith / GPT-2 / SIM-CoT it achieves 89.4%, compared with 91.7% for confidence. A controlled target ablation keeping features, predictor capacity, amount of supervision, and controller fixed is missing, as are experiments isolating the trajectory features, percentile transformation, and budget mapping. The predictive associations in Table 9 do not replace these controls. Consequently, the gains may reflect generic instance-dependent allocation or additional supervision, rather than the proposed progress definition and particular design choices.

Questions for authors

Q1 [Method clarification] What concrete failure modes of existing adaptive methods does this work address, and which design choices resolve each failure? Given Appendix C.2, under what conditions can computation be allocated primarily from Ω_t without explicitly considering the gap between the current state and the historical best, or the timing of future gains?

Q2 [Experimental validation] Can the authors compare against the closest dynamic latent CoT methods at matched actual average compute, and provide target ablations controlling features, model capacity, supervision, and controller, together with ablations of the key features and budget mapping? These results would materially affect the assessment of novelty and attribution.

Reviewer_1H75OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper shows that extending latent CoT to the full horizon can degrade reasoning performance. The study observes that latent CoT evolves through distinct functional stages: frontier advancement, candidate organization, and answer revision. The paper argues that the later stages reorganize candidates that have already emerged in the previous stages. The study also introduces a lightweight method that estimates the remaining reasoning progress and adaptively adjusts the latent horizon during inference.

Critical strengths
  1. The paper shows how the reasoning benefits evolve across latent steps during CoT generation.
  2. The study shows how continuing the latent reasoning can degrade performance.
Critical weaknesses
  1. The models used for latent CoT used in this study are different from reasoning models (Qwen3.X, gpt-oss, gemma-4 etc). It is easy to check if the next token with the highest probability is the correct answer when the answer is a number, but for complex reasoning this approach falls short.
  2. Figure 2 visually tells how extending latent reasoning to the full horizon can hurt accuracy, but the evidence to support this claim in general is limited. Table 1 also shows that using the estimator trained to stop CoT before reaching the horizon does increase accuracy only marginally, indicating that early stopping might not be learnable.
Questions for authors
  1. Why is MultiArith used for this study if the models already perform with ~95% accuracy on it?
  2. Would it be possible to fine-tune a model using GRPO/DAPO based methods to make the model learn to stop early using the task reward presented in the paper?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.