acceptodds
Under review as a conference paper at ICLR 2027

Diversity Without Discovery: Why Chain Perturbation Fails to Explore in Latent Test-Time Scaling

Abstract

Parallel test-time scaling (PTTS) improves reasoning by sampling multiple chain-of-thought (CoT) traces and aggregating them, marginalizing over the reasoning paths the model inherently finds plausible. Various SOTA latent-reasoning models instead evolve continuous hidden states under deterministic dynamics, following one trajectory per input with no native randomness to sample from. Recent work therefore perturbs the latent chain at test time and presents the resulting answer diversity as exploration potential in continuous space. To test this, we fairly compare latent perturbation methods with standard explicit reasoning on a shared backbone and find that, for unconstrained latent thoughts, i.e., thoughts not restricted to the embedding space, aggregating perturbed latent chains brings no gain over the no-sampling baseline, whereas explicit reasoning turns the same budget into accuracy gains. Our formal analysis demonstrates that explicit reasoning marginalizes over the trajectory distribution stochastically exhibited by the model, while a perturbed state comes from no such distribution, so its vote carries no belief of the model and no guarantee of favoring the correct answer. Empirically, the added diversity comes predominantly from borderline samples and, unlike with explicit reasoners, a correct answer reached through perturbation carries no gain in confidence. As such, even a frozen latent chain, perturbed at most once at the answer step, can be calibrated to reproduce or outperform every full-chain perturbation method we tested at no additional reasoning-token cost.

Primary area:
foundation or frontier models, including LLMs
Keywords:
latent reasoning, test-time scaling, chain-of-thought, parallel sampling, continuous-space reasoning, reasoning geometry, large language models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_6kWsOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates how to sample diverse reasoning trajectories in latent reasoning with large language models. The experiments show that gains in pass@k, whether achieved through standard sampling or full-chain perturbation, do not translate into improvements in maj@k. Motivated by this finding, the authors propose Single Calibrated Perturbation and provide a theoretical analysis of why pass@k behaves differently in latent reasoning than in explicit chain-of-thought (CoT) reasoning.

Critical strengths
  1. The paper addresses a meaningful problem and presents a coherent and thoughtful research approach. It begins with empirical observations, uses these findings to motivate an improved method, and then provides theoretical analysis to explain the underlying mechanisms in greater depth.

  2. The paper offers a valuable and insightful analysis of the differences between sampling reasoning trajectories in latent reasoning and in standard chain-of-thought (CoT) reasoning. This comparison provides useful perspectives for future research in the field.

Critical weaknesses
  1. The models used in the experiments are somewhat dated and relatively weak. This is a concern because CoT-related experimental results can depend strongly on model capability.

  2. My main concern is that, if a model struggles to produce correct answers, sampling multiple CoT trajectories may increase pass@k without improving maj@k. To what extent is this phenomenon specific to perturbations of latent reasoning trajectories, rather than a consequence of limited model capability? Can the proposed method effectively address this underlying issue?

Questions for authors

I find the paper’s analysis of the differences between sampling reasoning trajectories in latent reasoning and in standard chain-of-thought (CoT) reasoning valuable. However, the claim that latent-state perturbations cannot uncover better reasoning trajectories seems overly strong. If the underlying model is sufficiently capable, could such perturbations increase trajectory diversity and also improve maj@k? As noted in the weaknesses section, it remains unclear whether the observed limitation is inherent to latent-state perturbation or reflects the limited capabilities of the models evaluated.

6 comments

Weakness 1: The models used in the experiments are somewhat dated and relatively weak. This is a concern because CoT-related experimental results can depend strongly on model capability.

W1. We appreciate the valuable comment of the reviewer and the interest in our work. We agree that the models we evaluate are relatively small, with backbones from 124M to 3B parameters (Appendix E). These are the scales at which current latent reasoning models are released and evaluated by prior work, so our setting matches theirs [1, 2, 3, 4]. Our results nevertheless show a fundamental distinction between the two reasoning paradigms. With dual-mode models, we compare both paradigms on the same backbone. The same model shows signs of exploration in its explicit mode, with maj@k above its greedy accuracy, yet fails to improve maj@k under latent perturbation (Section 3.2). This failure holds for every unconstrained-thought model we tested, regardless of its size. Since the explicit and latent modes share one backbone, model capability alone cannot explain the gap, which instead points to how latent perturbation explores. Our reply to W2 further clarifies the degenerate exploration that we observe and formalize.

[1] Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training Large Language Models to Reason in a Continuous Latent Space. arXiv:2412.06769, 2025.

[2] Zhenyi Shen, Hanqi Yan, Linhai Zhang, Zhanghao Hu, Yali Du, and Yulan He. CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation. arXiv:2502.21074, 2025.

[3] Xilin Wei, Xiaoran Liu, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Jiaqi Wang, Xipeng Qiu, and Dahua Lin. SIM-CoT: Supervised Implicit Chain-of-Thought. arXiv:2509.20317, 2025.

[4] Runyang You, Yongqi Li, Meng Liu, Wenjie Wang, Liqiang Nie, and Wenjie Li. Parallel Test-Time Scaling for Latent Reasoning Models. arXiv:2510.07745, 2026.

Weakness 2: My main concern is that, if a model struggles to produce correct answers, sampling multiple CoT trajectories may increase pass@k without improving maj@k. To what extent is this phenomenon specific to perturbations of latent reasoning trajectories, rather than a consequence of limited model capability? Can the proposed method effectively address this underlying issue?

W2. We thank the reviewer for this valuable comment.

Model capability versus failure of exploration. We revised our manuscript to directly explain the degenerate exploration exhibited by latent perturbation methods, and to show that it stems from the reasoning paradigm, i.e., perturbation-based exploration in latent reasoning, rather than from limited model capability. As in our reply to W1, our dual-mode setting isolates this failure, since the same backbone improves maj@k in its explicit mode but not under latent perturbation. Formally, we characterize in Section 4 how both AGN and MC-dropout switch answers by their margin fragility [1], and we bound their switching probabilities, so the correct answer is one candidate among the alternatives and therefore gains no probability for being correct (Proposition 2, Corollaries 2.1 and 2.2, and Proposition 3 for the full chain). Empirically, the answers induced by latent perturbation remain within the reference support [2] of the original model. In contrast, explicit sampling not only discovers answers outside its support but also improves the majority vote steadily (newly added in Section 3.2, Figure 4). Together, these results demonstrate, both theoretically and empirically, the degenerate exploration that we observe.

Can the proposed method address this issue? We proposed the single calibrated perturbation as a control, to show that full-chain perturbation is not necessary given the degenerate exploration of the latent models in our study. With this experiment, we demonstrated that the answer distribution of full-chain perturbation can be recovered by a single calibrated perturbation (Section 3.4). In this revision, Corollary 2.2 and the Proposition 3 we added predict this behavior for both AGN and MC-dropout: to first order, within each method, full-chain perturbation and a single perturbation move the answer margins in the same way. We additionally verify our theoretical bounds by comparing the switching rates they predict with those observed under full-chain AGN, which closely match (new Appendix G). Accordingly, we establish that perturbation methods for unconstrained-thought models cannot be directly used to perform latent PTTS.

[1] Jeremy M. Cohen, Elan Rosenfeld, and J. Zico Kolter. Certified Adversarial Robustness via Randomized Smoothing. ICML 2019.

[2] John Hewitt, Christopher D. Manning, and Percy Liang. Truncation Sampling as Language Model Desmoothing. Findings of EMNLP 2022.

Question 1: I find the paper’s analysis of the differences between sampling reasoning trajectories in latent reasoning and in standard chain-of-thought (CoT) reasoning valuable. However, the claim that latent-state perturbations cannot uncover better reasoning trajectories seems overly strong. If the underlying model is sufficiently capable, could such perturbations increase trajectory diversity and also improve maj@k? As noted in the weaknesses section, it remains unclear whether the observed limitation is inherent to latent-state perturbation or reflects the limited capabilities of the models evaluated.

Q1. We thank the reviewer for raising this question about our main claim that latent-state perturbations cannot uncover better reasoning trajectories. As discussed in our reply to W2, we formally model this failure (Section 4) and connect it to the literature on adversarial and random perturbations [1, 2]: a perturbation switches the answer when it crosses the decision boundary of an alternative (new Figure 6, Appendix B.5), so the induced answers are governed by their margin fragility. We verified these claims experimentally with the answer-overlap analysis (Section 3.2, new Figure 4) and with a comparison of the predicted and observed switching rates (new Appendix G).

With regard to model capability, our replies to W1 and W2 show that the same backbone improves maj@k in its explicit mode but not under latent perturbation, and that this failure holds for every unconstrained-thought model we tested, from 124M to 3B parameters (Section 3.2, Appendix E).

[1] Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks. CVPR 2016.

[2] Alhussein Fawzi, Seyed-Mohsen Moosavi-Dezfooli, and Pascal Frossard. Robustness of Classifiers: From Adversarial to Random Noise. NeurIPS 2016.

Reviewer_Rc2aOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper investigates whether perturbing latent reasoning enables exploration for parallel test-time scaling. It finds that MC-dropout and additive Gaussian noise increase pass@k but provide little or no improvement in majority-voting accuracy, unlike explicit CoT. The study evaluates multiple latent reasoners, uses dual-mode models to compare explicit and latent reasoning on the same backbone, and examines answer entropy. It also shows that post-reasoning token perturbation can match or outperform full-chain perturbation. The paper provides a theoretical analysis intended to explain these findings through the distinction between sampling explicit reasoning trajectories and injecting noise into deterministic latent chains.

Critical strengths

The writing and delivery is easy to follow.

The paper identifies an interesting phenomenon: latent-chain perturbation improves pass@k without translating into voting gains.

The ablation in Section 3.4 is compelling. Dropout perturbation at the first answer token largely reproduces the answer distribution of full-chain perturbation. This provides more direct empirical support that propagating perturbations in the latent reasoning trajectory contributes little additional exploration.

The comparisons are solid and well controlled. Using dual-mode models such as CODI, MUX, and SIM-CoT allows explicit and latent reasoning to be compared on the same backbone, making the comparison more convincing than one across separately models. The experiments also cover quite a broad range of latent reasoners, lending further credibility to the empirical findings.

Critical weaknesses

Limited empirical contribution. Most of the evaluation reuses existing latent reasoning architectures and established perturbation methods from You et al. (MC-dropout and AGN). The primary new settings / experiments are checking entropy and post-reasoning perturbation, which are too minimal and trivial for a published paper.

The theoretical analysis is invalid. The paper's finding is that both AGN and MC-dropout fail to improve voting. Thus the reason behind such results should be based on property shared by both methods. But the proposed explanation (Section 4, Proposition 1 &2) is derived for noise only. Therefore the theoretical analysis is simply explaining adding noise instead of actually answering why. So again, this is a trivial contribution.

Overall, the paper identifies a simple but interesting phenomenon and supports it with well-controlled experiments. However, the empirical contribution remains limited, and the theoretical analysis did not explain why both AGN and MC-dropout fail. I find the overall contribution insufficient for acceptance.

Questions for authors

See weaknesses.

6 comments

Weakness 1: Limited empirical contribution. Most of the evaluation reuses existing latent reasoning architectures and established perturbation methods from You et al. (MC-dropout and AGN). The primary new settings / experiments are checking entropy and post-reasoning perturbation, which are too minimal and trivial for a published paper.

W1. We appreciate the comments of the reviewer. Our experimental settings deliberately reuse existing methods, with the aim of exploring and understanding the phenomenon of degenerate exploration in latent PTTS through the lens of dual-mode models. MC-dropout and AGN are the established perturbation methods for latent PTTS, so they are the subject of our study rather than its contribution. Our contribution is in line with analysis studies that examine established methods and clarify their limitations, where the analysis itself is the main contribution [3, 4, 5]. In this revision, we strengthen our empirical investigation with two new analyses of these methods, which provide direct evidence of the degenerate exploration that occurs under both AGN and MC-dropout. First, a new answer-overlap analysis (Section 3.2, new Figure 4) builds the reference support [1] of both paradigms, the set of answers to which the model assigns non-negligible probability under its fixed reasoning: the deterministic chain for latent reasoning and the greedy CoT for explicit reasoning. Across a wide range of sampling strengths, each perturbation method keeps near-complete overlap with this support, without improving maj@64. In contrast, explicit sampling not only discovers answers outside its support but also improves the majority vote steadily across sampling strengths. Hence, the diversity induced by latent perturbation is confined to answers that the unperturbed chain already supports, contrary to the behavior expected of PTTS. Second, in this revision, we characterize the probability of each induced answer (more details in our reply to W2) and show that answers are selected by a fixed transform, whose switching probabilities we bound theoretically. To test this characterization directly, we introduce a novel experiment (new Appendix G, Algorithm 1). For each alternative answer, we search for the smallest perturbation [2] of the latent chain that switches the answer to it, which measures its distance to the decision boundary, and we predict its switching rate from this distance. Across different models, the predicted rates closely match the rates observed under full-chain AGN (Figure 8), both across the rank of the alternative and across its margin to the greedy answer. The experiment connects our analysis with the well-studied literature on adversarial perturbations [2]. It offers a direct explanation of the degenerate exploration under both methods, and of how margin fragility (Figure 1b) governs it.

To summarize, the supporting experiments show the following.

  • Degenerate exploration is a common failure of AGN and MC-dropout. Both perturb the fixed latent reasoning chain, and the induced answers switch according to the margins between answers, where the correct answer is one candidate in the support and gains no probability for being correct. The flat maj@k is therefore a symptom of the perturbation mechanisms themselves, and not an isolated failure of the vote.
  • This phenomenon is characterized formally, to first order for the full chain, and verified with a direct experiment whose predictions closely match the observed switching rates (new Appendix G).
  • Together with our previous findings, these results reveal a non-trivial distinction between the two reasoning paradigms, and show that the spurious diversity of latent perturbation can be misleading when measured by pass@k.

[1] John Hewitt, Christopher D. Manning, and Percy Liang. Truncation Sampling as Language Model Desmoothing. Findings of EMNLP 2022.

[2] Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks. CVPR 2016.

[3] Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity Checks for Saliency Maps. NeurIPS 2018.

[4] Sarthak Jain and Byron C. Wallace. Attention is not Explanation. NAACL 2019.

[5] Miles Turpin, Julian Michael, Ethan Perez, and Samuel R. Bowman. Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting. NeurIPS 2023.

Weakness 2: The theoretical analysis is invalid. The paper’s finding is that both AGN and MC-dropout fail to improve voting. Thus the reason behind such results should be based on property shared by both methods. But the proposed explanation (Section 4, Proposition 1 & 2) is derived for noise only. Therefore the theoretical analysis is simply explaining adding noise instead of actually answering why. So again, this is a trivial contribution.

W2. We appreciate the comments of the reviewer and revised the manuscript to clarify these points. We agree that the explanation must rest on a property shared by AGN and MC-dropout. The submitted version partly covered this in the appendix (Proposition 7, now Proposition 8 in Appendix B.5), which holds for both methods. The revised Proposition 2 in Section 4 now establishes it in the main text. Our derivation proceeds backwards from the output head. We first derive the effect of AGN exactly for a single perturbation of the state that enters the output head, and then extend it to MC-dropout to first order through the model weights [1] (Appendix B.2). In both cases, the noise on the logit of each answer is a fixed function of the model weights rather than of the model likelihood. The added Corollaries 2.1 and 2.2 then quantify the margin fragility that causes the change in answers [2]: the probability of switching to each alternative is bounded by its margin relative to the noise, and the correct answer is one candidate among these alternatives that gains no probability for being correct (proofs in Appendix B.5). To complete this connection, we added Proposition 3, which extends this result to full-chain perturbation to first order. Although AGN and MC-dropout perturb the model in different ways, both switch answers through the margin fragility of the alternatives in the support of the original model, and this shared mechanism makes the exploration of each method degenerate. Empirically, as discussed in our reply to W1, each method keeps near-complete overlap with the reference support of the unperturbed chain (Section 3.2, new Figure 4), and the switching rates predicted by the theory closely match the observed ones (new Appendix G). Accordingly, our formulation explains degenerate exploration as a property of perturbation-based PTTS in latent models, and not of the added noise alone.

[1] Stefan Wager, Sida I. Wang, and Percy Liang. Dropout Training as Adaptive Regularization. NeurIPS 2013.

[2] Jeremy M. Cohen, Elan Rosenfeld, and J. Zico Kolter. Certified Adversarial Robustness via Randomized Smoothing. ICML 2019.

Overall, the paper identifies a simple but interesting phenomenon and supports it with well-controlled experiments. However, the empirical contribution remains limited, and the theoretical analysis did not explain why both AGN and MC-dropout fail. I find the overall contribution insufficient for acceptance.

We thank the reviewer for the careful assessment of our work, and for recognizing the phenomenon and the experiments that support it. We took both concerns seriously and revised the manuscript accordingly. Regarding the empirical contribution, we added a new answer-overlap analysis and a direct experiment on switching rates (our reply to W1). Regarding the theoretical analysis, the revised Section 4 now explains the failure through a mechanism shared by AGN and MC-dropout (our reply to W2). We hope that our rebuttal clarifies the significance of our contribution, and we are willing to address any further concerns that the reviewer may have.

Reviewer_oFjfOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper’s main finding is that, for several current latent reasoning models, injecting noise throughout the hidden reasoning process produces more varied answers but little improvement in majority-vote accuracy. Computing one unchanged latent reasoning chain and sampling multiple answers from it often works better, while requiring much less repeated reasoning. The authors interpret this as evidence that the perturbations mainly create output variation rather than useful exploration of alternative reasoning paths.

Several noteworthy findings:

  1. More samples uncover correct answers, but voting barely improves
  2. Sampling answers from a frozen chain is a strong baseline
  3. One perturbation near the answer can imitate perturbing the whole chain
  4. Their confidence analysis suggests that noise mainly affects fragile predictions
Critical strengths
  1. The observation is important and serves as a warning to the rest of the community in this regard. The frozen-chain baseline is valuable in that it is cheap and easily reproducible.
  2. The empirical results are shown to span several models. The authors do provide exceptions and note theoretical limitations.
Critical weaknesses
  1. the distinction between “trained belief” and “uninformed noise” is not justified
  2. failure of majority voting does not identify failure of exploration
Questions for authors

Flat majority accuracy does not establish an absence of useful alternative reasoning. Useful candidates could be present but require a better selector. Also similar output distributions do not establish identical internal computations. Can we do something to bridge these two gaps?

6 comments

Weakness 1: the distinction between “trained belief” and “uninformed noise” is not justified

W1. We thank the reviewer for pointing to this distinction. By trained belief we refer, in the context of explicit CoT, to the ability of the model to reach correct answers, which it acquires through training and which defines its distribution over reasoning paths (Section 2.1). PTTS then explores the reasoning space through this belief to locate a correct answer [1]. In contrast, we describe perturbation methods as uninformed because our analysis shows that they apply a fixed transform that switches the answers of the model. To make this clearer to the reader, we updated the manuscript to quantify both, theoretically (updated Proposition 2, and the added Corollaries 2.1 and 2.2 and Proposition 3 in Section 4) and empirically (new Figure 4 in Section 3.2 and new Appendix G). Formally, we derive a probability bound on the selected answers and show that they follow the ranks of the alternatives in the support [2] of the original model, and accordingly conclude that perturbation-based exploration is degenerate. We confirm this empirically with a new answer-overlap analysis (new Figure 4). Across a wide range of sampling strengths, perturbation (1) yields answers that almost perfectly overlap with the reference support [2] of the unperturbed chain, and (2) shows degenerate maj@k performance, unlike explicit CoT sampling.

[1] Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-Consistency Improves Chain of Thought Reasoning in Language Models. ICLR 2023.

[2] John Hewitt, Christopher D. Manning, and Percy Liang. Truncation Sampling as Language Model Desmoothing. Findings of EMNLP 2022.

Weakness 2: failure of majority voting does not identify failure of exploration

Question 1: Flat majority accuracy does not establish an absence of useful alternative reasoning. Useful candidates could be present but require a better selector. Also similar output distributions do not establish identical internal computations. Can we do something to bridge these two gaps?

W2 and Q1. We thank the reviewer for raising this point.

W2 and Q1, Gap 1 (useful candidates and a better selector). We clarify that the observed failure is not restricted to maj@k. As discussed in our reply to W1, perturbation selects answers by a fixed transform of the local decision geometry of the model. A random perturbation switches the answer when it crosses the decision boundary of an alternative (new Figure 6, Appendix B.5), the random-noise counterpart of the minimal adversarial perturbation [1, 2]. The selected answers therefore follow the ranks of the alternatives in the support [3] of the original model. Exploration is thus degenerate to the single chain of the latent model, and the flat maj@k quantifies this underlying failure of perturbation methods. This finding is indirectly supported by Wang et al. [4], who observe that perturbed latent chains are geometrically indistinguishable and that trained process and outcome reward models bring only marginal gains in selecting among them. Since the perturbed chains carry no distinguishable reasoning, a selector that picks the correct answer from the original support must already solve the question on its own. It is then an answer predictor rather than an aggregator of reasoning paths.

Q1, Gap 2 (internal computations). We agree that similar output distributions alone do not establish identical internal computations, and we do not infer them from the output match. Our claim is narrower: propagating the noise through the chain adds no exploration in how the answer is selected. The added Proposition 3 shows that, to first order, full-chain perturbation moves the answer margins in the same way as a single perturbation, and the two differ mainly in strength. Empirically, a single calibrated perturbation at the first answer token reproduces the answer distribution of full-chain perturbation question by question (Section 3.4). We also verify empirically the theory that explains this match (added Proposition 3, new Appendix G). The switching rates observed under full-chain AGN fall with the rank and the margin of each alternative, as the theory predicts. Both results indicate that the propagated noise selects answers through the same local margins, and not through alternative reasoning in the chain.

[1] Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks. CVPR 2016.

[2] Alhussein Fawzi, Seyed-Mohsen Moosavi-Dezfooli, and Pascal Frossard. Robustness of Classifiers: From Adversarial to Random Noise. NeurIPS 2016.

[3] John Hewitt, Christopher D. Manning, and Percy Liang. Truncation Sampling as Language Model Desmoothing. Findings of EMNLP 2022.

[4] Minghan Wang, Thuy-Trang Vu, Ehsan Shareghi, and Gholamreza Haffari. Towards Inference-time Scaling for Continuous Space Reasoning. Findings of ACL 2026.

We thank the reviewer again for the time and the constructive feedback, which helped us improve the manuscript. We have uploaded a revised version that incorporates the changes described in our replies above. We hope that these revisions address the concerns of the reviewer, and we are happy to clarify any remaining points during the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.