acceptodds
Under review as a conference paper at ICLR 2027

Learning to Reason More Implicitly Improves Explicit Reasoning on Harder Problems

Abstract

Large language models (LLMs) trained with chain-of-thought (CoT) on simpler problems often fail to generalize to harder problems that require more reasoning steps. Humans, by contrast, can learn to solve harder problems by practicing on simpler problems until they can perform known steps mentally. Inspired by this, we ask whether LLMs can also generalize to harder problems by learning to reason more implicitly, i.e., to perform more steps without generating intermediate tokens, even when they still generate CoTs at test time. In this work, we propose step dropout (SeDo), a simple approach for teaching models to reason more implicitly. SeDo trains models on both complete CoTs and CoTs with randomly omitted steps. This teaches models individual steps while also encouraging them to implicitly infer missing CoT steps. Across five models and four tasks, our results show that SeDo both (1) substantially improves the learning efficiency of implicit multi-hop reasoning, and (2) helps models generalize to harder problems. Surprisingly, SeDo models generalize better even when given complete CoT prefixes, indicating that implicit reasoning also teaches more generalizable skills, beyond robustness to incomplete steps. We further show implicit reasoning helps explicit reasoning in models not trained with SeDo: in looped Transformers, more loops, which allow more computation for implicit reasoning, monotonically improve single-step accuracy within a CoT, even on calculations the models already solve perfectly without CoT.

Primary area:
foundation or frontier models, including LLMs
Keywords:
implicit reasoning, complexity generalization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_rS3XOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Step Dropout (SeDo), which randomly omits complete reasoning steps from chain-of-thought (CoT) training examples and trains on both complete and partial CoTs. The goal is to teach models to predict across multiple reasoning steps, thereby improving implicit computation and generalization to problems with greater reasoning complexity. The paper also studies SeDo-R, which replaces omitted steps with mask tokens and adds prediction targets for the masked steps. The evaluation includes direct-answer accuracy on the DEPO multi-hop graph task, complexity generalization on Maze, Quickselect, and GSM-∞, and analyses using complete-CoT prompts and linear probes. The reported results show improvements from SeDo on several tasks and model families, although the evidence for the proposed internal mechanism and the behavior of SeDo-R on GPT2-RoPE remain unclear.

Critical strengths
  1. Simple, general, and practical. SeDo requires no architectural changes or auxiliary prediction heads and can be applied during post-training of pretrained models. It is lighter than methods such as MTP and FSP that require auxiliary layers.
  2. Broad evaluation with a relatively complete body of evidence. The paper evaluates implicit multi-hop reasoning on controlled DEPO and OOD complexity generalization on Maze, Quickselect, and GSM-∞. It also includes RL, self-generated CoT, linear-probe, and looped-Transformer experiments.
  3. Clear presentation and detailed experimental setup. The appendix describes data generation, training details, the SeDo-R implementation, RL, and self-generated-data settings, supporting reproducibility.
Critical weaknesses
  1. The probe and looped-Transformer results are correlational, not causal. The probe shows omitted values are more decodable in SeDo states, not that the model uses them. The looped-Transformer result is also correlational and warrants more cautious interpretation.

  2. SeDo's gains may come from structured data augmentation. SeDo creates step-dropped partial-CoT views of each training example, but current experiments do not distinguish the contribution of this augmentation from implicit computation of omitted steps. If the main gains come from data augmentation, the paper's motivation and contribution regarding implicit reasoning would be fundamentally weakened. A matched control is needed to distinguish these explanations.

  3. Important performance differences remain unexplained. The paper attributes SeDo's weaker small-data results on randomized Maze to the need for more training, but does not explain the large p@1p@1/p@16p@16 gap on hard Quickselect or the GPT2-RoPE SeDo-R collapse on DEPO.

Questions for authors

Please address the concerns raised in W1–W3 above.

3 comments
AuthorsOpenReview ↗

Response to Reviewer rS3X (1/3)

Thank you for your detailed and helpful review. We have revised the paper, with changes marked in blue.

  1. “The probe and looped-Transformer results are correlational, not causal. The probe shows omitted values are more decodable in SeDo states, not that the model uses them. The looped-Transformer result is also correlational and warrants more cautious interpretation.”

    Thank you for this comment. We agree that our probes alone do not show that models use the decoded values, and we have conducted an intervention experiment to test this.

    Probes and erasure. We test from three aspects whether SeDo models compute and use omitted values implicitly, with behavioral, probing, and intervention experiments on GSM-∞\infty. Specifically, we omit the line that calculates a variable VV, and ask the model to predict a dependent value UU, e.g., complete U = V * 2 =. We use the 500 held-out probing problems from Section 5.1 with Qwen3-1.7B.

    First, SeDo models can predict future steps without the omitted dependencies provided in the CoT context: without VV's defining line, SeDo predicts UU with 86.4% accuracy, compared with 2.6% for Full.

    Second, VV's value is represented in the hidden states: linear probes at the final = decode VV's value much more accurately from SeDo than from Full (Section 5.1, Figure 5).

    Third, SeDo models use this representation to predict UU, which the probes alone do not show. We erase VV's value at the same position with LEACE (Belrose et al., 2023), jointly in all layers of Qwen3-1.7B, and measure whether the model still predicts UU correctly. As a baseline, we instead remove as many random directions as LEACE does (averaged over five random seeds). We report the accuracy (%) of predicting UU:

CoT prefixModelNo erasureRandom erasureLEACE
With VV's defining lineFull98.2098.12 (−0.08)98.40 (+0.20)
SeDo99.8099.76 (−0.04)99.40 (−0.40)
Without VV's defining lineFull2.602.64 (+0.04)4.20 (+1.60)
SeDo86.4086.84 (+0.44)78.00 (−8.40)

With VV's defining line omitted, LEACE reduces SeDo's accuracy from 86.4% to 78.0%, while random erasure has no effect. In contrast, with the line present, where the model can read VV's value, LEACE has little effect. This suggests that SeDo models use the implicitly computed values to predict subsequent steps. The effect is partial, likely because LEACE does not remove all information about VV: after erasure, probes still decode VV with 16.0% accuracy (48.4% before erasure; 5% by chance). VV may also be represented at earlier positions or nonlinearly. We have added this experiment to Section 5.1 and Appendix H (Table 15).

Looped Transformers. We agree that this result is correlational, and have interpreted it more cautiously in the revision. However, looping is a simple approach to scale implicit computation without changing the model parameters. We show that more implicit computation improves both implicit reasoning and single-step CoT reasoning, even on calculations the models already solve perfectly without a CoT. This parallels the finding of Sinha et al. (2026) that larger models execute far more steps correctly, even when smaller models have near-perfect single-step accuracy; here, we increase computation through looping instead of model size. We include this experiment as a demonstration beyond SeDo: in models not trained with SeDo, stronger implicit reasoning also correlates with better explicit reasoning. We have revised Section 5.3 accordingly.

AuthorsOpenReview ↗

Response to Reviewer rS3X (2/3)

  1. “SeDo's gains may come from structured data augmentation.”; “If the main gains come from data augmentation, the paper's motivation and contribution regarding implicit reasoning would be fundamentally weakened. A matched control is needed to distinguish these explanations.”

    Thank you for raising this point. We agree that clear baselines are needed to distinguish the benefits of implicit reasoning from those of structured data augmentation. We address this in two ways. First, we compare SeDo with token-level augmentation baselines, and with a baseline trained on additional new problems. Second, we show that SeDo models compute intermediate results that are not written in the CoT, and use them to predict later steps.

    Comparison with augmentation baselines. To test whether SeDo's gains come from trace augmentation alone, in the paper we compare SeDo with two trace augmentation baselines, Token Drop and MFT (Table 2), which perturb the same CoTs by dropping or masking tokens instead of whole steps. SeDo outperforms both in nearly every OOD range across models and datasets. For example, on GSM-∞\infty with 31≤H<3631 \le H < 36, SeDo with Qwen3-1.7B reaches 49.2% / 77.1% (p@1 / p@16), compared to 34.2% / 62.1% for Token Drop and 39.2% / 69.6% for MFT.

    Since data augmentation approximates additional data, we further compare SeDo with an upper bound: a Full-fresh baseline on Qwen3-1.7B, which replaces SeDo's partial CoTs with complete CoTs of new problems. Concretely, Full-fresh is trained on four to five times as many distinct problems as SeDo. We report OOD p@1 / p@16, with OOD ranges merged:

TaskOOD rangeFullFull-freshSeDo
Maze[200, 650)35.96 / 50.8044.68 / 61.8645.53 / 70.74
Quickselect[140, 380)41.75 / 76.7546.42 / 77.4259.67 / 96.83
GSM-∞\infty[26, 36)47.71 / 67.9261.88 / 83.9655.00 / 80.83

SeDo outperforms Full-fresh on Maze and Quickselect (e.g., 96.83% vs. 77.42% p@16 on Quickselect). On these tasks, SeDo's gains thus exceed those from five times as many new problems, suggesting that they do not come from data augmentation alone. On GSM-∞\infty, Full-fresh is higher, but SeDo still achieves about half (p@1) to four fifths (p@16) of Full-fresh's gain over Full. We have added this baseline to Table 2 and Section 4.1.

Evidence of implicit computation. Our behavioral, probing, and intervention results in point 1 further suggest that SeDo models compute omitted values implicitly and use them in subsequent predictions. In particular, without VV's defining line, SeDo predicts the dependent value UU with 86.4% accuracy, compared with 2.6% for Full, and erasing VV's value with LEACE reduces this accuracy to 78.0%.

Together, these controls, behavioral tests, and interventions suggest that learning to infer omitted computations contributes to SeDo's gains.

AuthorsOpenReview ↗

Response to Reviewer rS3X (3/3)

  1. “The paper ... does not explain the large p@1/p@16 gap on hard Quickselect or the GPT2-RoPE SeDo-R collapse on DEPO.”

    Thank you for this comment. We explain both results below.

    Quickselect. To understand the large p@1/p@16 gap on the hardest range (300≤H<380300 \le H < 380), we inspected the sampled solutions of SeDo with Qwen3-1.7B. These problems require long CoTs with hundreds of steps, so a small error in one step can make the answer off by a few ranks. Indeed, many incorrect samples return an element close to the target rank, while other samples of the same problem give the correct answer. Among the incorrect samples, 63.5% return an element one rank away from the target, and 89.8% are within three ranks. For example, on a problem asking for the 39th-smallest element, greedy decoding stops partitioning one step too early and returns the 38th-smallest element, whereas 10 of 16 samples continue with the remaining candidates and return the correct answer. This helps explain why repeated sampling substantially increases success. A similar gap also appears for Full (Table 2). We have added this analysis to Appendix E.

    DEPO. SeDo-R's low K12 and K16 accuracy on GPT2-RoPE is due to the lower learning efficiency of both the architecture and the method.

    First, GPT2-RoPE learns less efficiently than Canon-Llama. We report the earliest checkpoint from which validation accuracy stays above 80% at every remaining evaluation through 150k updates (evaluated every 5k updates). Each entry shows Canon-Llama / GPT2-RoPE.

MethodK4 Canon-LlamaK4 GPT2-RoPEK8 Canon-LlamaK8 GPT2-RoPE
SeDo10k30k20k105k
SeDo-R20k25k55k150k
MTP15k40k50k105k
iCoT-SI10k15k15k>150k

GPT2-RoPE reaches this threshold later, or does not maintain it through the end of training. As a result, within our 150k-update budget, only SeDo and MTP achieve non-trivial K12 and K16 accuracy on GPT2-RoPE. In particular, SeDo-R reaches 80% at K8 only at the final 150k checkpoint, leaving no updates to learn K12 and K16.

Second, SeDo-R learns long chains less efficiently than SeDo. It needs more updates than SeDo at K8 on both architectures (table above). On Canon-Llama, SeDo-R still has 0% K16 accuracy after 50k updates, while SeDo already reaches 46.09%, although both eventually reach near-perfect K16 accuracy.

We attribute the slower learning of SeDo-R to a mismatch between its explicit and implicit reasoning data. In SeDo, the more steps are removed from a partial CoT, the closer it becomes to an answer-only example. In SeDo-R, however, even a fully masked CoT still contains one mask token per step, unlike answer-only examples, which contain none. To test this, we train and evaluate SeDo-R on GPT2-RoPE with every answer-only example replaced by a fully masked CoT (one mask token per intermediate node), so that its implicit reasoning data match its partial CoTs. SeDo-R learns substantially faster here: its K8 accuracy stays above 80% from 85k updates, compared with 150k for standard SeDo-R (table above). It thus has more updates left to learn K12, reaching 76.56% after 150k updates, compared with 2.34%. We have added these learning curves to Appendix F.3 (Figure 9).

References

  • Belrose et al. (2023). LEACE: Perfect linear concept erasure in closed form.
  • Sinha et al. (2026). The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs.
Reviewer_yAX1OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper explores whether training models to skip reasoning steps can improve generalization to harder problems. The authors propose step dropout (SeDo), which randomly removes reasoning steps from CoT training traces, and a variant that masks the omitted steps. Experiments across multiple models and reasoning tasks show substantial improvements in generalization to harder problems. Further experiments show that SeDo improves the ability to perform multi-step reasoning implicitly, that intermediate values from omitted steps can be recovered from hidden states, and that models trained with SeDo perform better even when given complete CoT prefixes. Experiments with looped transformers also show a connection between the amount of implicit computation and the accuracy of explicit reasoning steps.

Critical strengths

The extensive experimentation across multiple models, tasks, and training settings shows consistent generalization improvements from the step dropout (SeDo) training method. SeDo is also conceptually simple and easily adapted to new datasets with identifiable reasoning steps, making the improvements more surprising.

The additional experiments investigating why SeDo works are interesting and help support the main claims of the paper. In particular, the probing experiments suggest that models learn to implicitly compute omitted intermediate values, while the complete-prefix experiments show that the improvements are not solely due to robustness to missing reasoning steps.

Critical weaknesses

The core proposed training method, step dropout (SeDo), is closely related to previous work such as Dualformer, LiteReason, and LM_skip. These works also explore omitting or replacing reasoning steps during training in ways similar to SeDo, and should either be compared against more directly or the paper should reduce its claims of SeDo's novelty. The primary contribution of this work therefore appears to be its analysis and extensive experimentation rather than the method itself.

The experimental settings focus on very structured reasoning tasks, such as mathematical and algorithmic problems with clearly defined reasoning steps. General application to other reasoning domains is unclear, especially when steps are less clearly sequentially linked or cannot be easily inferred from previous steps.

Questions for authors
  1. Do you expect these results to extend to less structured reasoning tasks, such as narrative or knowledge-based reasoning?

  2. For future work, could SeDo be modified to selectively skip simpler reasoning steps rather than uniformly dropping steps, or in a way that maintains the interpretability of the reasoning process?

4 comments
AuthorsOpenReview ↗

Response to Reviewer yAX1 (1/3)

Thank you for your detailed and helpful review. We have revised the paper, with changes marked in blue.

  1. “The core proposed training method, step dropout (SeDo), is closely related to previous work such as Dualformer, LiteReason, and LM_skip.”; “The primary contribution of this work therefore appears to be its analysis and extensive experimentation rather than the method itself.”

    Thank you for this comment. We agree that SeDo is closely related to these works. However, these works mainly omit or compress reasoning steps for shorter, more efficient reasoning. Our contributions differ in two aspects. First, we study what step dropping teaches models: we show that it teaches models to perform the omitted steps implicitly without producing tokens. Second, we show that this improves CoT reasoning on harder problems that require more steps than those seen in training. Below, we first compare SeDo with each work, and then present evidence for both contributions.

    Comparison with each work. Dualformer. Dualformer (Su et al., 2025) is the most closely related work. However, Dualformer regards dropping trace steps as learning human-like shortcuts for efficient reasoning, and focuses on controllable fast and slow reasoning modes. It does not examine whether models perform the omitted steps internally, or show generalization to problems of longer horizons.

    LiteReason. LiteReason (Gurung et al., 2026) replaces randomly selected CoT sentences with continuous latent tokens. These latent tokens give the model additional positions to perform the replaced steps. In contrast, SeDo removes steps without any replacement, so models must perform the omitted steps within the forward passes of the remaining tokens. SeDo is also simpler to train: since each latent token depends on the previous ones, LiteReason must generate them one by one during training, whereas SeDo trains on text with standard teacher-forcing.

    LM_skip. Liu et al. (2024) iteratively retrain models on their own correct shorter solutions, which often requires warm-start data with manually constructed shorter solutions. Importantly, shorter solutions used in LM_skip do not necessarily require implicit reasoning: for example, they merge two single-digit additions into one, or remove steps that cancel out (e.g., turning right and then left). In contrast, in our tasks, SeDo omits steps whose results are needed later, so models must compute them implicitly to continue reasoning.

    We have clarified these distinctions in the related work (Section 2).

    Evidence for implicit reasoning. On DEPO, models must follow a randomly generated chain of edges without writing out intermediate nodes. Each node determines which edge to follow next, so answering requires computing those nodes internally. We show that SeDo learns this task more efficiently than answer-only training and iCoT-SI (Section 3), suggesting the improved efficiency in learning implicit reasoning.

    On GSM-∞\infty, we further test from three aspects whether SeDo models compute and use omitted values implicitly, with behavioral, probing, and intervention experiments. Specifically, we omit the line that calculates a variable VV, and ask the model to predict a dependent value UU, e.g., complete U = V * 2 =. We use the 500 held-out probing problems from Section 5.1 with Qwen3-1.7B.

    First, SeDo models can predict future steps without the omitted dependencies provided in the CoT context: without VV's defining line, SeDo predicts UU with 86.4% accuracy, compared with 2.6% for Full.

    Second, VV's value is represented in the hidden states: linear probes at the final = decode VV's value much more accurately from SeDo than from Full (Section 5.1, Figure 5).

AuthorsOpenReview ↗

Response to Reviewer yAX1 (2/3)

  1. (continued)

    Third, SeDo models use this representation to predict UU. We erase VV's value at the same position with LEACE (Belrose et al., 2023), jointly in all layers of Qwen3-1.7B, and measure whether the model still predicts UU correctly. As a baseline, we instead remove as many random directions as LEACE does (averaged over five random seeds). We report the accuracy (%) of predicting UU:

CoT prefixModelNo erasureRandom erasureLEACE
With VV's defining lineFull98.2098.12 (−0.08)98.40 (+0.20)
SeDo99.8099.76 (−0.04)99.40 (−0.40)
Without VV's defining lineFull2.602.64 (+0.04)4.20 (+1.60)
SeDo86.4086.84 (+0.44)78.00 (−8.40)

With VV's defining line omitted, LEACE reduces SeDo's accuracy from 86.4% to 78.0%, while random erasure has no effect. In contrast, with the line present, where the model can read VV's value, LEACE has little effect. This suggests that SeDo models use the implicitly computed values to predict subsequent steps. The effect is partial, likely because LEACE does not remove all information about VV: after erasure, probes still decode VV with 16.0% accuracy (48.4% before erasure; 5% by chance). VV may also be represented at earlier positions or nonlinearly. We have added this experiment to Section 5.1 and Appendix H (Table 15).

Evidence for improved CoT reasoning. Across three base models and three tasks, SeDo improves CoT reasoning on problems requiring more steps than those seen in training (Table 2 and Appendix). SeDo also outperforms Token Drop and MFT, which drop or mask tokens instead of whole steps, in nearly every OOD range. We also find improvements in predicting subsequent steps given complete CoT prefixes (Section 5.2).

Moreover, we add a Full-fresh baseline on Qwen3-1.7B, which replaces SeDo's partial CoTs with complete CoTs of new problems. Concretely, Full-fresh is trained on four to five times as many distinct problems as SeDo. We report OOD p@1 / p@16, with OOD ranges merged:

TaskOOD rangeFullFull-freshSeDo
Maze[200, 650)35.96 / 50.8044.68 / 61.8645.53 / 70.74
Quickselect[140, 380)41.75 / 76.7546.42 / 77.4259.67 / 96.83
GSM-∞\infty[26, 36)47.71 / 67.9261.88 / 83.9655.00 / 80.83

SeDo outperforms Full-fresh on Maze and Quickselect (e.g., 96.83% vs. 77.42% p@16 on Quickselect), and remains competitive on GSM-∞\infty. This suggests that SeDo is also a practical method for improving generalization from limited data. We have added this baseline to Table 2 and Section 4.1.

  1. “The experimental settings focus on very structured reasoning tasks”; “General application to other reasoning domains is unclear”

    Thank you for this comment. We agree that our experiments focus on structured tasks. This follows from our scope: we focus on generalization to harder problems that require composing familiar steps over longer horizons, rather than discovering new solution strategies. This capability remains a bottleneck for LLMs, even when they are given the algorithm or plan (Shojaee et al., 2025; Sinha et al., 2026). Structured tasks let us increase the number of required steps without introducing new skills, so we can test whether models generalize to longer compositions. Such control is difficult on real-world benchmarks, where harder problems can also require skills absent from easier problems. Nevertheless, to test applicability beyond structured tasks, we add an experiment on MATH, which contains diverse mathematical problems with free-form solutions.

    MATH. We train Gemma3-1B-PT for 1k steps on levels 1–3 of the MATH training set, then evaluate on levels 1–3 (ID) and 4–5 (OOD) of the official test set. We reserve 500 test problems (100 per level) for validation and report results on the remaining 4,500 problems. To obtain clearly separated steps, we ask Qwen-3.6-27B to rewrite the training solutions into steps while preserving their meaning.

MethodID (Levels 1–3) p@1 / p@16OOD (Levels 4–5) p@1 / p@16
Base (4-shot)2.50 / 23.541.07 / 10.61
Full5.27 / 26.972.22 / 16.42
SeDo6.11 / 33.162.74 / 18.73

SeDo outperforms Full on both ID and OOD problems, under both greedy decoding and sampling. It also achieves higher accuracy than Full at every learning rate we tried ({5e-6, 1e-5, 3e-5, 5e-5}). This suggests that the benefits of SeDo extend to natural mathematical reasoning. When omitted steps are harder to infer, more training data may be needed, as our randomized-Maze experiment illustrates (point 4). We have added the MATH experiment to Appendix J.

AuthorsOpenReview ↗

Response to Reviewer yAX1 (3/3)

  1. “Do you expect these results to extend to less structured reasoning tasks, such as narrative or knowledge-based reasoning?”

    Thank you for this question. We expect SeDo to extend to such tasks. Our MATH results (point 2) already show gains on less structured problems with free-form solutions. Below, we add new results on knowledge-based reasoning, and discuss narrative reasoning.

    (Parametric) knowledge-based reasoning. We test SeDo on the 4-hop task of Yao et al. (2025), where models must answer questions by composing facts stored in their parameters. Concretely, GPT-2 models are trained from scratch on entity profiles (e.g., “Jennifer's instructor is Mary”) and questions (e.g., “Who is Jennifer's instructor?”), and must answer held-out 4-hop questions without intermediate tokens. For SeDo, the CoT lists the intermediate entities, and we use a schedule similar to DEPO (Section 3). Following Yao et al., we vary the number of training questions as multiples of a base budget (×1 to ×20). Yao et al. show that answer-only training needs at least ×20 to learn this task. In contrast, SeDo learns this task with only a quarter of the training questions: with ×5, SeDo reaches 71% accuracy, and with ×10, 99%, whereas answer-only training reaches only 7% and 9%, respectively. The table shows the number of updates (in thousands) to reach each accuracy, for SeDo / answer-only (>20: not reached within 20k updates):

Budget≥10%≥50%≥70%≥90%≥99%
×1>20 / >20>20 / >20>20 / >20>20 / >20>20 / >20
×26.5 / >20>20 / >20>20 / >20>20 / >20>20 / >20
×51 / >205.5 / >2010 / >20>20 / >20>20 / >20
×101 / 11.5 / >202 / >202 / >204 / >20
×201 / 12 / 42 / 62 / 112.5 / >20

We have added these results to Section 3 and Appendix F.4 (Figure 10).

Narrative reasoning. Narrative reasoning also requires connecting information across many events, similar to the many dependent steps in our tasks. We therefore expect SeDo to help as well, and leave this to future work.

  1. “For future work, could SeDo be modified to selectively skip simpler reasoning steps rather than uniformly dropping steps, or in a way that maintains the interpretability of the reasoning process?”

    Thank you for this suggestion. We agree that selectively skipping steps is a promising direction. Our results support this: in our randomized Maze experiment (Figure 4, left), where the next step is not unique, SeDo needs more data to outperform Full. This suggests that omitting more predictable steps could make training more efficient and effective.

    Selective skipping may also improve interpretability. Very long CoTs are known to be hard to understand: if models learn to infer predictable steps implicitly and write out only key steps as landmarks, their CoTs could become shorter and easier to monitor. We have added these directions to the discussion (Section 6).

References

  • Su et al. (2025). Dualformer: Controllable Fast and Slow Thinking by Learning with Randomized Reasoning Traces.
  • Gurung et al. (2026). Lightweight Latent Reasoning for Narrative Tasks.
  • Liu et al. (2024). Can Language Models Learn to Skip Steps?
  • Belrose et al. (2023). LEACE: Perfect linear concept erasure in closed form.
  • Shojaee et al. (2025). The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity.
  • Sinha et al. (2026). The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs.
  • Yao et al. (2025). Language models can learn implicit multi-hop reasoning, but only if they have lots of training data.

I appreciate the authors' detailed response and additional experiments. In particular, the new MATH experiments add credence to the idea that SeDo can generalize to more realistic and diverse reasoning tasks, although broader experimentation is likely needed. The comparisons with prior work are also appreciated, although I still see the main contribution as the experimental findings and analysis rather than the method itself.

Overall, the rebuttal strengthens the paper, and I will maintain my score of 3.

Reviewer_vzncOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Step Dropout (SeDo), which trains language models on a mixture of complete chain-of-thought (CoT) traces and traces with randomly omitted reasoning steps. The goal is to encourage implicit multi-step computation and improve generalization to harder problems while retaining explicit reasoning at inference time.

Experiments on DEPO, Maze, Quickselect, and GSM-∞ show improvements over standard CoT training. The authors further investigate self-generated CoTs, reinforcement learning, complete-prefix continuation, linear probes, and recurrent Transformers.

The results are interesting within the evaluated task families, although the generality and practical significance of the proposed approach remain unclear.

Critical strengths

SeDo requires no architectural modifications and achieves consistent improvements over standard CoT training across multiple models and controlled reasoning tasks. The gains on harder Quickselect problems are particularly notable.

The authors go beyond final-answer accuracy by studying implicit multi-hop reasoning, complete-prefix continuation, intermediate-value probing, and recurrent Transformers. The improvements from complete CoT prefixes are especially interesting, as they suggest benefits beyond robustness to missing steps.

Critical weaknesses
  1. Limited practical relevance and generalization to realistic reasoning tasks.

My main concern is that the evaluation is dominated by highly structured, synthetic algorithmic tasks. Maze and Quickselect are useful for controlled analysis, but both have exact, efficient classical algorithms. Having an LLM simulate these algorithms over hundreds of steps is an interesting diagnostic exercise, yet its practical significance remains unclear.

GSM-∞ provides a mathematical reasoning setting, but it is generated from only three predefined templates, with difficulty primarily controlled by the number of arithmetic operations.

None of these benchmarks convincingly demonstrates that SeDo helps with challenging, open-ended mathematical or scientific reasoning, such as AIME, MATH, or GPQA, where the difficulty involves discovering solution strategies rather than simply executing more steps of a known procedure.

Thus, the paper establishes improved complexity extrapolation within specific algorithmic task families, but provides limited evidence for broader reasoning generalization.

  1. Unclear transfer to realistic CoT self-distillation.

The self-generated CoT experiment is potentially important, but it is evaluated only on Maze and Quickselect, using Qwen3-0.6B. Its applicability to real reasoning-model training remains untested.

In my own preliminary experiments with Qwen3-30B-A3B, self-distillation using shortened, self-generated CoTs followed by SFT caused substantial performance degradation on AIME and GPQA. While this is not a direct comparison with SeDo, it raises concerns about whether removing reasoning steps is beneficial when the original reasoning involves exploration, backtracking, or nontrivial intermediate insights.

Importantly, SeDo's mixture of complete and partial CoTs differs from directly distilling shorter reasoning traces, so these observations do not establish that SeDo would fail. Nevertheless, they motivate testing its applicability beyond deterministic procedural reasoning.

  1. Limited methodological novelty and incomplete mechanistic evidence.

The core strategy of dropping parts of reasoning traces is closely related to Dualformer (ICLR 2025), which already reports improvements from randomized trace dropping on maze-solving and mathematical reasoning tasks.

The additional experiments and interpretation offered here are valuable, but the methodological novelty is relatively modest. The practical generality of the new conclusions therefore becomes especially important.

Questions for authors
  1. Have the authors attempted SeDo on more realistic mathematical or scientific reasoning benchmarks, such as MATH-500, AIME, or GPQA? If not, what evidence or theoretical considerations suggest that the observed benefits should extend beyond structured algorithmic traces? A small-scale result on natural mathematical reasoning, or a clearer limitation of the claimed scope, would substantially strengthen the paper.

  2. The self-generated CoT experiments only cover Maze and Quickselect. How would SeDo identify safely removable steps in realistic mathematical reasoning, where some intermediate insights may be non-recoverable, ambiguous, or essential to discovering the solution? Does the method depend on having deterministic, easily segmentable reasoning trajectories?

  3. Beyond the additional analyses, what is the main methodological or conceptual advance over Dualformer's randomized trace dropping? Can the authors clarify which experiments distinguish improved implicit computation from the benefits of structured trace augmentation? Additionally, reporting comparable supervised-token exposure would help interpret training efficiency.

3 comments
AuthorsOpenReview ↗

Response to Reviewer vznc (1/3)

Thank you for your detailed and helpful review. We have revised the paper, with changes marked in blue.

  1. “the evaluation is dominated by highly structured, synthetic algorithmic tasks”; “have the authors attempted SeDo on more realistic mathematical or scientific reasoning benchmarks, such as MATH-500, AIME, or GPQA”; “A small-scale result on natural mathematical reasoning, or a clearer limitation of the claimed scope, would substantially strengthen the paper.”

    Here we address both points: we first clarify our scope and then present an experiment on MATH.

    Scope. We focus on generalization to harder problems that require composing familiar steps over longer horizons, rather than discovering new solution strategies. This generalization remains a bottleneck for LLMs: even when given the algorithm, LLMs collapse beyond a certain problem complexity (Shojaee et al., 2025); moreover, even when given the plan and the required knowledge, the per-step error rate of LLMs can rise as the task progresses, beyond what a constant per-step error rate would predict (Sinha et al., 2026).

    To study this capability in isolation, we use controlled tasks with well-defined solution steps: Maze and Quickselect, following classic algorithms, and GSM-∞\infty mathematical problems generated from controllable computation graphs. This allows us to increase the required number of steps without introducing new skills, and test whether models generalize beyond the training horizons. This controllability is difficult on real-world benchmarks, where harder problems may require both deeper compositions of familiar skills and new skills. We have stated this scope more explicitly in the introduction.

    MATH. We additionally evaluate SeDo on MATH to test whether its benefits extend to natural mathematical reasoning. We train Gemma3-1B-PT for 1k steps on levels 1–3 of the MATH training set, then evaluate on levels 1–3 (ID) and 4–5 (OOD) of the official test set. We reserve 500 test problems (100 per level) for validation and report results on the remaining 4,500 problems. To obtain clearly separated steps, we ask Qwen-3.6-27B to rewrite the training solutions into steps while preserving their meaning.

MethodID (Levels 1–3) p@1 / p@16OOD (Levels 4–5) p@1 / p@16
Base (4-shot)2.50 / 23.541.07 / 10.61
Full5.27 / 26.972.22 / 16.42
SeDo6.11 / 33.162.74 / 18.73

SeDo outperforms Full on both ID and OOD problems, under both greedy decoding and sampling. It also achieves higher accuracy than Full at every learning rate we tried ({5e-6, 1e-5, 3e-5, 5e-5}). This suggests that the benefits of SeDo extend to natural mathematical reasoning. We have added this experiment to Appendix J.

  1. “unclear transfer to realistic CoT self-distillation”

Thank you for sharing your self-distillation experiment. As you note, SeDo differs from distilling shorter CoTs. Distilling shorter CoTs (similar to Liu et al., 2024) aims to make reasoning more efficient: models learn to produce shorter CoTs, moving part of the reasoning from the output into hidden computation. SeDo instead aims to improve explicit CoT reasoning on harder problems, without shortening CoTs. Concretely, complete CoTs supervise every step, so models can still generate complete CoTs at test time, including exploration and backtracking steps (e.g., in Maze). Meanwhile, partial CoTs train models to perform omitted steps in their hidden computation, which teaches them to tolerate missing previous steps, and to use the existing CoT context better. As a result, SeDo improves CoT reasoning on harder problems.

AuthorsOpenReview ↗

Response to Reviewer vznc (2/3)

  1. “How would SeDo identify safely removable steps in realistic mathematical reasoning, ...? Does the method depend on having deterministic, easily segmentable reasoning trajectories?”

    Thank you for these questions. Indeed, like most implicit reasoning approaches, SeDo's applicability depends on two properties: whether the omitted steps can be inferred from context, and whether the trajectory can be segmented into steps.

    Determinism. On MATH (point 1), we omit each step independently with probability 0.3, without identifying which steps are removable, and SeDo still improves over Full. In our Maze experiment with randomized traces (Figure 4, left), where the next step is not unique, SeDo still improves generalization, but requires more data: it underperforms Full with less data, but outperforms it with >=400k mazes. This is likely because when a non-deterministic step is omitted, the next step depends on which choice was made, so the model must implicitly consider all possible choices to predict it, which is harder to learn and thus requires more data. This suggests a direction for future work: instead of omitting steps uniformly at random, a sampler could only omit steps that are easier to infer, and keep steps such as key insights or exploration in long reasoning traces. We have added this discussion to Section 6.

    Segmentation. For natural-language solutions without clear steps, a solution is to ask another LLM to rewrite them into steps, by providing human examples of stepwise solutions, as in our MATH experiment (point 1).

  2. “Limited methodological novelty”; “can the authors clarify which experiments distinguish improved implicit computation from the benefits of structured trace augmentation?”; “incomplete mechanistic evidence”

    Thank you for raising these points. We first clarify our contribution relative to Dualformer. We then distinguish improved implicit computation from the benefits of regular trace augmentation: we compare SeDo with token-level augmentation baselines and with a baseline trained on additional new problems, and show that SeDo models compute intermediate results that are not written in the CoT, and use them to predict later steps.

    Novelty. We agree that SeDo is closely related to Dualformer (Su et al., 2025). However, our contributions differ in two aspects. First, we study what step dropping teaches models. Dualformer regards dropping as learning human-like shortcuts for efficient reasoning; in contrast, using the DEPO multi-hop reasoning task (Section 3), we clearly show that it teaches models to perform the omitted steps implicitly without producing tokens. Second, we further show that this improves CoT reasoning on harder problems that require more steps than those seen in training, an important capability that remains a bottleneck for LLMs (point 1). We discuss this distinction in Section 2.

    Comparison with augmentation baselines. To test whether SeDo's gains come from trace augmentation alone, in the paper we compare SeDo with two trace augmentation baselines, Token Drop and MFT (Table 2), which perturb the same CoTs by dropping or masking tokens instead of whole steps. SeDo outperforms both in nearly every OOD range across models and datasets. For example, on GSM-∞\infty with 31≤H<3631 \le H < 36, SeDo with Qwen3-1.7B reaches 49.2% / 77.1% (p@1 / p@16), compared to 34.2% / 62.1% for Token Drop and 39.2% / 69.6% for MFT.

    Since data augmentation approximates additional data, we further compare SeDo with an upper bound: a Full-fresh baseline on Qwen3-1.7B, which replaces SeDo's partial CoTs with complete CoTs of new problems. Concretely, Full-fresh is trained on four to five times as many distinct problems as SeDo. We report OOD p@1 / p@16, with OOD ranges merged:

TaskOOD rangeFullFull-freshSeDo
Maze[200, 650)35.96 / 50.8044.68 / 61.8645.53 / 70.74
Quickselect[140, 380)41.75 / 76.7546.42 / 77.4259.67 / 96.83
GSM-∞\infty[26, 36)47.71 / 67.9261.88 / 83.9655.00 / 80.83

SeDo outperforms Full-fresh on Maze and Quickselect (e.g., 96.83% vs. 77.42% p@16 on Quickselect). On these tasks, SeDo's gains thus exceed those from five times as many new problems, suggesting that they do not come from data augmentation alone. On GSM-∞\infty, Full-fresh is higher, but SeDo still achieves about half (p@1) to four fifths (p@16) of Full-fresh's gain over Full. We have added this baseline to Table 2 and Section 4.1.

AuthorsOpenReview ↗

Response to Reviewer vznc (3/3)

  1. (continued)

    Evidence of implicit computation. We test from three aspects whether SeDo models compute and use omitted values implicitly, with behavioral, probing, and intervention experiments on GSM-∞\infty. Specifically, we omit the line that calculates a variable VV, and ask the model to predict a dependent value UU, e.g., complete U = V * 2 =. We use the 500 held-out probing problems from Section 5.1 with Qwen3-1.7B.

    First, SeDo models can predict future steps without the omitted dependencies provided in the CoT context: without VV's defining line, SeDo predicts UU with 86.4% accuracy, compared with 2.6% for Full.

    Second, VV's value is represented in the hidden states: linear probes at the final = decode VV's value much more accurately from SeDo than from Full (Section 5.1, Figure 5).

    Third, SeDo models use this representation to predict UU. We erase VV's value at the same position with LEACE (Belrose et al., 2023), jointly in all layers of Qwen3-1.7B, and measure whether the model still predicts UU correctly. As a baseline, we instead remove as many random directions as LEACE does (averaged over five random seeds). We report the accuracy (%) of predicting UU:

CoT prefixModelNo erasureRandom erasureLEACE
With VV's defining lineFull98.2098.12 (−0.08)98.40 (+0.20)
SeDo99.8099.76 (−0.04)99.40 (−0.40)
Without VV's defining lineFull2.602.64 (+0.04)4.20 (+1.60)
SeDo86.4086.84 (+0.44)78.00 (−8.40)

With VV's defining line omitted, LEACE reduces SeDo's accuracy from 86.4% to 78.0%, while random erasure has no effect. In contrast, with the line present, where the model can read VV's value, LEACE has little effect. This suggests that SeDo models use the implicitly computed values to predict subsequent steps. The effect is partial, likely because LEACE does not remove all information about VV: after erasure, probes still decode VV with 16.0% accuracy (48.4% before erasure; 5% by chance). VV may also be represented at earlier positions or nonlinearly. We have added this experiment to Section 5.1 and Appendix H (Table 15).

Together, these controls, behavioral tests, and interventions suggest that learning to infer omitted computations contributes to SeDo's gains.

  1. “reporting comparable supervised-token exposure would help interpret training efficiency”

    Thank you for this suggestion. Within each task and model, all methods use the same number of optimization steps, batch size, and pack size. The specific stats are shown below (in millions): SeDo uses 0.5–13% fewer supervised tokens than Full, and is similar to Token Drop, yet outperforms both in nearly every OOD range. We have added these numbers to Appendix D (Table 7).

ModelTaskFullToken DropMFTSeDo
Qwen3Maze657620657616
Quickselect1,2591,2631,2591,253
GSM-∞\infty442385442385
Gemma3Maze707659707659
Quickselect1,2711,2561,2711,256
GSM-∞\infty435377435377

References

  • Shojaee et al. (2025). The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity.
  • Sinha et al. (2026). The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs.
  • Liu et al. (2024). Can Language Models Learn to Skip Steps?
  • Su et al. (2025). Dualformer: Controllable Fast and Slow Thinking by Learning with Randomized Reasoning Traces.
  • Belrose et al. (2023). LEACE: Perfect linear concept erasure in closed form.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Changes

We thank the reviewers for their constructive feedback and have revised the paper accordingly (changes marked in blue). Below, we summarize how we address the main concerns.

1. Do SeDo's gains only come from data augmentation? (rS3X, vznc) Three results suggest that they do not come from data augmentation alone. First, SeDo outperforms token-level augmentation (Token Drop and MFT) in nearly every OOD range across all three models (Section 4.1). Second, we introduce a new baseline, Full-fresh, which replaces SeDo's partial CoTs with complete CoTs of new problems, i.e., up to five times as many problems. SeDo is competitive with Full-fresh, and outperforms it on Maze and Quickselect (Table 2). Third, we perform a new intervention experiment to show that SeDo models use the values they compute implicitly: erasing these values with LEACE (Belrose et al., 2023) reduces SeDo's accuracy in predicting a dependent value from 86.4% to 78.0%, while random erasure has no effect (Section 5.1, Appendix H).

2. Why structured tasks, and does SeDo extend beyond them? (vznc, yAX1) Our focus is generalization to harder problems that require composing familiar steps over longer horizons, which remains a bottleneck for LLMs (Shojaee et al., 2025; Sinha et al., 2026). Structured tasks let us increase the number of steps without introducing new skills, which is difficult on real-world benchmarks.

Nevertheless, we have added two experiments to further illustrate SeDo's applicability to other tasks. First, we train Gemma3-1B-PT on levels 1–3 of MATH, and show that SeDo improves CoT reasoning over Full on both easier (levels 1–3) and harder (levels 4–5) problems (Appendix J). We further show that SeDo works when reasoning requires parametric knowledge: on the 4-hop knowledge task of Yao et al. (2025), SeDo learns the task with a quarter of the training questions that answer-only training needs (71% vs. 7% accuracy with 20k questions; Section 3, Appendix F.4).

3. How does SeDo differ from prior work? (yAX1, vznc) Dualformer (Su et al., 2025), LiteReason (Gurung et al., 2026), and LM_skip (Liu et al., 2024) mainly omit or compress reasoning steps for shorter, more efficient reasoning. We instead show that step dropout teaches models to perform the omitted steps implicitly, and that this improves CoT reasoning on harder problems. Moreover, unlike LiteReason, SeDo removes steps without replacing them with latent tokens, and unlike the shorter solutions in LM_skip, the steps that SeDo omits are needed later in our tasks. Models must therefore compute the omitted steps implicitly. We clarify these differences in Section 2.

Besides these three points, we also explain SeDo-R's slower learning on GPT2-RoPE and the p@1/p@16 gap on Quickselect (Appendices F.3 and E), revise our claims about looped Transformers (Section 5.3), and report supervised-token counts (Appendix D).

We thank the reviewers again for their time and effort.

References

  • Belrose et al. (2023). LEACE: Perfect linear concept erasure in closed form.
  • Gurung et al. (2026). Lightweight Latent Reasoning for Narrative Tasks.
  • Liu et al. (2024). Can Language Models Learn to Skip Steps?
  • Shojaee et al. (2025). The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity.
  • Sinha et al. (2026). The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs.
  • Su et al. (2025). Dualformer: Controllable Fast and Slow Thinking by Learning with Randomized Reasoning Traces.
  • Yao et al. (2025). Language models can learn implicit multi-hop reasoning, but only if they have lots of training data.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.