acceptodds
Under review as a conference paper at ICLR 2027

Schrödinger’s Thinking: Reasoning via Superposition in Diffusion Large Language Models

Abstract

Diffusion large language models offer a promising alternative to sequential reasoning by predicting many parts of a solution in parallel. However, we find that their reasoning is still limited by an important form of sequential commitment: each prediction must collapse to a single discrete token, even when the future remains uncertain. We introduce reasoning via superposition, a framework that delays this commitment. Instead of selecting one token immediately, our approach represents an intermediate prediction as a continuous mixture of several plausible tokens. This allows multiple possible reasoning paths to remain active and jointly influence later predictions until the model has enough context to make a discrete decision. We further introduce reinforcement learning that teaches diffusion language models to reason through these superposed states. Across several diffusion language models and reasoning benchmarks, superposition improves reasoning accuracy at inference time without changing model parameters, and reinforcement learning further increases the gains to 7.3 points. Our analysis shows that preserving alternative hypotheses is especially important early in reasoning, when the correct future has not yet been determined.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Diffusion Large Language Models, Continuous Token Representations, LLM Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_G4kmOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses premature token commitment in diffusion language models. It introduces SuperInfer, which replaces discrete token predictions with probability-weighted mixtures of top-K token embeddings, allowing uncertain predictions to influence subsequent denoising steps. The authors further propose SuperRL, a PPO-style framework for learning from superposition-based rollouts. Experiments across four diffusion models and several reasoning benchmarks show improvements over standard decoding, with additional gains from reinforcement learning.

Critical strengths
  1. The proposed inference mechanism is simple and requires no parameter updates. Its applicability to both full-attention and block-diffusion models is a useful practical contribution.

  2. The empirical results are encouraging. In particular, SDAR-4B improves from 62.4 to 67.2 on MATH500 using SuperInfer alone. The parallel decoding and collapse analyses provide suggestive, although not conclusive, evidence for the proposed mechanism.

Critical weaknesses
  1. The experiments do not sufficiently isolate the benefit of superposition from delayed discrete commitment. A compute-matched delayed-unmasking baseline is missing. Similarly, SuperRL is not compared against standard discrete PPO under the same training setup, making its additional contribution difficult to assess.

  2. The SuperRL objective needs further justification. Eq. (3) uses top-1 token probabilities as a surrogate for continuous superposition trajectories, while Eq. (6) appears to omit the importance weighting and clipping terms in Eq. (4). Additionally, the baseline accuracies in Figure 3 differ from those in Table 1, and the full-collapse intervention does not necessarily reproduce standard discrete decoding. These issues limit the interpretability of the reported results.

Questions for authors
  1. How does SuperInfer compare against simply leaving uncertain positions masked for additional denoising steps, under matched inference cost?

  2. Can the authors clarify the trajectory likelihood and gradient computation in Eqs. (3)--(6)? How much does SuperRL improve over conventional discrete PPO using the same training data?

  3. How is the trigger token defined for frozen pretrained models, and what happens if it is never generated?

  4. Could the authors explain the discrepancy between the baselines in Figure 3 and Table 1, particularly the large degradation under full forced collapse?

Reviewer_ytsVOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes Schrodinger Thinking, a simple plug and play framework for improving reasoning in dLLMs by avoiding premature token commitment. The main idea is superposition inference, which replaces a selected discrete token with a weighted average of its top K candidate embeddings. The authors also propose SuperRL to attempt to learn the method, which uses superposition-based rollouts and PPO to further improve reasoning performance. Experiments on LLaDA and SDAR across several math and reasoning benchmarks show improvements from SuperInfer, with additional gains from SuperRL on MATH500.

Critical strengths

Simple and interesting inference idea. I appreciate that SuperInfer can be directly applied to existing dLLMs without changing model parameters. The results on several models including LLaDA and SDAR suggest that using continuous token representations during inference can be useful.

Useful experiments and analysis. The paper provides some interesting experiments on decoding parallelism, early versus late token commitment and different superposition capacities. I like the controlled collapse experiments in Figure 3, which give some insights that delaying token commitment can help reasoning.

Critical weaknesses

The main claim about multiple reasoning paths is not fully convincing. My biggest concern is that the authors claim to preserve multiple reasoning paths, but the actual implemented method simply computes a weighted average of Top K token embeddings. I am not convinced that a single mixture embedding is equivalent to maintaining multiple independent reasoning trajectories. I am strongly concerning that the improvements might just come from smoothing the token representations rather than actually exploring multiple reasoning paths (please see my Q1 below).

Some claims in the paper seem too strong and lack sufficient evidence. For example, the Introduction claims that SuperRL guarantees the model learns to optimise its planning, and that the method scales efficiently with the "density" of the future. I could not find enough evidence to support such strong claims. Also, SuperRL uses Top 1 token likelihood even though the rollouts are generated with Top K mixtures, which makes me question whether the training objective really captures the proposed superposition process (see my Q2).

The novelty regarding delayed commitment is somewhat limited. The idea of avoiding premature token commitment has already been explored in previous dLLM work, such as VSB (arXiv:2604.23994). I think the authors should better explain what is fundamentally new about their approach beyond applying weighted embedding mixtures during diffusion decoding.

Unclear decision boundary and trigger token design and implementation. The method relies on a trigger token to decide when to stop superposition and switch back to discrete decoding. However, I could not find a clear explanation of what this trigger token actually is, how it is chosen, or whether it is learned. This special token seems to be an important part of the method, especially since SuperInfer is training-free.

Limited RL evaluation. SuperRL is only trained on Math and evaluated on two math benchmarks, making its generalisation to other domains unclear. Also, Table 3 shows a significant performance drop when more tokens are decoded per step, even with SuperInfer. Although Figure 4 studies different K values, the best settings vary quite a lot with the decoding configuration. So I suggest that more detailed experiments are needed to support the true effectiveness of the method.

Questions for authors

Q1 Does SuperInfer really preserve multiple reasoning paths? My main concern is that the proposed superposition is simply a weighted average of Top K token embeddings. How does this actually correspond to preserving multiple reasoning trajectories rather than just producing a smoother embedding? Also, the mixture weights are directly computed from the pretrained model's logits without any additional learning. Could the authors explain where the improvements really come from and provide controlled experiments like comparisons with simpler embedding averaging or smoothing methods?

Q2 What exactly is SuperRL learning? Equation 3 calculates the trajectory likelihood using only the Top 1 discrete token, even though the rollout is generated using Top K mixture embeddings. I am not fully convinced why this is a valid likelihood for the actual superposition rollout. Could the authors provide more technical justification for this design? Also, I’m really confused about how does this objective specifically encourage the model to maintain multiple hypotheses rather than just improving the Top 1 prediction?

Q3 How is the decision boundary actually determined? Figure 2 introduces a trigger token to switch from superposition to discrete decoding, but I could not find its actual implementation. What exactly is this trigger token, and how is it selected or learned? Since SuperInfer does not involve training, how does the model know when it should stop superposition or not? I think this is quite important for the method and there is also no code for reproducing this.

Q4 What is the main novelty compared with existing delayed commitment methods? Several previous works have already explored avoiding premature commitment in dLLMs, such as VSB, which already focuses on the problem of when to commit a block. I understand that the proposed method operates at the token level instead of block level, but could the authors better explain its key differences and advantages since most experiments are on dLLMs with block-based generation?

Q5 How robust is the method when increasing decoding parallelism and K? Table 3 shows that SuperInfer still suffers a large performance drop when increasing from 1 to 4 tokens per step. For example, SDAR-4B drops from 67.4 to 20.6 on MATH500. Also, Figure 4 shows that the best K depends quite heavily on the confidence threshold and block length. So my observation is the proposed method may rely heavily on hyperparameter tuning. Given that preserving multiple hypotheses is the main motivation, could the authors provide insights into how K affects performance across different models and tasks? Does increasing the number of candidates consistently help, or does the method require careful tuning?

Reviewer_qBudOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies premature discrete commitment in diffusion language models. The authors propose Superposition Inference: take a selected token and replace it with a probability-weighted mixture of embeddings from its top-KK candidate tokens. This continuous representation stays through the rest of denoising, and decoding switches back to standard discrete mode towards the end.

They also introduce Superposition Reinforcement Learning, or SuperRL, using PPO to train these diffusion language models while working with continuous intermediate states. Tests on LLaDA, MMaDA and SDAR across MATH500, GSM-Plus, GPQA-Diamond and MMLU-Pro show up to 6.7 points gain when Superposition Inference is used at inference time. SuperRL pushes MATH500 and GSM-Plus further, with a peak improvement of 7.3 points over the base model. The authors also run extra ablations around decoding parallelism, sampling temperature, superposition capacity, and the difference between early and late collapse.

Critical strengths
  1. The paper targets a real, relevant issue for diffusion language models: the model locks in a token too early, before later reasoning steps have built up enough context. The fix they propose is fairly simple. At least from an algorithm perspective, it can be applied to both full-attention and block-diffusion models at inference time, without retouching model weights,That’s nice.

  2. The empirical evaluation isn’t just a quick demo on one model. They test across multiple model families and reasoning benchmarks. The analysis on parallel decoding and early vs late collapse gives some supporting evidence for their main intuition.

Critical weaknesses
  1. The core idea is fun, but many implementation details are underspecified, which makes reliable reproduction hard. Section 3.1 mentions a trigger token vtrigv_{\mathrm{trig}}. It’s not clear what this token actually is — whether it’s an existing one like EOS or a new special token, how detection works, and what happens to other superposition states already sitting at other positions once the trigger fires. The text says decoding returns to normal discrete decoding, but I couldn’t fully piece together the state transition at that boundary from the writing or Figure 2. These details also matter if we want to verify the claim that final outputs stay human-readable.

    In addition, Equation (2) outputs one combined weighted embedding instead of keeping KK separate trajectories around. It still works as a soft-token mechanism, but I’m not fully sold on the “multiple reasoning paths” framing. This would land better with comparisons to simpler baselines: expected embedding, full-vocabulary soft decoding, or actual multi-branch decoding.

  2. The empirical trends look promising, but some evaluation numbers didn’t line up for me. Figure 3 plots dashed Base values of 65.4 for SDAR-4B and 54.3 for SDAR-1.7B, while Tables 1 and 3 report different Base numbers under what appears to be a similar setup. The naming also jumps between SDAR-4B / SDAR-1.7B and SDAR-4B-Chat / SDAR-1.7B-Chat. This could easily come from a different checkpoint, decoding schedule, sampling setup, or random seed, but the paper never acknowledges or explains the discrepancy.Since decoding is stochastic, reporting results across multiple seeds or with uncertainty estimates would help interpret small differences. That’s especially true for the early-versus-late collapse experiment. SuperRL results are only shown on two models and two benchmarks. The abstract reads like those larger RL gains apply to all four listed benchmarks, but Table 2 only includes SuperRL for MATH500 and GSM-Plus. The claims around SuperRL need tighter scope.

Questions for authors
  1. Can you give the exact definition and detection rule for vtrigv_{\mathrm{trig}}? Specifically: is it an existing token from the vocabulary or a newly added one? Is detection based on top-1, top-KK, or sampling? And how are existing superposition states converted or represented once the trigger activates?

  2. Can you clarify the setup used for Figure 3? Please state the exact checkpoint, model variant, decoding schedule, temperature, sampling procedure and seed, and explain why the Base numbers differ from Tables 1 and 3.

  3. Under SuperRL, rollout states are updated using the full top-KK continuous mixture, yet Equation (3) computes trajectory likelihood over the top-1 discrete token. This might be a deliberate surrogate objective, but the paper never fully lays out the action-state relationship, nor why this surrogate is suitable for PPO. A matched discrete-PPO or TraceRL baseline, plus ablation over the likelihood formulation, would help separate gains coming from superposition itself versus gains from post-training.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.