acceptodds
Under review as a conference paper at ICLR 2027

The Confidence Shortcut: A Reasoning Failure Mode in Masked Diffusion Models

Abstract

Chain-of-thought reasoning helps autoregressive models solve complex problems by generating intermediate steps that support later predictions. Masked diffusion models (MDMs) offer a similar opportunity through arbitrary-order generation: they can ideally reveal intermediate results along logical dependencies. In practice, however, standard decoding simply prioritizes high-confidence tokens, which need not align with this dependency order. We identify this discrepancy as the confidence shortcut: models commit with high certainty to plausible tokens while neglecting long-range dependencies. In multi-digit addition, models predict higher-order digits without properly tracking carries through long chains. Controlled pretraining across diverse reasoning tasks confirms that confidence-guided ordering often selects suboptimal sequences, and confidence-aligned training schemes can exacerbate these failures—for example, increasing addition error rates by an order of magnitude. Our findings caution against relying solely on confidence to choose generation orders and against training objectives that reinforce this preference.

Primary area:
generative models
Keywords:
Masked Diffusion Models, Generative Models, Reasoning, Decoding Strategies, Shortcut Learning

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_FJnBOpenReview ↗
Rating
4(+2)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the confidence shortcut, showing that confidence sometimes selects answers for which the model has enough information, but sometimes the model may choose a position it does not know how to answer yet and fill it with something that only looks like a plausible answer. It also gives a deep analysis of order selection and shows that random masking has some advantages, because confidence-based selection may be biased, and training around the selected order may make things worse.

Critical strengths
  1. Comprehensive experiments. The paper includes controlled training experiments across several reasoning tasks and an evaluation of LLaDA-8B Base. This helps show that the observed failure is not limited to a single small model or experimental setting.

  2. Showing the value of random-mask training. The results suggest that standard random-mask training can be more robust than confidence-aligned training on difficult reasoning tasks. I think this is a useful message to the community, as reducing training–inference mismatch may not always improve performance when the decoding policy itself is suboptimal.

Critical weaknesses

My main concern is whether the paper provides enough new insight beyond existing work on generation order. Zhang et al. (2026) already study addition, carry dependencies, and different generation orders.

The analysis of premature commitment and the effects of confidence-aligned training adds useful observations, and I find the results supporting random-mask training interesting. For models trained with random masking, why does changing to a task-specific reference order help on addition and Maze, but not consistently on ListOps and Countdown? I think a clearer analysis of when order selection is the bottleneck would help connect these experiments.

I would be interested in seeing whether mixing standard random-mask training into PUMA could recover performance. Would a small proportion of random-mask training be enough, and would the improvement appear under confidence decoding or only under the reference order? This could help clarify the value of broader training-context coverage.

Reference

Zhang, S., Yu, L., Brekelmans, R., Tang, L., Asif, S., and Ver Steeg, G. (2026). Generation Order and Parallel Decoding in Masked Diffusion Models: An Information-Theoretic Perspective. arXiv:2602.00286.

Questions for authors
  1. Section 4 is hard to follow. I think Section 4 could be organized by the findings rather than by dataset. For example, the authors could group the experiments into:
    • Cases where changing the decoding order recovers performance.
    • Cases where a training gap remains even after changing the order.
    • Cases where confidence-aligned training helps. A summary table would make it easier to understand what each experiment contributes to the main claims.

2.Additional remasking comparison. I think the analysis of remasking could go deeper. Although the paper evaluates ReMDM and PRISM, it would be useful to also include WINO (Hong et al., 2025), which uses bidirectional context to identify and remask suspicious tokens. This comparison could help clarify whether the observed failures persist across different error-detection and remasking strategies.

Reference Hong, F., et al. (2025). Wide-In, Narrow-Out: Revokable Decoding for Efficient and Effective DLLMs. arXiv:2507.18578.

3 comments
AuthorsOpenReview ↗

Response to Reviewer FJnB

Thank you for the constructive suggestions on related work and additional experiments, which shaped the new Section 3 and Table 5.

W1. Novelty beyond Zhang et al.

We identify a regime where confidence's advantage reverses, then show how aligned training amplifies the failure.

Zhang et al. [1]: On addition with ≤10-digit operands, confidence outperforms R2L under parallel decoding with pretrained LLaDA-8B-Instruct. Their difficulty groups vary operand length; the edge-case score combines overflow cascades (9,999,999,999+1) with cases such as A+0. They train AR arithmetic models but compare MDM decoders with frozen LLaDA weights.

Our controlled comparison: Even under single-token decoding, fixing 32-digit operands and varying carry span—including chains ending at internal digits—reveals a systematic confidence shortcut. While chain digits remain masked, the model substitutes the chain head's local outgoing carry for its unresolved incoming carry, causing ±1 errors when they differ (§4.3). Matched Random/PAPL/PUMA training shows that alignment amplifies this bias. LSD-first recovers Random/PUMA accuracy and improves frozen LLaDA-8B (§4.4; Table 5).

Difficult reasoning is precisely where order should help. CoT and test-time computation are valuable because intermediate results make hard predictions tractable. Our controlled sweep asks whether confidence organizes the computation that the same model can perform under a dependency-following order (§3).

[1] Zhang et al., Generation Order and Parallel Decoding in Masked Diffusion Models: An Information-Theoretic Perspective, 2026.

W2/Q1. Connecting the cross-task findings

Table 5 now organizes the tasks by what order and training change:

  • Ordering bottleneck: LSD-first/DEF recover Addition and long-corridor Maze accuracy on the same weights.
  • Cost of alignment: PAPL Addition, deeper ListOps, and PUMA Countdown remain below Random under confidence and reference decoding.
  • Effective alignment: Sudoku's confident placements constrain later choices, and PUMA improves performance.

Layered ListOps and sequential Countdown expose natural intermediate steps, while leaving scheduling or arithmetic choices unresolved. Their optimality is unnecessary for comparing training schemes under the same decoder. The results favor preserving broad training contexts and improving how inference uses them.

W3. Mixing random masking with PUMA

Mixing is sensitive to its ratio, and switching objectives does not readily reverse PUMA's learned behavior. Following your suggestion, we added static mixing (300k updates) and continuation (+100k).

Random–PUMA mixing on Addition: confidence / LSD-first exact match, by minimum carry length (Table 16).

TrainingStandard≥4≥12≥28
Random1.000 / 1.0000.998 / 1.0000.996 / 1.0000.992 / 1.000
PUMA0.999 / 1.0000.868 / 1.0000.864 / 1.0000.908 / 1.000
25% random-mask mix0.009 / 0.0090.006 / 0.0100.006 / 0.0120.006 / 0.114
50% random-mask mix0.999 / 0.9990.998 / 0.9980.998 / 0.9980.996 / 1.000

Effect of 100k continuation updates on long-carry Addition (≥28; Table 17).

ContinuationConfidence exact match: before → after
PUMA→Random0.908→0.906
PUMA→PUMA0.908→0.884
Random→PUMA0.992→0.968
Random→Random0.992→0.994

The 25% mixture fails to converge within the budget, while 50% recovers Random-level accuracy with the overhead of maintaining PUMA trajectories. After PUMA training, 100k random-mask updates give no recovery under confidence; the reverse switch weakens Random. LSD-first stays at 1.000 throughout. This suggests that confidence-aligned training can settle into behavior that is hard to change later, favoring random masking from the start.

Q2. WINO remasking

WINO-single gives up parallel drafting and retains verification to recover Random/PUMA accuracy. We added WINO and its single-token variant alongside PRISM and exact LOO (Appendix E.3).

Remasking and correction on Addition: exact match at minimum carry length 28 (Table 15).

DecoderRandomPAPLPUMA
Confidence0.9910.0270.901
PRISM0.9930.0280.951
WINO0.9600.0340.984
WINO-single1.0000.0321.000
Exact LOO (after completion)1.0000.0310.999

PRISM/WINO partially repair PUMA; single-token drafting brings WINO to the accuracy of exhaustive LOO correction, while retaining verification. Exact LOO itself uses 33 queries per round. PAPL remains below 0.04 across these strategies: stronger verification does not remove its conditional-prediction deficit. Maze errors likewise resist revision because neighboring wrong labels can support one another, whereas DEF builds the solution through successive eliminations.

Thank you for the detailed response and the new experiments. I think this paper provides new insight into what random-mask training contributes. Random masking is usually viewed as a source of train–inference mismatch, and many recent papers try to close this gap with confidence-aligned training or self-distillation. This paper shows that the broad context coverage of random masking is valuable in itself, and that removing the mismatch can hurt.

My remaining concern is the presentation and flow of the paper. I encourage the authors to keep improving it in the final version. Given the new insights, I am raising my score to 3.

Thank you for recognizing our contribution and for your positive reassessment. We appreciate your emphasis on our key finding: broad context coverage during training is valuable in itself, and reducing train–inference mismatch can hurt reasoning performance. We will further improve the presentation and flow in the final version to make this message and how it connects our results across tasks even clearer.

Reviewer_EUTzOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper identifies the problem of "confidence shortcut" in MDMs, where models prioritize high-confidence tokens while neglecting long-range dependencies. The authors test this assumption on tasks including Addition, Maze, ListOps, Countdown and Sudoku, and conclude that confidence favors premature shortcuts that overlook reasoning dependencies, and confidence-aligned training can reinforce these shortcuts.

Critical strengths

This paper identifies an interesting problem in MDMs and raises important questions about the relationship between decoding order and reasoning capabilities. Also, it is thought-provoking, encouraging readers to reconsider the current confidence-aligned training approaches.

The experimental design is thoughtful and creative. The authors carefully select tasks such as Addition and Maze, where logical dependencies are closely related to decoding order, making the experiments particularly insightful.

Critical weaknesses

My main concern is whether confidence shortcut is really a problem with decoding order. Imagine asking an autoregressive LLM, for example, GPT, to do the multi-digit addition task. They adopt a less suitable generation order (from left to right, instead of from right to left), but I assume they could get most of the answers correct (or not? you may train a similar AR model to test this). Both AR models and MDMs can generate in a "wrong order" according to your definition, and both can see the whole input expression through their attention mechanism. Why might AR models handle tasks while MDMs cannot? More experiments may be needed to show that it is indeed a matter of decoding instead of architecture or other reasons.

Secondly, in Section 4, the authors aim to test two assumptions “whether confidence decoding follows the dependencies of other reasoning tasks“ and ”whether confidence-aligned training worsens failures to follow this reasoning order”, but some of the arguments in the subsections are a little bit confusing to me. I will elaborate on them in the question part and hope the authors can clarify them.

Questions for authors

Q1: How can you tell whether the confidence shortcut is primarily a decoding problem rather than a limitation of the model architecture or capability? As a comparison, you may train an AR model with a similarly sized transformer and test it on multi-digit addition.

Q2.1: In Section 4, if I understand correctly, one of the argument authors aim to demonstrate is that, confidence decoding is not always the best decoding way. That being said, it makes sense for the authors to manually design reasonable decoding orders for different tasks to demonstrate their better performance. I can see this in the results of the Addition and Maze experiment. But in ListOps, the manually designed layered strategy doesn't significantly improve accuracy for the random baseline; in Countdown, step-sequential also doesn't improve. Then why would one expect them to improve PAPL and PUMA? If the advantage of MDM is a more flexible decoding order, then what evidence supports treating these handcrafted orders as better approximations to the tasks' reasoning dependencies?

Q2.2: Another concern is the consistency of the argument. The authors conclude that confidence decoding could be bad, and aligned training may reinforce this. But in Sudoku, we see the opposite - confidence decoding performs better, and PUMA performs even better. If in some cases confidence is good, but harmful in others, then how could we distinguish them?

Q3: Echoing to my main concern, I wonder whether the paper's conclusion generalizes to diffusion LLMs. The authors have tested LLaDA on the addition task. I'm curious about the results on other tasks.

1 comment
AuthorsOpenReview ↗

Response to Reviewer EUTz

Thank you for the thoughtful questions on model capability, decoding, and generalization. They helped shape the new Section 3 and Table 5.

W1/Q1. Why does decoding matter for an imperfect model?

Intermediate results can turn a difficult prediction into simpler successive calculations. Seeing all operands does not make a long carry easy to compute in one prediction. LSD-first makes the next carry locally available, much as CoT helps an AR model build on earlier reasoning.

Exact conditionals make sequential sampling order-invariant. Larger models solve more problems directly, while harder tasks still benefit from intermediate computation. For a given imperfect model, finding an order that uses those intermediates is therefore test-time computation (§3). Our decisive control changes only that order: the same MDM weights fail under confidence and recover under LSD-first.

Arithmetic remains challenging even for pretrained AR LLMs: Baeumel et al. [1] report systematic carry errors in left-to-right addition, linked to limited lookahead.

For a matched comparison in our setting, we trained the requested AR Transformer (2 layers, 0.35M) on the same data and update budget, using MSD-first notation without a scratchpad.

Addition exact match by minimum carry length. MDM: confidence / LSD-first; AR: left to right.

Model / trainingStandard≥4≥12≥20≥24≥28
AR0.9380.3760.0200.0000.0000.000
MDM / Random1.000 / 1.0000.998 / 1.0000.996 / 1.0000.984 / 1.0000.994 / 1.0000.992 / 1.000
MDM / PUMA1.000 / 1.0000.868 / 1.0000.864 / 1.0000.826 / 1.0000.868 / 1.0000.908 / 1.000

The AR model also struggles on long carries, while the same MDM succeeds once its order supplies useful intermediate results.

[1] Baeumel et al., The Lookahead Limitation: Why Multi-Operand Addition is Hard for LLMs, 2025.

Q2.1. What do the reference orders establish?

Addition and Maze offer explicit dependency-following procedures; the other reference schedules are reasonable strategies for organizing computation.

  • Addition / Maze: LSD-first follows carry propagation, and DEF follows constructive dead-end elimination. Both recover accuracy on unchanged weights.
  • ListOps / Countdown: Layered respects child→parent dependencies, while step-sequential organizes equations and leaves compatible operands and operations to the model. These schedules need not be closer than confidence to the best order for a learned predictor.

Matching a reference therefore does not establish confidence's optimality. The training comparison remains decisive: aligned models fall below Random at ListOps depths 5–6 and for PUMA Countdown overall under both decoders. Alignment can make even confidence decoding worse; this training cost remains meaningful whether or not the reference is optimal (Table 5).

Q2.2. Why does Sudoku differ?

Confidence helps when its preferred predictions support subsequent reasoning. A correct Sudoku cell restricts candidates in its row, column, and box; confidence can adapt as constraints accumulate, and PUMA improves that policy. An Addition chain head, however, can be highly confident before its carry is resolved. Varying dependency difficulty, inspecting commitment timing, and comparing orders on the same weights expose this distinction (§3, Table 5).

Q3. Why evaluate pretrained LLaDA on Addition?

We prioritized Addition as a direct LLM-scale test of the mechanism identified in our controlled experiments. In preliminary trials, LLaDA could not perform our Countdown and ListOps tasks, so extending these evaluations would require task-specific fine-tuning. This entails additional LLM-scale training and validation, while our main focus is understanding the interaction between training and decoding in controlled settings.

We share the motivation of concurrent work by Yeom et al. [2]: the authors find that diffusion LLMs often commit answer tokens before their supporting reasoning tokens, a behavior termed “answer-first.” Their observations on GSM8K/MATH-500 and controlled answer-delay experiments show that commitment order can affect reasoning accuracy. We focus on small, controlled models with known dependencies to trace individual decisions and examine how training changes them (§3).

Our LLaDA-8B result connects this analysis to diffusion LLMs: changing only the order raises long-carry accuracy 183/200→200/200, with high-confidence errors at unresolved chain heads (§4.4, Appendix C.1). The controlled tasks explain how confidence bypasses useful computation and how alignment can amplify the behavior. This motivates broad training contexts and better decoding for difficult reasoning.

[2] Yeom et al., Answer First, Reason Later: When Commitment Order Costs Accuracy in Diffusion Language Models, 2026.

Reviewer_B4kqOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces "the confidence shortcut" phenomenon, that normal decoding in masked diffusion models by confidence ranking commits to plausible tokens before their dependencies are revealed. And confidence-aligned training (PAPL, PUMA) amplifies this failure. These evidence come mainly from addition with long carry chains, where LSD-first decoding outperforms confidence decoding with the same model. Analysis is extended to four tasks including Maze, ListOps, Countdown and Sudoku.

Critical strengths
  1. The error analysis is precise. Failures are $ \pm 1$ errors at chain heads, and all 751 early PUMA commitments follow the shortcut rule.
  2. The failure is reproduced in LLaDA-8B Base.
  3. The diagnostic setup on addition is clean and separates decoding order from model capability by evaluating the same trained model under confidence and LSD-first decoding.
Critical weaknesses
  1. These papers already shows that generation order matters, and that LSD-first helps arithmetic (Lee et al., 2024; Kim et al., 2025). The reference orders are oracle and task-specific. There is no comparison with learned order-selection methods like P2 and SAS.
  2. Cross-task results do not consistently support the claims. Confidence decoding is not consistently worse than the reference orders: it matches or beats them on ListOps, Countdown, Sudoku, and Maze corridor strata 20–30.
  3. Confidence-aligned training is not consistently harmful.
  • PAPL ≈ Random on Countdown
  • PUMA clearly improves Sudoku,
  • PAPL + DEF matches or exceeds Random + DEF on most Maze strata.
  1. The gaps appear almost only in constructed stress strata (e.g., carry chains ≥28, which essentially never occur under uniform operands), while ordinary additions are near 1.0. No commonly used reasoning benchmark in MDMs (e.g., GSM8K, MATH) is evaluated.
  2. The confidence-shortcut mechanism is shown only on addition. For the other tasks, only accuracy is reported.
Questions for authors
  1. Does the confidence shortcut appear on reasoning benchmarks such as GSM8K or MATH with LLaDA or Dream?
  2. Does PAPL or PUMA training at LLM scale degrade such benchmarks?
  3. Can learned order-selection methods recover LSD-first or DEF-like orders on these tasks?
  4. What's the result of PUMA’s Countdown on single-solution puzzles?
3 comments
AuthorsOpenReview ↗

Response to Reviewer B4kq

Thank you for the constructive questions on learned orders, cross-task behavior, and LLM-scale reasoning.

W1/Q3. Can decoding discover useful orders?

SAS learns near-LSD-first orders when its training exposes long dependencies. MDMs should find useful orders across formats, without relying on AR-style string reversal [1]. Kim et al. [2] study confidence/margin decoding itself. A good decoder should compete with RL-learned policies. Following your suggestion, we added P2 (η=1) and RL-trained SAS on frozen denoisers (Table 13).

Addition decoding: exact match and LSD-first rank correlation ρ (Table 13).

Denoiser trainingDecoder (SAS training data)Carry ≥4Carry ≥12Carry ≥28ρ
RandomConfidence0.9950.9950.990+0.17
RandomP21.0001.0001.000−0.07
RandomSAS (standard)0.9350.7300.710−0.45
RandomSAS (≥12)1.0001.0001.000+0.96
PAPL (α=1)Confidence0.9050.6200.025+0.04
PAPL (α=1)P20.9100.6000.025−0.11
PAPL (α=1)SAS (standard)0.9000.5550.010+0.54
PAPL (α=1)SAS (≥12)0.9300.6650.025+0.97
PUMAConfidence0.8700.8500.905+0.11
PUMAP21.0000.9951.000−0.02
PUMASAS (standard)0.9450.7300.800+0.09
PUMASAS (≥12)0.9901.0001.000+0.96
  • Order discovery: Training SAS on carries ≥12 yields ρ≈0.96–0.97; ordinary-addition training transfers poorly. Knowing the carry structure lets us select informative hard examples, an advantage that is harder to obtain on less structured tasks.
  • Recovery depends on the predictor: P2 recovers accuracy through revision despite low LSD-first correlation. SAS recovers Random/PUMA, but PAPL remains inaccurate at ρ=0.97: a good order needs useful conditional predictions.

[1] Lee et al., Teaching Arithmetic to Small Transformers, 2024.

[2] Kim et al., Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions, 2025.

W2/W3. Why do order and training effects vary?

Alignment helps when confidence supports reliable intermediate steps, but can reinforce shortcuts or restrict useful contexts. PUMA benefits Sudoku's constraint propagation but struggles with Countdown's chain construction; PAPL retains random masks and fares well on Countdown and Maze+DEF. The effects are highly task-dependent, and switching to a reference order cannot always undo the training-induced distortions. This supports random-mask pretraining as a robust foundation while improving decoding order (Table 5).

W4. Why focus on difficult strata?

Difficult reasoning is where order should matter most. Ordinary additions let a local rule score near-perfectly. Varying carry span at fixed length tests whether generation supplies useful intermediate computation when that shortcut fails—the motivation behind CoT and test-time computation (§3). Effects also appear at carries ≥4, in-range ListOps depths 4–5, and overall Countdown.

W5. What evidence extends beyond Addition?

The additional tasks reveal how order interacts with different reasoning structures. Addition/Maze have explicit carry/elimination procedures; ListOps/Countdown use reasonable schedules that need not outperform confidence for a learned predictor. Together with Sudoku, these tasks distinguish recoverable order gaps, persistent training deficits, and beneficial alignment (Table 5). Maze produces locally consistent wrong paths (Figure 3); Countdown can complete a chain once compatible choices are partly supplied (Table 4b).

Q1/Q2. What does this suggest at LLM scale?

Concurrent work by Yeom et al. [3] reported diffusion LLMs often commit answer tokens before their reasoning tokens. Their answer-first observations on GSM8K/MATH-500 and answer-delay results highlight the same issue (§3). Known dependencies let us trace decisions and training effects in small models; LLaDA-8B reproduces the addition failure and LSD-first recovery (§4.4).

We expect the same alignment risk on difficult LLM reasoning. PAPL/PUMA share the ideal target of exact conditionals, where order is immaterial. In practice, emphasizing confidence-preferred predictions or states can reinforce the shortcuts seen in our matched-training experiments. LLM-scale training is the next test.

[3] Yeom et al., Answer First, Reason Later: When Commitment Order Costs Accuracy in Diffusion Language Models, 2026.

Q4. Countdown with one recorded solution

PUMA's advantage in the pooled m=1–3 group (Table 4a) does not extend to m=1.

Countdown m=1 under confidence decoding (1,303 puzzles/seed; 3 seeds, mean ± std; Table 10).

TrainingValid solution rate
Random0.138±0.014
PAPL0.122±0.013
PUMA0.102±0.004

Higher solution multiplicity mm offers many valid chains, yet PUMA struggles to construct one. At m=1, the most constrained subset, it must discover a narrow successful route. Both require constructing a chain, unlike Sudoku's local constraint propagation.

Thanks for author's response and the new experiments. Some of my questions are answered, but some concerns remain unresolved:

  • Q1/Q2: There is still no evidence on common reasoning benchmarks (e.g., GSM8K, MATH), and none on PAPL/PUMA training at LLM scale. The LLM-scale evidence remains limited to addition with LLaDA-8B.
  • W2/W3: Table 5 explains the cross-task inconsistencies after the fact, but gives no criterion for predicting when confidence ordering helps or hurts. The abstract still says confidence ordering "often selects suboptimal sequences", yet an order gap appears only on Addition and long-corridor Maze.
  • W4: With standard random-mask training, confidence decoding loses only about 1% even at carry ≥28 (0.991 vs. 1.000). Outside confidence-aligned training, the confidence shortcut seems to have limited practical impact.

I will maintain my current score.

We appreciate your careful consideration of our responses and additional experiments. We recognize that important questions remain open and would like to clarify why we believe our controlled analysis nevertheless provides a meaningful contribution.

Q1/Q2. Scope and motivation

We agree that testing the confidence shortcut on LLM-scale benchmarks like GSM8K/MATH, or evaluating aligned training directly at scale remains an open direction. Still, large-scale dLLM reasoning is what motivated our study in the first place. We turned to small controlled tasks because they make the underlying dependencies explicit at every step, allowing us to trace directly where confidence decoding and aligned training may become fragile.

W2/W3. Table 5 and cross-task regimes

We also agree that Table 5 does not provide an a priori predictive criterion. Instead, we read it as suggesting that broad context coverage in random-mask training can be valuable in itself, while closing the train–inference mismatch may hurt reasoning. Broad coverage preserves frozen-weight capability where order gaps appear (Addition, long-corridor Maze), while confidence-aligned training lowers accuracy on four of five tasks even without a reference-order gap (ListOps, Countdown). Sudoku marks when certainty aligns with local constraints. We will refine the final version to deliver our message about these inference and training-time regimes more clearly.

W4. Baseline accuracy gap and practical relevance

On scratch Addition, the ~1% gap is indeed small in raw accuracy. At the same time, we find its regularity noteworthy: even on an elementary rule the model otherwise solves reliably, errors follow a repeatable carry-chain pattern rather than random noise. One way to view this is as a subtle blind spot persisting near saturation. That perspective may help explain larger drops in less saturated tasks and pretrained dLLMs, and suggest caution given growing interest in confidence-aligned training and self-distillation.

Thank you again for your thoughtful feedback. This discussion helps us state our contribution and its scope more clearly, and we will reflect these points in the final version.

Reviewer_fDpNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that top-probability ("confidence") decoding in masked diffusion models (MDMs) can select generation orders that ignore a task's logical dependencies: the model commits confidently to a locally plausible token whose value depends on still-masked context (the "confidence shortcut"). It further argues that confidence-aligned training (PAPL, PUMA) reinforces this preference. The core evidence is controlled 32-digit addition stratified by carry-chain length (Sec. 3), where one trained model is decoded under confidence and under an LSD-first reference order.

Critical strengths
  • Fixing operand/answer length while stratifying by carry-chain length, and decoding the same model under two orders, isolates order selection from model quality.
  • A careful stress test of confidence-aligned training. PUMA raises long-chain error about 11× (9.2% vs 0.8%; Table 1) and already fails 13% of chains ≥4, which do occur in training, versus 0.2% for Random. On the Maze tail (≥60), PUMA/PAPL drop to 0.675/0.560 vs 0.930 for Random (Table 2). The appendix rules out the obvious explanations for PUMA's addition gap: a 26× larger model is worse (0.730; Table 8), remasking and PRISM reach only 0.909–0.951 (Table 9), and pass@128 (98.3%) stays below Random's pass@1 (Fig. 4). Since these training schemes are recent and motivated by training–inference alignment, a documented failure mode is valuable.
Critical weaknesses
  • The headline claim is broader than the evidence, and the only comparisons are against oracle orders. The abstract states that confidence ordering "often selects suboptimal sequences" across diverse tasks, but an order gap is shown only on Addition and the Maze tail.
  • The case against confidence-aligned training rests on unvalidated reimplementations and thin statistics. PAPL at α=5 scores 0.000 even on ordinary additions (Table 1, App. B.2). With per-position loss multipliers bounded in [1, 1+α], a collapse on a task Random solves perfectly points to an optimization or implementation problem rather than a confidence shortcut; yet α=5 is used for every Sec. 4 task without a per-task sweep. At α=1, PAPL fails equally under LSD-first (0.026 vs 0.030), so that failure is not order-related either.
  • Seeds are reported only for Maze and the base Addition models (3 each); the remaining runs appear to be single-seed, and no table reports standard deviations or intervals.
Questions for authors
  1. Why does PAPL with α=5 collapse on ordinary additions, and why was α=5 then used for all Sec. 4 tasks? A per-task α sweep, plus one reproduction of a result from the original PAPL and PUMA papers, would address W2.
  2. How many seeds back each entry outside Maze and base Addition? Please add standard deviations or binomial intervals to Tables 1–4 and 8.
  3. Are the Maze strata nested (≥30 ⊇ ≥60)? If so, why does DEF score 0.943–0.950 for Random on ≥20–30 but 1.000 on ≥50 and ≥60, and what do the DEF failures look like?
2 comments
AuthorsOpenReview ↗

Response to Reviewer fDpN - part 1/2

Thank you for the careful methodological feedback. We added parameter checks and multi-seed results to address these concerns.

W1. Claim scope and comparisons without reference orders

Confidence can miss useful reasoning orders, and aligning training with it can amplify the failures. We test two complementary questions:

  • Changing order, same weights: Addition and long-corridor Maze recover under dependency-following schedules.
  • Changing training, matched conditions: Even where a reasonable reference matches confidence, aligned models can fall below Random under both decoders, as in deeper ListOps and PUMA Countdown.

Table 5 connects these findings with Sudoku's positive result.

Beyond supplied reference orders: A carry-frontier probe trained on dependency labels improves PUMA 141/180→180/180 and LLaDA-8B 165/180→179/180 without test-time answers (Table 12). P2 and SAS (with long-carry training) likewise recover Random/PUMA addition accuracy with frozen denoisers (Appendix D.2).

W2/Q1. PAPL weighting and PUMA settings

PAPL weighting: long-carry failures remain at a clearly converged setting. Our sweep includes PAPL's recommended starting value α=1 and stronger weights such as α=5 tested in the original paper. We use α=1 for Addition: ordinary accuracy reaches 99.9%, yet long-carry errors remain even under LSD-first. Stronger weighting destabilizes Addition, with collapse at α=5. The other tasks use α=5, where training remains effective; smaller-α checks preserve the main cross-task findings (Appendix B.2).

PAPL weighting on Addition: confidence / LSD-first exact match, by minimum carry length (Table 7).

TrainingStandard≥4≥12≥20≥28
Random (α=0)1.000 / 1.0000.997 / 1.0000.993 / 1.0000.985 / 1.0000.991 / 1.000
PAPL α=10.999 / 0.9990.901 / 0.9320.594 / 0.6810.432 / 0.5630.024 / 0.033
PAPL α=20.391 / 1.0000.266 / 0.7860.162 / 0.3200.090 / 0.1520.000 / 0.000
PAPL α=30.000 / 0.0000.000 / 0.0000.000 / 0.0000.004 / 0.0120.004 / 0.016
PAPL α=40.000 / 0.0000.000 / 0.0000.000 / 0.0040.010 / 0.0020.074 / 0.004
PAPL α=50.000 / 0.0010.001 / 0.0010.000 / 0.0010.001 / 0.0000.001 / 0.006

PAPL weighting on Maze: confidence / dead-end filling (DEF) exact match, by minimum corridor length (Tables 2, 8).

Training≥4≥20≥50≥60
Random0.991 / 0.9880.986 / 0.9350.913 / 0.9980.911 / 0.997
PAPL α=10.893 / 0.9800.897 / 0.9970.680 / 1.0000.690 / 0.993
PAPL α=30.890 / 0.9900.883 / 0.9970.670 / 0.9800.590 / 0.990
PAPL α=50.916 / 0.9930.896 / 0.9610.716 / 0.9890.569 / 1.000

PAPL weighting sensitivity across tasks under confidence decoding (Tables 3, 4, 9).

TaskMetricRandomPAPL α=1PAPL α=5
ListOpsFull-trace exact match: depth 5 / 60.995 / 0.3460.745 / 0.0530.458 / 0.211
CountdownValid solution rate: overall / m∈[1,3]m \in [1,3]0.511 / 0.1740.540 / 0.1820.502 / 0.171
SudokuBlank-cell accuracy: overall / top 1%0.629 / 0.3580.620 / 0.3400.632 / 0.374

PUMA settings: Sudoku improves with the original paper's fixed-K=8 schedule as well as our progressive schedule. K counts reveal stages; our main Sudoku schedule increases K from 8→40 (Table 6; Appendix B.3). Fixed K=8 also improves over its Random baseline, while the progressive schedule gives a larger gain.

PUMA schedule sensitivity on Sudoku: Random / PUMA accuracy under confidence decoding (top 1% = hardest puzzles by rating).

ScheduleOverall blank-cellOverall exact matchTop-1% blank-cellTop-1% exact match
K=8→400.629 / 0.7530.398 / 0.5820.357 / 0.6090.028 / 0.326
Fixed K=80.610 / 0.6290.380 / 0.4040.336 / 0.3680.008 / 0.026
AuthorsOpenReview ↗

Response to Reviewer fDpN - part 2/2

W3/Q2. Seed counts and uncertainty

Tables now report mean ± standard deviation: 5 seeds for Addition, Maze, and capacity; 3 for ListOps, Countdown, and Sudoku. The initial submission used 3 seeds for base Addition/Maze and 1 elsewhere. We trained additional independent seeds across every benchmark for statistical significance.

Addition exact match (5 seeds; PAPL α=1). Conf. = confidence; LSD = LSD-first.

Min. chainRandom Conf.Random LSDPAPL Conf.PAPL LSDPUMA Conf.PUMA LSD
Standard1.000±0.0001.000±0.0000.999±0.0010.999±0.0010.999±0.0011.000±0.000
≥40.997±0.0021.000±0.0000.901±0.0230.932±0.0390.869±0.0461.000±0.000
≥280.991±0.0051.000±0.0010.024±0.0140.033±0.0110.884±0.0360.999±0.002

Maze exact match (5 seeds). DEF = dead-end filling.

Min. corridorRandom Conf.Random DEFPAPL Conf.PAPL DEFPUMA Conf.PUMA DEF
≥200.986±0.0070.935±0.0210.896±0.0150.961±0.0170.809±0.0410.901±0.022
≥300.985±0.0080.943±0.0250.898±0.0190.992±0.0070.851±0.0340.949±0.019
≥600.911±0.0380.997±0.0040.569±0.0491.000±0.0010.654±0.0280.986±0.009

ListOps full-trace exact match (3 seeds). Layered = bottom-up; depth 6 is outside training.

DepthRandom Conf.Random LayeredPAPL Conf.PAPL LayeredPUMA Conf.PUMA Layered
50.995±0.0040.994±0.0020.458±0.0190.479±0.0070.843±0.0190.795±0.017
60.346±0.0240.321±0.0140.211±0.0270.185±0.0230.215±0.0150.191±0.011

Countdown valid solution rate from the input alone (3 seeds).

DecoderRandomPAPLPUMA
Confidence0.511±0.0130.502±0.0060.315±0.011
Random order0.493±0.0170.472±0.0090.284±0.018
Step-sequential0.472±0.0110.462±0.0140.315±0.013

Sudoku blank-cell accuracy under confidence decoding (3 seeds).

DifficultyRandomPAPLPUMA
Overall0.629±0.0050.632±0.0070.743±0.008
Hardest 1% (rating)0.358±0.0230.374±0.0140.601±0.019

Across seeds, the main results are consistently reproduced: reordering recovers Addition/Maze, alignment reduces accuracy on deeper ListOps/PUMA Countdown, and PUMA improves Sudoku.

Q3. Nested Maze strata and DEF errors

Yes, the strata are nested (≥60 ⊆ ≥30). Longer corridors can be easier for DEF because fewer side branches remain to prune. We newly added Figure 4 to clearly illustrate this structural difference and a DEF failure: an off-path cell is first predicted as on-path, followed by errors along that branch. Corridor length therefore stresses confidence's long-range decisions without making DEF elimination uniformly harder (Appendix B.5).

Comments on the paper

1 comment
AuthorsOpenReview ↗

General comment by Authors

We thank all four reviewers for their constructive feedback. Across the reviews, a shared question emerged: if confidence is suboptimal, why don't reference orders outperform it on every task, and why does confidence-aligned training help Sudoku while hurting elsewhere?

The new Section 3 starts from the purpose of decoding: help an imperfect model solve difficult problems by building on useful intermediate results. Sequential sampling from exact conditionals is order-invariant; for practical models, reveal order becomes a test-time computation problem. Confidence can favor a familiar shortcut before the required computation is resolved.

We deliberately use small models with known dependencies to trace these decisions and intervene on training. Varying carry span at fixed input length isolates dependency difficulty; LLaDA-8B reproduces the addition failure.

Section 3 and Table 5 establish two central points:

1. Reference schedules expose reasoning dependencies; their effectiveness depends on the learned predictor.

LSD-first follows carry propagation, DEF eliminates dead ends, Layered evaluates children before parents, and Step-sequential organizes equations. These are meaningful tests of how the model uses intermediate results. A natural procedure need not give the best reveal order for a learned predictor: matching a reference does not establish confidence's optimality or make it a reliable training target. Matched-training comparisons test the latter directly.

2. Order and training produce three informative patterns (Table 5).

  • Inference-time ordering bottleneck: On Addition and long-corridor Maze, reordering recovers accuracy on the same weights. The model can solve problems on which confidence fails, exposing reasoning capability that the default order leaves unused.
  • Cost of confidence alignment: PAPL/PUMA increase errors in several settings. Reordering can recover PUMA Addition and the Maze tail, while PAPL Addition, deeper ListOps, and PUMA Countdown remain below Random even under reference schedules. Alignment can affect both the chosen order and the predictions learned during training.
  • Effective alignment: On Sudoku, confident placements constrain subsequent choices, and PUMA improves an already effective policy. This positive case illustrates when reinforcing confidence supports further reasoning.

Key revisions and new experiments:

  • Reliability and settings (fDpN): Multi-seed results in Tables 1–4 and 14, PAPL α sweeps, and PUMA stage-schedule checks.
  • Alternative decoders (B4kq): P2/SAS comparisons assess learned ordering and recovery of reasoning accuracy.
  • Architecture control (EUTz): A parameter-matched AR Transformer trained on the same data and update budget.
  • Training and correction (FJnB): Random–PUMA mixing/continuation and WINO remasking comparisons.

Together, these results support broad random-mask training as a foundation and better inference-time ordering to use the model's reasoning capability.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.