acceptodds
Under review as a conference paper at ICLR 2027

Multi-Turn Reflective Masking Elicits Reasoning in Mask Diffusion Models

Abstract

While reasoning on autoregressive (AR) models is often performed by chain-of- thought reasoning and reflection, their refinement of previous outputs still relies on fully sequential generation, even when only local edits are needed. In contrast, the masking mechanism in Mask Diffusion Models (MDMs) naturally supports explicit local edits on previous outputs, allowing selective refinement without discarding previous answers and generating another from scratch. While this property more closely aligns with how humans correct mistakes by iterative local refinement, standard MDM decoding does not allow previously revealed tokens to be actively revisited and re-masked for further revision. We propose Reflective Masking (RM), a lightweight post-training framework that equips MDMs with context-dependent iterative self-correction. RM enables a model to revisit and revise prior outputs based on the evolving context, providing an adaptive way to allocate additional inference computation to selective revision rather than forward generation alone. To exploit information from previous turns, we further introduce History Reference, a parameter-free mechanism that incorporates intermediate denoising states during revision. Our approach requires no architectural changes and is easily applicable to existing MDMs. Across diverse tasks including text generation, Sudoku, and image editing, Reflective Masking consistently outperforms standard masking-based baselines, suggesting that explicit in-place revision can serve as a useful primitive for iterative reasoning and self-correction in MDMs.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Mask Diffusion Models, Reasoning, Reflective Masking, Iterative Refinement, Lightweight Post-Training.

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_8hnAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Reflective Masking (RM), a post-training framework that allows mask diffusion models (MDMs) to revisit previously revealed tokens. At each denoising step, a visible token is re-masked when the model assigns a higher probability to MASK than to the current token, while masked positions are filled with a predicted vocabulary token. The method trains this behavior using synthetic revision trajectories and oracle labels derived from target sequences. The paper further introduces History Reference (HR), which aggregates embeddings of previous denoising states using relative-distance rotations and temporal decay, and uses the resulting history-aware representation to condition the model. The paper provides a Bayes-consistency argument for predicting oracle revision actions, a plug-in excess-risk bound, and an information-monotonicity proposition for adding history. Experiments cover instruction-based image editing with Lumina-DiMOO, Sudoku correction with a small custom model, and text tasks using LLaDA. The reported results improve over base-model and vanilla-SFT baselines on the selected tasks. The paper positions RM as a lightweight alternative to approaches that add a dedicated remasking policy stream or require larger training pipelines.

Critical strengths
  1. A simple and interpretable mechanism for in-place revision.

The rule re-masks a visible token when the model assigns greater probability to MASK than to the current token, allowing the existing MDM output distribution to drive revision without introducing a separate policy stream. This is a practical design choice and is easy to understand.

  1. Evaluation across several task types with useful ablations.

The paper evaluates image editing, Sudoku correction, and text tasks. Reported results include 99.73% edit precision and 73.02% edit coverage for image editing; 93.4% exact Sudoku accuracy for the full RM+HR+decay+HER variant versus 82.4% for RM without HR; and MATH-500 accuracy of 24.8% versus 22.4% for Vanilla SFT, as well as MBPP pass rate of 39.4% versus 30.6%. The paper also studies the accuracy–computation trade-off and the contribution of HR under different corruption proposals.

Critical weaknesses
  1. The incremental contribution over existing remasking methods is not fully isolated.

Remasking and self-reflective revision have already been explored by methods such as ReMDM (NeurIPS 2025) and RemeDi (ICLR 2026). The paper’s distinctive value appears to be the lightweight post-training recipe and History Reference, rather than the general idea of revisiting committed tokens. However, the direct comparison with RemeDi is limited to MATH-500, where the reported scores are close (24.8% for RM versus 24.4% for RemeDi), and the paper does not provide a broad, compute- and data-matched comparison with existing remasking samplers across its main tasks. The theoretical results also provide limited support for the practical claims: the history monotonicity proposition is a generic “more information cannot worsen Bayes-optimal risk” result, while Theorem 1 assumes a sufficiently rich model family and conditions on an analyst-side variable that contains more information than the compressed representation actually passed to the model. Thus, the paper does not yet establish how much of the gains are uniquely attributable to the proposed RM/HR design.

  1. The training/evaluation protocol and empirical robustness need clearer validation.

The training procedure constructs histories by independently sampling per-position state transitions. For text, the practical wrong-token proposal is drawn from a frozen model’s top-k predictions on the clean target sequence, which may differ from the correlated error patterns encountered in the model’s own inference trajectories. The paper also states that MBPP is split randomly into 30% training and 70% testing, but does not make clear whether this split is within the official training partition or across the full benchmark; this makes the reported 39.4% difficult to compare with standard MBPP results. Main results are reported without multi-seed variance or confidence intervals, which matters for relatively small gains such as +2.4 percentage points on MATH-500 and +0.4 points over RemeDi. Finally, the submitted code archive lacks a README/environment manifest and a fully self-contained text preprocessing workflow. The code also instantiates a trainable history_output_norm LayerNorm even though HR is described as parameter-free, and the provided text launchers use different history-step defaults for training and evaluation. These details should be clarified for reproducibility and for interpreting the “parameter-free” claim.

Questions for authors
  1. Comparison and attribution.

Could the authors provide a more direct comparison with an inference-time remasking baseline such as ReMDM, and clarify the exact matching of base checkpoint, decoding budget, and training data when comparing with RemeDi? Which existing ablation best isolates the contribution of the RM decision rule from HR and ordinary task-specific SFT?

  1. MBPP protocol and uncertainty.

Does the stated 30%/70% MBPP split apply only within the official training split, or is it drawn from the full benchmark? Please clarify the exact data partition and any overlap with the canonical test set. Are multiple seeds or confidence intervals available for MATH-500, MBPP, Sudoku, and the image-editing user-study results?

  1. Training/inference alignment and HR implementation.

How well do the synthetic, independently sampled histories match the model’s actual inference-time error trajectories? Also, the released code includes a trainable history_output_norm LayerNorm and the provided training/evaluation launchers use different history-step defaults (6 versus 3). Could the authors clarify which settings produced the reported results, in what sense HR is parameter-free, and provide a fully reproducible setup for text preprocessing and training?

4 comments

We thank the reviewer 8hnA for the careful assessment, for inspecting our code, and for recognizing the simple and interpretable revision rule and the range of tasks and ablations in our evaluation.

W1/Q1. Comparison and attribution

As the reviewer notes, the distinctive parts of our method are the post-training recipe, which obtains revision from the model's own output distribution without a separate policy (Prop. 4), and HR. The existing ablations attribute the gains to these parts as follows.

Attribution of the gains. Each of the following comparisons changes one factor and keeps the others fixed.

  1. The RM decision rule. Tab. 6 uses the same RM-trained checkpoint and decoding settings, and only switches Eq. 1 on or off at inference. The rule improves MATH-500 by 2.2–2.6 points at all four reveal rates. We consider this the cleanest test of the rule itself.
  2. HR. With the RM objective and training data fixed, adding HR improves MATH-500 by 3.2 points with model-based wrong tokens and by 1.8 points with uniformly sampled wrong tokens (Sec. 3.3). On Sudoku, the full model reaches 93.4% exact accuracy, against 82.4% for RM without HR (Tab. 2).
  3. Ordinary task-specific SFT. Vanilla SFT and RM start from the same base model, are trained on the same task data, and use the same LLaDA decoding. RM improves MATH-500 from 22.4% to 24.8% and MBPP from 30.6% to 39.4% (Tab. 3). In image editing, with the same training data, edit precision rises from 71.84% to 99.73% (Tab. 1).

Comparison with inference-time re-masking(ReMDM). Item 1 above is also the most direct comparison with an inference-time re-masking sampler in the paper. RM-off decodes the same checkpoint with LLaDA's standard low-confidence re-masking, and RM-on adds the learned decision of Eq. 1. The gain does not come from extra forward passes. RM-on reaches 22.6% with 132 forward passes when two tokens are revealed per step, and 21.6% with 70 when four are revealed, while RM-off reaches only 20.2% even with 256. ReMDM belongs to the same class of samplers. It returns decoded tokens to MASK with a probability set by a schedule, optionally scaled by their confidence, and the model is not trained to judge whether a committed token is wrong. RM trains this judgment into the model, so ReMDM can also be run on top of an RM model.

RemeDi. The comparison is matched in the base checkpoint and the evaluation. Both methods are built on LLaDA-8B-Instruct, and we evaluate RemeDi's released checkpoint on the same MATH-500 problems with block size 512, our evaluation setting. Training is not matched, because RemeDi is used as released and retraining it on our data would not reflect the method as proposed:

MethodMATH-500 (%)#Parameters#Training samples
Vanilla SFT22.48B7.5K
RemeDi24.48.9B (adds a 4-block policy stream)~18.8M public + 140K in-house (Remask SFT + RL)
RM (ours)24.88B (no architectural change)7.5K

RM reaches comparable accuracy without an auxiliary policy network and with more than 2,000 times fewer training samples. RemeDi generates text only, so it cannot be compared in image editing.

W1/Q1. Comparison and attribution

Theory. Prop. 9 is only a sanity check that history cannot hurt at the Bayes level, and whether HR helps is left to the experiments (App. B.3). The method relies on Prop. 4 and Theorem 7. By Prop. 4, the minimizer at a visible position is α δz+(1−α) δMASK\alpha\,\delta_{z} + (1-\alpha)\,\delta_{\mathrm{MASK}}, where α\alpha is the probability that the current token is correct. Eq. 1 therefore re-masks a token exactly when α<1/2\alpha < 1/2, which is the Bayes decision on whether it is wrong. The rich-family assumption is the usual one for cross-entropy, and Theorem 7 covers an imperfect model: the excess decision error is at most twice the expected TV distance to the minimizer.

On the analyst-side variable, App. B conditions on (z,ϕ(H,c),c,E)(z, \phi(H, c), c, E), while the model receives (a(t),c)(a^{(t)}, c). We close this gap with the following result, which we will add to App. B.3.

Proposition. If the history length is smaller than d/2d/2, then for almost every embedding matrix, the accumulated embedding a(t)a^{(t)} of Eq. 2 uniquely determines the current state and the full history. This also holds when a normalization with non-zero gain is applied, as in App. D.

Hence, conditioning on (a(t),c)(a^{(t)}, c) is equivalent to conditioning on (z,H,c)(z, H, c), where zz is the current state, HH the earlier states of the trajectory, and cc the task condition. Theorem 1 therefore applies to the input the model actually receives. The key is the rotation in Eq. 2, which depends on the distance from the current step and works like a time stamp. Two different trajectories give the same input only if the embedding matrix satisfies a linear equation. On the jj-th rotation block, the coefficient of one token's embedding in this equation is a non-zero polynomial of degree at most tt, evaluated at γe−iθj\gamma e^{-i\theta_j}. With d/2>td/2 > t distinct angles, the equation is non-trivial on at least one block, so such embedding matrices form a measure-zero set. For LLaDA-8B, d/2=2048d/2 = 2048, far above the history lengths we use. The same argument covers the text implementation, where the history sum is normalized before it is added to the current embedding (see Q3 below).

W2/Q2. MBPP protocol and uncertainty

MBPP split. As described in Sec. 3.3, MBPP is split at random. We take all 974 problems of the full MBPP configuration (Muennighoff/mbpp) and divide them with seed 42 into 30% for training (292 problems) and 70% for testing (682 problems). The two parts are disjoint. Vanilla SFT and RM are trained on the same 292 problems, and LLaDA, Vanilla SFT and RM are evaluated on the same 682 problems, so the MBPP comparison in Tab. 3 is controlled in both training data and evaluation. We will add the seed and the problem counts to Sec. 3.3.

Uncertainty. We report 95% binomial intervals and tests computed from the existing evaluations. On MBPP, RM solves 269 of the 682 test problems and Vanilla SFT 209. The intervals are 39.4 ± 3.7 and 30.6 ± 3.5, which do not overlap, and a two-proportion test gives p≈0.0007p \approx 0.0007. This unpaired test is conservative, since both models are evaluated on the same problems. On Sudoku, the full model and RM without HR reach 93.4 ± 2.2 and 82.4 ± 3.3 on the 500 test boards (Tab. 2). On MATH-500, the effect of revision can be measured on the same problems. After reflective revision, 21 problems change from incorrect to correct and none from correct to incorrect, which gives p<10−6p < 10^{-6} under an exact two-sided sign test. The direction of the effect is also stable across settings. In all four paired runs of Tab. 6, turning revision on with the same checkpoint improves MATH-500 by 2.2–2.6 points. On the larger 5,000-problem Minerva MATH evaluation (App. F), RM improves the aggregate accuracy over Vanilla SFT from 22.62% to 24.10% and is better or equal in all seven subject categories.

W2/Q3. Training–inference alignment and the HR implementation

Synthetic histories and inference-time errors. The synthetic histories target the two main failure modes at inference. The first is a plausible but wrong token, so training wrong tokens are drawn from the frozen backbone's top-50 predictions (App. D). This is closer to inference than uniform wrong tokens and gives 24.8% against 20.0% on MATH-500 (Sec. 3.3). The second is putting back an error that was just erased. The training trajectories contain this wrong token → MASK → gold token path, and with HR the replay-mistake rate on Sudoku drops from 0.57% to 0.03% (Tab. 2). Although targets are built per position, every decision at inference is predicted from the full sequence and history, so corrections remain coupled (Fig. 4). On the model's own inference trajectories, revision turns 21 incorrect MATH-500 answers into correct ones and none in the other direction.

Settings of the reported results. All reported text models are trained with trajectory length T = 6 (train.sh) and decoded with a history window of 3 (eval.sh). The current step is sampled along the six-step trajectory, so training covers histories of different lengths, and the window of 3 lies within this range.

In what sense HR is parameter-free. The aggregation of Eq. 2 has no learned parameters, since the rotations are fixed sinusoidal blocks, the decay is a fixed hyperparameter, and the rotated states are summed. In the text model, the history sum is normalized before it is added to the current embedding, and only the gain and bias of this normalization (history_output_norm) are trained. They amount to 8,192 parameters, about 10−610^{-6} of LLaDA-8B, compared with roughly 0.9B for the policy stream of RemeDi. We will state this in Sec. 2.2.

W2/Q3. Training–inference alignment and the HR implementation

Reproducible setup.

Following the reviewer's suggestion, we provide the complete recipe below, and we will add it to the appendix.

Text recipe (LLaDA, implemented with the dLLM library; the same training recipe is used for MATH-500, MBPP and ARC-Challenge):

Text
Base modelLLaDA-8B-Instruct, full fine-tuning
Training dataMATH: 7,500 training problems; MBPP: a random 30% of the full benchmark (292 of 974 problems, seed 42); ARC-C: training split (1,119 questions)
Test dataMATH-500; MBPP: the remaining 70% (682 problems); ARC-C: test split (1,172 questions)
OptimizerAdamW (fused), β₁ = 0.9, β₂ = 0.999, ε = 1e-8, weight decay 0, gradient clipping 1.0
LR schedulelr 2e-5, cosine decay, warmup ratio 0.1
Precisionbf16, FSDP full-shard
Batch and lengthbatch 2 per GPU, gradient accumulation 4, 2 GPUs (global batch 16); max length 2048
Epochs1
Lossper-position cross-entropy on keep / re-mask / reveal targets (Eq. 4), all weights 1.0
Wrong tokenssampled from the top-50 predictions of the frozen base model, computed once on the clean target and excluding the target token (App. D)
History (HR)trajectory length T = 6, temporal_history_decay = 0.2, rotary distance encoding (base 10000)
Decoding0-shot, chat template, lm-evaluation-harness; Eq. 1 applied at every step; cfg scale 0. ARC-C is evaluated forward-only with revision disabled (Sec. 3.3)
Decoding lengthTab. 3: generation length 512 in a single block (block size 512).
Decoding stepseach step both unmasks and re-masks tokens, so the number of steps is not fixed; decoding stops once the sequence no longer changes

Sudoku recipe:

Sudoku
Model4-layer Transformer MDM, hidden size 128, 4 heads, MLP ratio 4, dropout 0.1 (0.81M parameters)
DataRitvik19/Sudoku-Dataset on HuggingFace (about 17M puzzle–solution pairs). Training boards are sampled from its train split; validation and test boards come from its valid split, divided by a fixed hash of each puzzle (2,000 validation and 500 test boards)
Stage 14,000 boards: the dataset's original puzzles, with blank cells as MASK
Stage 28,000 boards: solved boards with 4–20 cells masked and 4–20 cells set to wrong digits
OptimizerAdamW, lr 1e-3, weight decay 1e-4, cosine schedule, batch 16, gradient clipping 1.0, fp32
Losscross-entropy over the full board, edited cells weighted 8×
History (HR)T = 6, temporal_history_decay = 0.4, rotary distance encoding, history feature dropout 0.1, trajectory perturbation rate 0.3
Seed7
Test500 boards: solved boards with injected wrong digits;

Code. The training and history settings above are those in Text/examples/llada/train.sh and Sudoku/configs/repair_9x9_temporal_rope.yaml, and the submitted eval.sh is set to the Tab. 6 decoding configuration. The public code release will also include a README with the environment setup and the text preprocessing scripts (top-k wrong-token sampling and trajectory construction).

We thank the reviewer 8hnA again for the constructive feedback.

Reviewer_ofnzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces Reflective Masking, a lightweight post-training framework that enables mask diffusion models to revisit previously committed tokens, re-mask unreliable predictions, and regenerate them in place. The method trains a single output distribution to support three position-wise actions—keep, re-mask, and reveal—without adding a separate revision-policy module. It also proposes History Reference, a parameter-free recurrent embedding that summarizes previous denoising states and helps avoid repeated mistakes

Critical strengths
  1. The paper formulates self-correction as three position-wise actions—keep, re-mask, and reveal—which align naturally with the generation mechanism of mask diffusion models.
  2. Reflective Masking requires no additional policy head or architectural modification, making it easier to apply to existing MDMs than methods such as RemeDi.
Critical weaknesses
  1. Learned re-masking and iterative self-correction have already been explored by methods such as RemeDi and DiffusionGemma. The main novelty lies in simplifying the implementation.
  2. The model is post-trained separately using task-specific data for each benchmark, so the experiments do not demonstrate that Reflective Masking is a transferable or task-agnostic skill.
  3. The theoretical analysis conditions separately on the current state and history, whereas the implementation adds their embeddings together. Consequently, the history monotonicity result does not directly guarantee that the implemented History Reference cannot hurt performance.
Questions for authors
  1. A substantial body of prior work has explored re-masking strategies for improving the generation quality of MDMs. Could the authors provide a more comprehensive comparison with these methods? In addition, the current evaluation covers only a limited set of base models; could the authors evaluate the proposed approach on a broader range of MDM families?

  2. Does training the model to use the [MASK] token as an explicit revision action affect its existing capabilities? Compared with alternative approaches—such as introducing an external verifier or directly using the model’s own confidence scores to guide re-masking—what are the specific advantages of the proposed learned masking mechanism?

4 comments

We thank the reviewer for the careful assessment, and for noting that the keep / re-mask / reveal formulation fits the generation mechanism of MDMs and that RM requires no additional policy head.

W1. Novelty relative to RemeDi and DiffusionGemma

RemeDi and DiffusionGemma show the value of revising committed predictions, and we discuss both in Sec. 4 and App. A. Our contribution goes beyond a simpler implementation in four ways.

First, RM needs no separate revision policy. A visible token is re-masked when the model assigns MASK a higher probability than the token itself, so the decision is read from the model's existing output distribution. At the population level, the model's output therefore already contains the decision that RemeDi learns with a dedicated policy stream.

Second, RM matches RemeDi at a much lower cost. Both are built on LLaDA-8B-Instruct, and we evaluate RemeDi's released checkpoint on MATH-500 with block size 512, our evaluation setting:

MethodMATH-500 (%)#Parameters#Training samples
Vanilla SFT22.48B7.5K
RemeDi24.48.9B (adds a 4-block policy stream)~18.8M public + 140K in-house (Remask SFT + RL)
RM (ours)24.88B (no architectural change)7.5K

RM reaches comparable accuracy without any extra module and with more than 2,000 times fewer training samples, and its post-training takes under 5 hours on 2 H100 GPUs. DiffusionGemma obtains self-correction through large-scale pretraining and task-specific fine-tuning.

Third, RM is more general. It works through the model's existing output vocabulary and needs no architectural change, so the same recipe applies to a multimodal MDM. In instruction-based image editing with Lumina-DiMOO, RM improves over SFT on the same data on every metric in Tab. 1. RemeDi and DiffusionGemma both generate text only.

Fourth, History Reference lets the model condition on its denoising history, while RemeDi decides re-masking from the current sequence alone. On MATH-500, HR improves accuracy by 3.2 points with model-based wrong tokens and by 1.8 points with uniformly sampled wrong tokens, using the same training data (Sec. 3.3).

W2. Task-specific post-training and transferability

RM is a post-training recipe. Our claim of generality refers to the training recipe rather than to a single model: the same objective, the same construction of training trajectories and the same inference rule (Eq. 1) are applied without modification to text generation with LLaDA, Sudoku, and image editing with Lumina-DiMOO. In each setting, RM is compared with SFT on the same task data, so the gains come from the revision capability rather than from additional data.

W3. The theory conditions on the current state and history separately, while the implementation adds their embeddings

We thank the reviewer for this observation. The gap can be closed. In Eq. 2, each historical state is rotated according to its distance from the current step, so the rotation works like a time stamp. The same token at different steps is rotated differently, and contributions from different steps cannot cancel unless the embeddings are specially aligned. The summed input therefore still determines the current state and every historical state. We will add the following result to App. B.3.

Proposition. If the history length is smaller than d/2d/2, then for almost every embedding matrix, the accumulated embedding a(t)a^{(t)} of Eq. 2 uniquely determines the current state and the full history. This also holds after the normalization step in App. D (with non-zero gain).

Hence, conditioning on the implemented input (a(t),c)(a^{(t)}, c) is equivalent to conditioning on (z,H,c)(z, H, c), where zz is the current state, HH the earlier states of the trajectory, and cc the task condition (notation of App. B). Theorem 1 therefore applies to the input the model actually receives, and by Prop. 9, adding HR cannot increase the best achievable error (Bayes risk) compared with the model without history.

Proof idea. For two different trajectories, the equation "their summed embeddings are equal" is linear in the embedding matrix. On the jj-th rotation block, the coefficient of one token's embedding is a non-zero polynomial of degree at most tt, evaluated at γe−iθj\gamma e^{-i\theta_j}. This polynomial has at most tt roots, and there are d/2>td/2 > t distinct angles, so the equation is non-trivial on at least one block. The embedding matrices that satisfy it form a measure-zero set. The full proof will be included in App. B.3.

For LLaDA-8B, d/2=2048d/2 = 2048, far above the history lengths we use. Empirically, adding HR improves MATH-500 under both wrong-token proposals (Sec. 3.3).

Q1. Broader comparison with re-masking methods and more MDM families

Our main comparison is already against a re-masking strategy. LLaDA's standard decoding uses low-confidence re-masking, and Vanilla SFT and RM both decode with it, starting from the same base model and trained on the same data. With the same decoding, RM improves MATH-500 from 22.4% to 24.8% and MBPP from 30.6% to 39.4% (Tab. 3). Among learned approaches, RemeDi is the closest, and we compare with it under W1.

ReMDM, mixed noise schedules and per-step resampling modify the sampler. They decide which tokens to re-mask by a fixed schedule or a confidence rule, and the model is never trained for this decision. RM instead trains the model itself to decide whether a committed token should be re-masked (Eq. 1). The two act at different levels, so these samplers can also be run on top of an RM model.

Regarding model families, our experiments cover three MDMs: LLaDA-8B for text, the multimodal Lumina-DiMOO for image editing, and a small MDM trained from scratch for Sudoku. RM needs no architectural change, so the same recipe applies to other MDMs such as Dream. For block diffusion models, RM applies within the active block, and extending it to completed blocks is discussed in Sec. 5.

Q2. Effect on existing capabilities, and comparison with confidence-based re-masking and external verifiers

Effect on existing capabilities. We find no negative effect. RM keeps the training target at masked positions unchanged. By Prop. 6, the minimizer at a MASK position is the conditional distribution of the gold token, which is the same target as LLaDA's original objective. The new supervision is placed only on visible positions, which LLaDA's loss does not cover. Empirically, on ARC-Challenge, evaluated forward-only with revision disabled, the RM model scores 86.1% against 81.3% for Vanilla SFT (Tab. 3). On MATH-500, revision produces 21 incorrect-to-correct and 0 correct-to-incorrect transitions. In image editing, background SSIM rises from 0.66 to 0.97 (Tab. 1), so regions that should not change are preserved.

Advantages over confidence-based re-masking. As noted in Q1, LLaDA's decoding already re-masks by confidence, and RM improves over it in Tab. 3. Tab. 6 gives the same comparison on a single checkpoint: turning on the learned re-mask decision improves MATH-500 by 2.2–2.6 points at all four reveal rates. The learned mechanism has two advantages. First, its signal is trained. Standard MDM training never supervises the output at visible positions, because visible tokens are always correct during training, so the confidence of a committed token says little about whether it is wrong. RM trains this output directly. At a visible position, p(MASK)p(\mathrm{MASK}) estimates the probability that the token is wrong given the current context (Prop. 4), and it is recomputed at every step. Second, it needs no threshold. A confidence rule must specify how low the confidence should be, or how many tokens to re-mask per step. Eq. 1 compares two probabilities, which is the Bayes-optimal decision at the population optimum (App. B), so the model itself decides how many tokens to re-mask at each step.

Advantages over external verifiers. RM needs no second model. The re-mask decision is made in the same forward pass used for unmasking, and in Tab. 6 it costs only 3–10% more forward passes. A verifier usually scores a whole sequence or reasoning step, so it does not indicate which tokens to change. The two approaches are complementary, and a verifier can still be applied on top of RM.

We thank the reviewer ofnz again for the constructive feedback.

Thank you for the detailed rebuttal and clarifications. I appreciate the additional discussion of RM’s efficiency relative to RemeDi. However, my main concerns remain unresolved: the novelty over existing learned re-masking approaches is still limited, the task-specific post-training does not establish transferability, and no broader comparison across re-masking methods or text MDM families is provided. Therefore, while the rebuttal improves the presentation and positioning of the work, it does not sufficiently change my overall assessment, and I will maintain my original score.

Reviewer_K8XHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The motivation of this paper is to help generative models correct earlier mistakes while preserving correct content. Standard masked diffusion decoding fixes generated tokens, even when information generated in subsequent steps suggests a correction. The proposed Reflective Masking method uses post-training on synthetically corrupted examples to teach the model to keep correct tokens, erase incorrect ones by predicting MASK, and fill masked positions. It also uses a compact history of previous attempts to help avoid repeating mistakes during revision. Experiments show improved image-editing background preservation, Sudoku and text-task performance, and an improved accuracy–latency tradeoff in a MATH-500 decoding sweep.

Critical strengths
  1. The correction rule is simple and works through the model’s existing output vocabulary. Predicting MASK provides the decision to erase a token. Image editing gives a clear practical result: background SSIM improves from 0.66 with matched SFT to 0.97, while VQAScore also improves (Table 1). This supports the usefulness of local revision.

  2. The history ablations and inference sweep provide useful evidence. Sudoku accuracy rises from 82.4% without history to 93.4% with the full mechanism, and history also improves MATH-500 performance. Table 6 shows a useful accuracy–compute tradeoff: one RM configuration achieves higher accuracy with roughly half the forward passes of a slower configuration without revision.

Critical weaknesses
  1. The added value over existing correction methods needs clearer evidence. RemeDi already learns to remask generated tokens, and CDLM trains on visible incorrect tokens. This paper offers a simpler MASK-action rule and a history mechanism, but its comparison with RemeDi covers only one math benchmark and shows similar accuracy. The lighter adaptation is promising, although different training setups make its advantage difficult to assess. A focused comparison under comparable training and inference budgets would strengthen the contribution.

  2. Important details of the text experiments are missing. MBPP uses a custom random split without item IDs or a split seed. The optimizer and update settings, history-decay value, and Sudoku dataset sizes are also unspecified. These details are needed to reproduce the results and assess the claimed training cost. Providing the recipe during rebuttal would address this concern.

  3. The motivation and presentation could be clearer. The paper moves between efficient inference, self-correction and reasoning, but the relationship between these goals needs more explanation. Stating the main problem first, then explaining the roles of Reflective Masking and revision history, would make the method easier to follow.

Questions for authors
  1. What evidence shows a practical advantage over a close correction method such as RemeDi or CDLM under comparable training and inference budgets?

  2. Could you provide the complete text-training and evaluation recipe, including the exact MBPP split?

3 comments

We thank the reviewer for the careful assessment and for recognizing the simplicity of the correction rule, the image-editing results, and the accuracy–computation trade-off of Reflective Masking (RM). Below, we clarify the comparison with RemeDi and CDLM, provide the complete training and evaluation recipe, and clarify the motivation.

W1 / Q1: What evidence shows a practical advantage over RemeDi or CDLM under comparable training and inference budgets?

RemeDi is a learned-remasking method that also targets the revision of committed predictions, and is therefore the most relevant comparison. Both are built on LLaDA-8B-Instruct, and we evaluate RemeDi's released checkpoint on MATH-500 with block size 512, which matches our evaluation setting:

MethodMATH-500 (%)#Parameters#Training samples
Vanilla SFT22.48B7.5K
RemeDi24.48.9B (adds a 4-block policy stream)~18.8M public + 140K in-house (Remask SFT + RL)
RM (ours)24.88B (no architectural change)7.5K

RM reaches comparable accuracy without an auxiliary policy network and with more than 2,000 times fewer training samples, and its post-training finishes within 5 hours on 2 H100 GPUs. At a similar accuracy, the advantage of RM is that it needs far less training and no change to the model.

Within our own experiments, Vanilla SFT and RM start from the same base model, are trained on the same task data, and use the same standard LLaDA confidence-based decoding strategy, so their comparison isolates the benefit of enabling active revision of committed tokens: MATH-500 improves from 22.4% to 24.8% and MBPP from 30.6% to 39.4% (Tab. 3). The extra inference cost is small. In Tab. 6, RM-off and RM-on use the same checkpoint and decoding settings, and RM improves MATH-500 by 2.2–2.6 points at all four reveal rates with only 3–10% more forward passes (e.g., 5.73 s → 5.93 s per query with one token revealed per step).

CDLM is concurrent work (NeurIPS 2026). CDLM fine-tunes all transformer weights only for a 0.5B code model (Open-dCoder-0.5B). At 8B, it trains only a LoRA adapter on LLaDA-8B-Base, and reports that its objective does not improve generation from scratch over the base model at that scale. Its main evaluation is instead the repair of given corrupted programs (its Code Revision Benchmark). Our method RM post-trains all parameters of LLaDA-8B-Instruct and improves generation from scratch over both the base model and SFT on the same task data (Tab. 3), so it is effective at the 8B scale where CDLM reports no gain. We will add this discussion to the related work.

W2/Q2. Complete training and evaluation recipe

Following the reviewer's suggestion, we provide the complete recipe below, and we will add it to the appendix.

Text recipe (LLaDA, implemented with the dLLM library; the same training recipe is used for MATH-500, MBPP and ARC-Challenge):

Text
Base modelLLaDA-8B-Instruct, full fine-tuning
Training dataMATH: 7,500 training problems; MBPP: a random 30% of the full benchmark (292 of 974 problems, seed 42); ARC-C: training split (1,119 questions)
Test dataMATH-500; MBPP: the remaining 70% (682 problems); ARC-C: test split (1,172 questions)
OptimizerAdamW (fused), β₁ = 0.9, β₂ = 0.999, ε = 1e-8, weight decay 0, gradient clipping 1.0
LR schedulelr 2e-5, cosine decay, warmup ratio 0.1
Precisionbf16, FSDP full-shard
Batch and lengthbatch 2 per GPU, gradient accumulation 4, 2 GPUs (global batch 16); max length 2048
Epochs1
Lossper-position cross-entropy on keep / re-mask / reveal targets (Eq. 4), all weights 1.0
Wrong tokenssampled from the top-50 predictions of the frozen base model, computed once on the clean target and excluding the target token (App. D)
History (HR)trajectory length T = 6, temporal_history_decay = 0.2, rotary distance encoding (base 10000)
Decoding0-shot, chat template, lm-evaluation-harness; Eq. 1 applied at every step; cfg scale 0. ARC-C is evaluated forward-only with revision disabled (Sec. 3.3)
Decoding lengthTab. 3: generation length 512 in a single block (block size 512).
Decoding stepseach step both unmasks and re-masks tokens, so the number of steps is not fixed; decoding stops once the sequence no longer changes

MBPP split. As described in Sec. 3.3, MBPP is split at random. We take all 974 problems of the full MBPP configuration (Muennighoff/mbpp) and divide them with seed 42 into 30% for training (292 problems) and 70% for testing (682 problems). The two parts are disjoint. Vanilla SFT and RM are trained on the same 292 problems, and LLaDA, Vanilla SFT and RM are evaluated on the same 682 problems, so the MBPP comparison in Tab. 3 is controlled in both training data and evaluation. We will add the seed and the problem counts to Sec. 3.3.

Sudoku recipe:

Sudoku
Model4-layer Transformer MDM, hidden size 128, 4 heads, MLP ratio 4, dropout 0.1 (0.81M parameters)
DataRitvik19/Sudoku-Dataset on HuggingFace (about 17M puzzle–solution pairs). Training boards are sampled from its train split; validation and test boards come from its valid split, divided by a fixed hash of each puzzle (2,000 validation and 500 test boards)
Stage 14,000 boards: the dataset's original puzzles, with blank cells as MASK
Stage 28,000 boards: solved boards with 4–20 cells masked and 4–20 cells set to wrong digits
OptimizerAdamW, lr 1e-3, weight decay 1e-4, cosine schedule, batch 16, gradient clipping 1.0, fp32
Losscross-entropy over the full board, edited cells weighted 8×
History (HR)T = 6, temporal_history_decay = 0.4, rotary distance encoding, history feature dropout 0.1, trajectory perturbation rate 0.3
Seed7
Test500 boards: solved boards with injected wrong digits;

Code. The training and history settings above are those in Text/examples/llada/train.sh and Sudoku/configs/repair_9x9_temporal_rope.yaml, and the submitted eval.sh is set to the Tab. 6 decoding configuration. The public code release will also include a README with the environment setup and the text preprocessing scripts (top-k wrong-token sampling and trajectory construction).

W3. Motivation and presentation

We thank the reviewer for this suggestion. We will restructure the introduction to state the problem first and then explain the roles of RM and HR.

The problem is that standard MDM decoding follows an absorbing process: once a token is committed, it is fixed, so an early error persists and affects the tokens generated after it. RM addresses this with self-correction. When the model assigns a higher probability to MASK than to a committed token, the token is returned to MASK and reconsidered in later denoising steps. Once tokens can be erased, the model may put back the same wrong token it has just removed, and HR gives the model access to its earlier states so that it can avoid repeating the error.

Efficiency follows from how this correction is done. Revision happens in place, so the model re-predicts only the tokens it re-masks instead of regenerating the whole output, as an AR model would. Extra inference computation therefore goes to selectively revisiting committed content rather than only to forward denoising, and in Tab. 6 this costs 3–10% more forward passes.

By reasoning, we mean a concrete behavior: the model repeatedly revises its current output, and each revision depends on the context left by earlier ones. For example, in Fig. 4, revising an earlier expression changes the context, after which the model revises the dependent computation and finally the answer. Our central claim is therefore not that RM already solves the hardest long-horizon reasoning problems, but that it enables context-dependent, in-place revision as a reasoning primitive native to MDMs. We will use this wording consistently throughout the paper.

We thank the reviewer K8XH again for the constructive feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.