acceptodds
Under review as a conference paper at ICLR 2027

Anchors Enhance Controllability and Reasoning of Diffusion Language Models

Abstract

Diffusion language models (DLMs) offer new opportunities for controllable text generation. However, their controllability, defined as the ability to satisfy task constraints while preserving their original capabilities, remains underexplored. DLMs’ bidirectional attention allows control tokens to guide generation from arbitrary positions in the output sequence, but effective guidance depends on the compatibility between their content and placement. To this end, we propose AnchorDLM, a framework in which an anchor generator produces an anchor consisting of a short output fragment and its position for each query. The anchor is placed as control tokens at the specified position to initialize the output sequence of a frozen DLM and guide subsequent denoising. We train the anchor generator with reinforcement learning (RL) to jointly optimize anchor content and placement, using rewards computed from the DLM's generated outputs based on answer correctness and constraint satisfaction. Moreover, through the lens of discrete flow matching (DFM), we interpret anchor placement as helping to reshape the initial velocity field that guides generation. Experiments on code generation and mathematical reasoning across five DLM backbones show that AnchorDLM improves controllability and further yields gains in reasoning performance across diverse benchmarks.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Diffusion Language Model

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_4HS7OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The author provides a new framework regarding the controllability of the DLMs. It uses an external anchor generator to generate a {"Position": "Content"} to guide the decoding of the DLMs. The method has good performance on the selected metrics.

Critical strengths
  1. The paper uses something AR models can’t do. It exploits DLMs’ bidirectional denoising to place guidance anywhere in the output, not just at the start.

  2. The approach is model-agnostic.

  3. The controllability problem is good and important.

Critical weaknesses
  1. The most important concern for me is the experimental part. The vanilla/prompt accuracies are well below the published numbers (especially LLaDA; refer to the LLaDA paper), and LLaDA-Ins scores are below LLaDA-Base on GSM8K. The numbers can vary in the actual experiments, which is understandable, but shouldn't be so different. No decoding configuration is reported: generation length, steps, few-shot examples, chat template. I assume no block diffusion is applied. What decoding strategies does LLaDA/Dream use? Low-confidence and high-entropy?

  2. Comparison is weak, and the reasoning gain is minimal. Given there is no variability (+- std) reported, a lot of the performance gain can be simply attributed to randomness. How many seeds are reported for the main experiments? I assume better controllability can somehow lift the accuracy performance. If not, what is the point anyway? AnchorDLM gets Claude Opus 4.6 rewritten training data, SFT, and RL with 8 DLM rollouts per query. All baselines (Prompt, S3, Ti) are training-free. A trained baseline is missing, e.g., LoRA fine-tuning the DLM on the same Claude-rewritten data, or few-shot prompting with examples in the target style.

  3. Anchor could just leak answers. Table 2 itself shows that anchors from bigger models increase pass@1. So gains may come from Qwen’s or Claude’s reasoning being written into the answer, not from better control of the DLM.

Questions for authors
  1. How much of AnchorDLM’s gain comes from the learned position rather than the anchor content? Please report an ablation that keeps the generated content but places it (a) at a fixed position, e.g., p = 0 as in SFT Stage 1, (b) at a random position, or (c) at the position the DLM would naturally produce that content. Also, how is the predicted position index mapped to the DLM’s token positions, given that Qwen and LLaDA/Dream use different tokenizers? According to Figure 3, the anchor position does not have a significant effect on G-mean?

  2. An analysis of the anchor's actual outputs?

  3. Prompt setting: What is the prompt? ICL with answer formatting is a natural baseline to compare with, so it should be included.

  4. The most important flaw is the experiment's authenticity, and this directly affects my evaluation of this paper. If the author can resolve this, I am willing to raise my evaluation.

3 comments

Thank you for raising these questions about anchor placement, baseline strength, and evaluation. Moving the same anchor to a fixed or random position lowers all six reported scores. EOS handling explains a substantial part of the GSM8K baseline gap, and AnchorDLM outperforms vanilla LLaDA-Instruct under both EOS settings. We also clarify how we prevent test-answer leakage: Claude annotates only training examples, and the trained anchor generator receives only the question at test time.

We hope these clarifications and additional results address your concerns, and we would be grateful if you would consider raising your score. We are happy to answer any further questions.

Reply to Questions 1–2 and Weakness 2:

Anchor placement. In Figure 3, we held the generated content CC fixed and added an offset to its predicted position PP at inference time. To test whether the predicted position matters more broadly, we used the trained generator to produce (P,C)(P,C) once per query and placed the same content at the predicted position, at P=0P=0, or at a random position. Here, PP is a zero-based index into the target DLM’s masked output sequence, and P=0P=0 denotes the first output token. The generator emits PP as an integer and CC as text. The target DLM’s tokenizer determines the token span occupied by CC.

Placement on LLaDA-8B-BaseGSM8K-fmt GGSM8K-bwd GHumanEval-cmt GHumanEval-str GGSM8K Acc.HumanEval pass@1
Predicted position69.3371.0843.8950.4167.1737.20
Prefix (P=0)42.4363.3638.6828.7658.834.88
Random position13.6721.3734.519.1444.433.05

Both interventions lower all six scores, although sensitivity varies by task. Moving the same anchor to the prefix lowers HumanEval-cmt G-mean by 5.21 points and HumanEval pass@1 by 32.32 points. Because the anchor content is identical across placements for each query, these differences isolate the contribution of placement and support adapting it to the query and the generated span.

Actual anchor outputs. Figure 5 illustrates what an anchor contributes on MBPP task 389, which asks the model to count distinct unordered pairs. The reasoning anchor supplies the condition if (first, second) in result:. The DLM completes the surrounding logic, including ordering each pair before the lookup and updating the dictionary. The unanchored completion counts reversed pairs separately. This example shows both the algorithmic information supplied by the anchor and the work performed by the backbone.

Variability and trained baseline. The original experiments used a single seed. We evaluated all methods on LLaDA-8B-Base across three seeds and report means ± sample standard deviations. We also added a baseline that directly fine-tunes LLaDA-8B-Base with LoRA to generate the full Claude-rewritten solutions for the same training questions used to construct the supervised anchor targets. G-mean evaluates controlled generation, while GSM8K accuracy and MBPP pass@1 evaluate the reasoning configurations.

MethodGSM8K-fmt GMBPP-cmt GGSM8K Acc.MBPP pass@1
Vanilla5.61 ± 0.8429.20 ± 0.7766.06 ± 0.6540.07 ± 0.23
Prompt8.90 ± 0.8331.39 ± 0.5766.69 ± 0.5840.73 ± 0.76
S³37.00 ± 0.7931.25 ± 0.9053.02 ± 0.8717.60 ± 0.72
Ti38.21 ± 1.0931.09 ± 0.9351.03 ± 1.1817.53 ± 0.95
LoRA-fine-tuned DLM55.82 ± 1.6938.04 ± 0.8266.48 ± 0.6741.73 ± 0.95
AnchorDLM68.94 ± 1.3739.20 ± 0.8367.73 ± 0.4944.00 ± 0.40

AnchorDLM achieves the highest mean on all four metrics, including against the new LoRA baseline. These results support our conclusion on the two benchmarks tested: AnchorDLM improves controlled generation, with the largest gain on GSM8K format control, and yields more modest gains in reasoning performance.

Controllability and accuracy. Control comes at a small accuracy cost, which we report separately below.

BackboneMethodHumanEval-cmt Pass@1HumanEval-cmt Control ScoreHumanEval-cmt G-meanMBPP-cmt Pass@1MBPP-cmt Control ScoreMBPP-cmt G-mean
LLaDA-8B-BaseVanilla34.156.2314.5939.8022.2729.77
Prompt31.7117.1023.2937.4025.9631.16
S³14.6321.4117.7020.6044.5630.30
Ti15.2418.6516.8621.2042.7730.11
AnchorDLM30.4963.1743.8935.0045.8340.05
LLaDA-8B-InstructVanilla39.0222.8029.8340.8019.5628.25
Prompt34.7659.9245.6440.8057.2948.35
S³37.8039.7638.7731.8034.4733.11
Ti34.7640.7537.6424.0036.3129.52
AnchorDLM34.1562.1646.0736.4068.5649.96

Overall, AnchorDLM achieves a good balance between accuracy and controllability, with the highest G-mean in all four settings.

Reply to Question 3:

We used the backbone’s released in-context learning template and rewrote the demonstration answers in the target style. The few-shot example below is one of the four demonstrations we used to evaluate LLaDA-8B-Instruct on GSM8K-bwd. Before performing any arithmetic, the answer works backward from the final total to the original item counts, tracing the number of cards to the number of frisbees and then to the known number of marbles.

dict(
    role='HUMAN',
    prompt="Question: Bella has two times as many marbles as frisbees. She also has 20 more frisbees than deck cards. If she buys 2/5 times more of each item, what would be the total number of the items she will have if she currently has 60 marbles?\nLet's think step by step\nAnswer:",
),
dict(
    role='BOT',
    prompt="To find Bella's total after the purchase, we need her final count of each item. To find each final count, we need its original count so we can add 2/5 more. To find the original deck card count, we need the frisbee count. To find the frisbee count, we can work from the 60 marbles given in the question.\nBella has twice as many marbles as frisbees, so she has 60/2 = 30 frisbees. She has 20 more frisbees than deck cards, so she has 30-20 = 10 deck cards.\nShe buys 2/5*60 = 24 more marbles, giving 60+24 = 84 marbles. She buys 2/5*30 = 12 more frisbees, giving 30+12 = 42 frisbees. She buys 2/5*10 = 4 more deck cards, giving 10+4 = 14 deck cards.\nTogether, Bella will have a total of 84+42+14 = 140 items.\nThe answer is 140\n",
),

Reply to Question 4 and Weakness 1:

Thanks for pointing this out.

All results use full-sequence (pure) diffusion sampling; no block diffusion is applied. We use a generation length of 256 with 128 denoising steps, i.e. two tokens per step as in d1 [2], and each backbone's official unmasking strategy: low-confidence remasking for LLaDA and entropy-based unmasking for Dream.

To reduce interference from the heuristic that lowers EOS token confidence [1], we used full-sequence diffusion decoding with this heuristic disabled, so that changes in decoding order are attributable to the anchor alone. The lower GSM8K accuracy of LLaDA-8B-Instruct under this setting is consistent with the explanation in Appendix B.4 of LLaDA [1]. Heavy EOS padding during supervised fine-tuning can cause premature termination under pure diffusion sampling, shortening responses and reducing accuracy.

We now report both settings:

Model / Method (EOS confidence suppression)GSM8K-fmt GHumanEval-cmt GMBPP-str GGSM8K Acc.HumanEval pass@1MBPP pass@1
LLaDA-8B-Base · Vanilla5.0114.5943.1666.0334.1539.80
LLaDA-8B-Instruct · Vanilla (off)17.0829.8332.0656.1039.0240.80
LLaDA-8B-Instruct · AnchorDLM (off)61.8546.0750.9058.5341.4641.40
LLaDA-8B-Instruct · Vanilla (on)19.3031.3732.2766.1942.0741.00
LLaDA-8B-Instruct · AnchorDLM (on)66.4148.1552.8667.1744.5143.20

As shown in the table, enabling EOS confidence suppression improves the vanilla baseline. With EOS confidence suppression enabled, AnchorDLM outperforms baselines, demonstrating that our method remains effective under this decoding setting.

Reply to Weakness 3:

There is no test-answer leakage in our anchor training or evaluation procedure. Claude was used exclusively to rewrite and annotate training examples. All supervised anchors and preference pairs were constructed from training data. At test time, the 1.7B generator receives only the question and predicts an anchor without access to Claude or the reference solution. For mathematical reasoning tasks that require a final numerical answer, our annotation instructions explicitly exclude the final answer itself when selecting the anchor (Appendix A.2). To limit the amount of guidance provided to the DLM, we use short anchor spans. Across benchmarks, these spans account for 4.33%–10.88% of the generated tokens on average (Table 4).

Table 2 examines how generator capacity affects downstream performance. Increasing the generator size from 0.6B to 1.7B improves pass@1 on both benchmarks, while larger generators yield inconsistent further gains. We therefore selected 1.7B to balance anchor quality and inference cost.

References

[1] Nie et al. Large Language Diffusion Models. NeurIPS, 2025. Appendix B.4 and official OpenCompass evaluation configuration.

[2] Zhao et al. d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learning. NeurIPS, 2025. Appendix D.

Reviewer_J5nJOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces AnchorDLM, which employs a trained generator to predict semantic segments—along with their corresponding positions in the output—for each input. These segments are then inserted into the initial masked sequence of a frozen diffusion language model (DLM) to guide the subsequent generation process. The generator undergoes initial supervised fine-tuning, followed by optimization via WPO (Weighted Preference Optimization) using feedback derived from the downstream DLM’s outputs. The authors evaluate the method’s controllable generation and inference performance across five model configurations—based on LLaDA and Dream—and further analyze its effectiveness through ablation studies on the training process, experiments involving positional perturbations, and transferability tests.

Critical strengths
  1. The motivation for the method is clear, and its design holds practical value. By learning semantic segments and their corresponding positions for various inputs, the approach leverages the bidirectional completion capabilities of DLM (Decoder-only Language Models) with greater flexibility. The design—freezing the backbone model while training an external generator—also facilitates the reuse of existing models.

  2. The results of controlled generation are highly consistent. Across the 30 control settings tested, AnchorDLM consistently outperformed the strongest baseline (sharing the same backbone) in terms of G-mean, with an average improvement of 11.30 points. These results—spanning mathematical expressions, logical reasoning, and code style/structure—validate the effectiveness of the proposed method as a control module.

  3. The experimental analysis is comprehensive. The authors examine the impacts of data augmentation, SFT (Supervised Fine-Tuning), preference optimization, and positional perturbations, while also providing transfer learning results. These experiments offer valuable insights into the training regimen and the compatibility between content and positional information.

Critical weaknesses
  1. The sensitivity of the conclusions to the weighting of metrics—as well as the trade-off between accuracy and control—remains unclear. The control experiments in Table 1 employ an unweighted geometric mean (G-mean) of the accuracy score and the control score, yet the two components are not reported separately. An improvement in the G-mean does not rule out a potential decline in accuracy; furthermore, the inference results presented on the right—which utilize a different set of generators—do not confirm that the "controlled" versions maintain the original level of accuracy.

  2. The definition of the "annotation control" metric requires clarification. Appendix A.3 describes a process of first merging consecutive identical code/comment labels, followed by calculating the proportion of transitions between adjacent labels. If implemented literally, the merged adjacent labels would necessarily be distinct; provided that both types of labels are present, this ratio would always equal 1. Consequently, this metric fails to distinguish between an initial block of comments and a truly interleaved layout of code and comments.

  3. The gains in inference performance are relatively modest, and their origins could be explained more clearly. Compared to the strongest baseline sharing the same backbone, the inference results show an average improvement of 0.91 percentage points. In Table 3, the SFT-only approach yields only limited improvements in accuracy on the GSM8K and MBPP datasets, with the primary gains emerging only after subsequent preference optimization. Additionally, the incremental benefit of WPO over DPO varies significantly between the two tasks.

Questions for authors
  1. Could you report the accuracy (A) and control score (S) for the same set of outputs in the control experiments? Additionally, please conduct a sensitivity analysis across all baselines using a weighted geometric mean (Gα=AαS1−αG_\alpha = A^\alpha S^{1-\alpha})—for instance, by comparing results for α∈{0.25,0.5,0.75}\alpha \in \{0.25, 0.5, 0.75\}. When higher weight is assigned to accuracy, do the rankings of the methods—and the relative advantage of the strongest baseline—remain consistent?

  2. Could you provide the precise formula or pseudocode for the "annotated control" metric, clearly specifying the numerators and denumerators both before and after merging labels? Please also illustrate the differences in scoring using two examples: one involving a leading annotation block and another with interleaved annotations. Furthermore, please clarify which specific implementation was used for the results presented in Table 1.

  3. In a representative experimental setup, could you compare your approach against a semantic generator that undergoes similar SFT/WPO training but is constrained by fixed positional requirements (e.g., a model trained to generate content specifically for a prefix position)? This comparison would help clarify the extent to which adaptive positioning provides additional gains under equivalent preference optimization conditions.

2 comments

Thank you for the constructive review. Our sensitivity analysis shows that AnchorDLM achieves the strongest comment control across the tested settings, while method rankings depend on the weight assigned to accuracy. We clarify how the comment score distinguishes interleaved comments from a leading comment block and show that learned anchor placement outperforms fixed placement on both benchmarks.

We hope these clarifications and additional results address your concerns, and we would be grateful if you would consider raising your score. We are happy to answer any further questions.

Reply to Question 1 and Weakness 1:

For HumanEval-cmt and MBPP-cmt, we report pass@1 (AA) and the comment-interleaving score (SS), both computed on the same controlled outputs. We evaluate all four same-backbone baselines using Gα=AαS1−αG_{\alpha} = A^{\alpha} S^{1-\alpha}, where larger α\alpha assigns more weight to correctness.

HumanEval-cmt

BackboneMethodASG_{0.25}G_{0.50}G_{0.75}
LLaDA-8B-BaseVanilla34.156.239.5314.5922.32
Prompt31.7117.1019.9523.2927.17
S³14.6321.4119.4717.7016.09
Ti15.2418.6517.7316.8616.03
AnchorDLM30.4963.1752.6543.8936.58
LLaDA-8B-InstructVanilla39.0222.8026.0829.8334.12
Prompt34.7659.9252.2945.6439.83
S³37.8039.7639.2638.7738.28
Ti34.7640.7539.1637.6436.17
AnchorDLM34.1562.1653.5246.0739.67

MBPP-cmt

BackboneMethodASG_{0.25}G_{0.50}G_{0.75}
LLaDA-8B-BaseVanilla39.8022.2725.7529.7734.42
Prompt37.4025.9628.4431.1634.14
S³20.6044.5636.7430.3024.98
Ti21.2042.7735.8930.1125.27
AnchorDLM35.0045.8342.8440.0537.44
LLaDA-8B-InstructVanilla40.8019.5623.5128.2533.95
Prompt40.8057.2952.6348.3544.41
S³31.8034.4733.7833.1132.45
Ti24.0036.3132.7429.5226.62
AnchorDLM36.4068.5658.5249.9642.64

AnchorDLM achieves the highest control score SS in all four settings and ranks first in G-mean at both α=0.25\alpha = 0.25 and α=0.50\alpha = 0.50. At α=0.75\alpha = 0.75, it remains in the top two, ranking first on both LLaDA-Base tasks and second on both LLaDA-Instruct tasks. Overall, AnchorDLM achieves a good balance between AA and SS.

Reply to Question 2 and Weakness 2:

Thank you for pointing out the ambiguity in Appendix A.3. We compute the comment-interleaving score on the original sequence of line labels, before merging. After the same post-processing used for pass@1, let t1,…,tnt_1,\ldots,t_n denote the code or comment labels of the nn scored lines in a function body. For n≥2n\geq 2, the per-sample score is

scmt=∑i=1n−11[ti≠ti+1]n−1.s_{\mathrm{cmt}} = \frac{\sum_{i=1}^{n-1}\mathbf{1}[t_i\neq t_{i+1}]}{n-1}.

The denominator counts all adjacent line pairs in the original sequence. The numerator counts how many of those pairs have different labels. Merging consecutive identical labels gives an equivalent way to obtain the numerator: if the merged sequence contains rr segments, there are exactly r−1r-1 switches. The denominator remains n−1n-1.

For example, M M M C C C has five adjacent pairs and one switch, giving 1/5=0.201/5=0.20. M C M C M C also has five adjacent pairs, but all five switch labels, giving 5/5=1.005/5=1.00. Here, M denotes a comment line and C a code line. Samples with fewer than two scored lines receive zero. We report the mean per-sample score as a percentage and will clarify this definition in Appendix A.3.

Reply to Question 3 and Weakness 3:

We trained three anchor generators with the position fixed at p∈{0,40,80}p\in\lbrace 0,40,80\rbrace. Each generator still learns query-dependent anchor content. We also evaluated a generator that receives a randomly assigned position before generating its content. The table reports means and sample standard deviations across three runs on LLaDA-8B-Base. The two GG columns evaluate controlled generation. Accuracy and pass@1 evaluate the corresponding reasoning configurations.

VariantGSM8K-fmt (GG)GSM8K Acc.MBPP-cmt (GG)MBPP pass@1
Vanilla5.61 ± 0.8466.06 ± 0.6529.20 ± 0.7740.07 ± 0.23
Fixed (p=0p=0)37.77 ± 2.3661.54 ± 0.5933.32 ± 0.9740.73 ± 0.31
Fixed (p=40p=40)34.13 ± 1.5565.22 ± 0.6635.37 ± 0.9442.00 ± 0.20
Fixed (p=80p=80)27.26 ± 1.4458.15 ± 1.3430.90 ± 0.7939.93 ± 0.42
Random pp, then generate content18.29 ± 3.5959.16 ± 1.0628.98 ± 1.5420.93 ± 1.94
AnchorDLM (learned position)68.94 ± 1.3767.73 ± 0.4939.20 ± 0.8344.00 ± 0.40

According to the experiment results, fixing the anchor position lowered performance across all evaluated metrics. This supports adapting position to each query, since an anchor’s effectiveness depends on where its content appears in the response.

Moreover, table 3 clarifies the roles of the two training stages. SFT teaches the generator to produce valid position–content pairs using supervised anchors alone. Preference optimization then provides feedback from downstream DLM outputs, allowing the generator to favor anchors that lead to more accurate completions.

Reviewer_7xsoOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors aim to explore a timely topic of interest: controllability in diffusion language models. To do so, they embrace the concept of anchors. Anchors initialize the text canvas with a text sequence generated by a specialized, light weight model. Then, text is generated as normal with a diffusion language model. The presence of the anchor, however, impacts subsequent generation output from the model. The authors connect anchors theoretically to the discrete flow matching framework and demonstrate convincing empirical evidence across a variety of models, benchmarks, metrics, and baselines.

Critical strengths
  • Controllability in DLMs is an important topic of interest
  • Strong experimental results and thorough benchmarks
  • Clear explanation of "anchoring" and how they perform it with an anchor generator
Critical weaknesses

None

Questions for authors
  • Are there any relevant prompt-based methods that take a different approach to control in diffusion language models?
  • This method works in masked diffusion language models such as Llada 8B. However, recent interest has extended to uniform and continuous diffusion language models as well. Crucially, these newer diffusion language models allow for a token to change across timesteps. Can you please explain how your method would perform in such a setting? Can it be adopted to still work with these models?
2 comments

We thank the reviewer for highlighting the distinction between prompt-based and anchor-based control and for asking how anchors behave when tokens can change during denoising.

We hope these clarifications and additional results address your concerns, and we would be grateful if you would consider raising your score. We are happy to answer any further questions.

Reply to Question 1:

Yes, but the related approach of Mirbeygi and Beigy [1] differs from ours in the diffusion process. Their model uses cross-attention to condition the denoising of continuous text embeddings on a structured prompt. Our Prompt baseline prepends a natural-language instruction specifying the target style to the input of a frozen masked diffusion language model that denoises discrete tokens.

Reply to Question 2:

Our main experiments use full-sequence masked diffusion with fixed anchor tokens to isolate the effect of anchor placement. The same conditioning principle could extend to other diffusion processes. In uniform discrete diffusion, the sampler could keep the anchor span fixed while allowing other tokens to change. In a continuous model, the reverse process could condition on anchor embeddings corrupted to the noise level of each timestep, analogous to diffusion inpainting [2, 3].

To test compatibility with token revision directly, we evaluated AnchorDLM on LLaDA-8B-Base with two remasking variants following Zhai et al. [4]. The first permits revision only outside the anchor span. The second also permits anchor tokens to change as additional context becomes available. The first four columns report control G-mean, followed by GSM8K accuracy and HumanEval pass@1.

AnchorDLM decoding variantGSM8K-fmtGSM8K-bwdHumanEval-cmtHumanEval-strGSM8K Acc.HumanEval pass@1
Original decoder69.3371.0843.8950.4167.1737.20
Remask non-anchor tokens only69.8570.7445.0650.8867.6339.02
Allow anchor tokens to be remasked68.2671.5344.0251.1268.2339.63

Allowing anchor revision increased GSM8K accuracy by 1.06 percentage points and HumanEval pass@1 by 2.43 points relative to the original anchored decoder. Three of the four control scores also increased, while GSM8K-fmt decreased by 1.07 points. These results show that AnchorDLM can operate with revisable tokens.

References

[1] Mirbeygi and Beigy. Prompt Guided Diffusion for Controllable Text Generation. WNUT, 2025.

[2] Song et al. Score-Based Generative Modeling through Stochastic Differential Equations. ICLR, 2021.

[3] Lugmayr et al. RePaint: Inpainting Using Denoising Diffusion Probabilistic Models. CVPR, 2022.

[4] Zhai et al. CORE: Context-Robust Remasking for Diffusion Language Models. ICML, 2026.

I thank the authors for their response. The remasking experiments are a useful first step toward showing that anchors work with revisable tokens, though they are still run within masked diffusion; testing on a uniform or continuous DLM would make the claim more convincing. My questions have been addressed, and I maintain my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.