acceptodds
Under review as a conference paper at ICLR 2027

Breaking the Factorization Barrier in Diffusion Language Models with External Memory

Abstract

Masked diffusion language models generate tokens in parallel, but factorized reverse steps cannot represent dependencies among simultaneously decoded tokens. At a fixed conditioning state, the KL divergence from the true joint conditional to the product of its exact marginals equals the block's total correlation. We introduce R-Joint, a training-free decoding framework that injects joint token structure into frozen diffusion language models through an editable external memory. Retrieved clean text spans define an empirical joint distribution over masked token blocks. We combine this distribution with the model's factorized proposal through a closed-form KL barycenter, with exact normalization using the sparse retrieval support. A gate tuned on validation text selects blocks for joint commitment. Experiments on LLaDA-8B and Dream-7B across Wikipedia, Python code, legal opinions, and synthetic entity corpora show consistent improvements in block exact match and likelihood at block length two. R-Joint improves exact match by up to percentage points on global real-world corpora and by up to points with repository-local memory. On Dream-7B, gains reach points on synthetic entity spans and points with denser memory. Matched ablations at the same block length show that preserving joint structure contributes beyond token-wise retrieval. At identical commit decisions, replacing the fused distribution with the product of its marginals reduces block exact match by to points. In iterative restoration of partially masked windows, joint commitment improves token accuracy with matching memory, even against a base decoder given more steps, and complements retrieved context. These results highlight the representational limits of factorized reverse steps and show that editable external memory can supply joint structure without retraining the model.

Primary area:
generative models
Keywords:
Masked diffusion language models, Retrieval-augmented language modeling, Joint token modeling, External memory, Parallel decoding

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_HrKvOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces R-Joint, a training-free method that combines a frozen diffusion language model's token predictions with joint distributions over retrieved token tuples. The method supports efficient normalization and gated block commitments. Experiments with LLaDA and Dream evaluate block prediction and iterative masked-text restoration across natural text, code, legal text, and synthetic entities.

Critical strengths
  1. R-Joint provides a tractable way to introduce joint token dependence without retraining the backbone. Sparse retrieved support enables exact normalization without enumerating the full vocabulary product space, and joint commitments integrate into decoding without additional backbone passes.
  2. The empirical controls help isolate the contribution of joint dependence.
Critical weaknesses

The joint-specific benefit in natural-text iterative decoding is not yet statistically well established.

Questions for authors

Can you provide a matched R-Joint-versus-Marg/B decoder comparison for legal text, code, or repository-local memory, with intervals clustered by the relevant document or repository?

1 comment

Thank you for the positive assessment and for the precise question.

The joint-specific benefit in natural-text iterative decoding is not yet statistically well established.

On Wikipedia the in-decoder margin of the joint tuple over Marg and B is +0.10 to +0.15 pp, the window-level intervals exclude zero, and the article-level intervals (59 articles) do not (Section 4.5, Appendix G).

Can you provide a matched R-Joint-versus-Marg/B decoder comparison for legal text, code, or repository-local memory, with intervals clustered by the relevant document or repository?

1. Block level, clustered by document (Appendix F, LLaDA). At b = 2, with commit decisions held fixed, the joint-minus-marginal interval is [+0.48, +0.95] pp on legal text (249 opinions) and [+0.30, +0.56] pp on code (226 files). Both exclude zero. On Wikipedia it is [−0.04, +0.55].

2. Decoder with repository-local memory (new experiment). We evaluated generation from fully masked answers on frozen LLaDA-8B-Base and Dream-v0-Base-7B, using plain prompts (no chat template) and temperature 0. Within each model–task pair, R-Joint, Marg and B share the base prompt, answer length, scorer, memory and host decoder; each runs its own decoding trajectory.

Tasks and memory. RepoEval-API (Zhang et al., 2023) has 1,600 test examples, prompted with the code preceding the target call in the same file, with 96 answer tokens and the official EM scorer. Memory contains other files of the same repository, excluding the query file, its near-duplicates and calibration files under the submission's deduplication rule. NQ-open has 3,610 test questions, eight fixed training demonstrations and 32 answer tokens; scoring uses normalized EM of the first output line against any reference answer. Its memory contains training question–answer pairs excluding the demonstrations; 3,000 development examples are used for calibration. Training records whose normalized question equals a test question are removed.

Each memory record is written in the task template, with its answer masked from a given position onward and clean blocks stored at the first masked position. No test-set answer annotations are used for key construction, prompts or calibration.

Shared decoder and controls. The host is an uncached confidence-threshold decoder: within 32-token blocks, each pass commits masked positions with confidence at least 0.9, and at least one. R-Joint retains the submission's fusion, gate (5) and calibration grid. Its 55% reveal threshold is counted over the last up to 512 positions of the prompt and answer. Each pass commits at most three non-overlapping blocks of length 2–4 that pass the gate; these are added to the host's selected positions, with joint values used at overlaps. B is the token-wise retrieval control of Table 5 and is calibrated separately. Marg commits the product of the fused marginals with R-Joint's parameters.

RepoEval-API EM is reported in %:

MethodLLaDADream
B: token-wise retrieval22.1929.31
Marg: product of fused marginals22.1329.13
R-Joint22.4429.56

R-Joint leads by 0.25 to 0.43 pp. On NQ-open the same comparison gives +0.36 pp over B and +0.44 / +0.70 pp over Marg (LLaDA / Dream). These generation results are reported as point estimates without document- or repository-clustered intervals.

Reference: Zhang et al. (2023). RepoCoder: Repository-level code completion through iterative retrieval and generation. EMNLP.

Thank you again for the helpful question. We would be happy to address any further questions or concerns.

Reviewer_CCVnOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes R-Joint, a training-free decoding framework that addresses the factorization limitation of masked diffusion language models (MDMs), where independently predicted tokens cannot capture dependencies within simultaneously decoded blocks. R-Joint constructs an editable external memory of clean text spans and retrieves joint token distributions using hidden-state representations from a frozen MDM. It then combines the retrieved joint distribution with the model's factorized predictions through a closed-form KL barycenter and uses a calibrated gate to determine when to commit entire token blocks. The authors evaluate R-Joint on LLaDA-8B and Dream-7B across Wikipedia, Python code, legal text, and synthetic entity datasets, comparing it against token-wise retrieval, alternative fusion methods, and standard iterative decoding. Results show consistent improvements in block exact match and likelihood, with larger gains on synthetic entities and repository-specific data. Ablation studies demonstrate that preserving joint token dependencies provides additional benefits beyond token-wise retrieval, while iterative decoding experiments show improvements in token accuracy without modifying model parameters.

Critical strengths
  1. Timely and Important Research Problem.

The paper addresses the factorization limitation of parallel decoding, which is a timely and important problem for MDMs. Modeling dependencies among simultaneously generated tokens is an interesting research direction with clear relevance to improving parallel generation.

  1. Clear Mathematical Formulation.

The paper provides a clear mathematical characterization of the factorization gap using total correlation. In particular, decomposing the KL divergence into a dependency gap and a marginal prediction gap provides a useful theoretical motivation for the proposed method. The closed-form fusion formulation also makes the overall approach mathematically well-grounded.

  1. Systematic Experiments and Ablation Studies.

The authors evaluate R-Joint across two MDM families and several different corpora. In particular, the controlled comparison between joint distributions and products of their marginals, using identical retrieval results and commitment decisions, is well-designed. This helps distinguish the benefits of joint dependency modeling from those of retrieval itself.

Critical weaknesses

Major Weaknesses

  1. Limited Evaluation in Standard Text Generation Settings.

The end-to-end experiments focus on partially masked text restoration rather than standard text generation from fully masked answer sequences. Since partially masked inputs provide ground-truth context that may facilitate retrieval, it remains unclear whether R-Joint would achieve similar improvements in general generation settings. Although the authors acknowledge this limitation, evaluating the method in a standard generation setting would substantially strengthen its practical relevance.

  1. Dependence on External Memory Quality and Domain Alignment.

The effectiveness of R-Joint appears to depend substantially on how well the external memory matches the target corpus. The authors already show that memory from unrelated repositories provides almost no improvement. However, a more systematic quantitative analysis of this dependency would strengthen the paper. For example, measuring how retrieval hit rate and downstream performance change with varying degrees of domain mismatch would help clarify the conditions under which R-Joint remains effective.

Minor Weaknesses

  1. Memory Scalability and Cost-Effectiveness.

The authors provide memory construction costs and inference overhead measurements. However, the trade-off between additional memory construction and retrieval costs and the resulting performance improvements at larger corpus scales could be analyzed more thoroughly. This would help clarify the scalability of the proposed approach in practical deployments.

  1. Limited Block Length.

R-Joint considers only contiguous blocks of length 2–4, and the performance gains generally decrease as block length increases. It remains unclear whether the proposed approach can effectively scale to longer blocks or capture more complex dependencies between non-contiguous tokens.

  1. Presentation of End-to-End Results.

Section 4.5 presents many experimental results and numerical comparisons in lengthy paragraphs, making it somewhat difficult to follow the main findings. Moving the relevant tables or figures closer to the corresponding discussion and organizing the results more clearly would improve readability.

  1. Limited Practical Gains from Joint Dependency Modeling.

Although the controlled ablations demonstrate the benefits of preserving joint token dependencies, the improvements are relatively modest on natural corpora. For example, the joint distribution improves block exact match over its marginal counterpart by only 0.27 percentage points on LLaDA Wikipedia, with a confidence interval that includes zero. Similarly, the end-to-end improvements attributable specifically to joint modeling are relatively small (+0.13 to +0.44 pp). These results raise questions about the practical significance of joint dependency modeling relative to the additional complexity introduced by the proposed framework.

Questions for authors
  1. Memory Scalability. How would the memory construction and nearest-neighbor retrieval costs scale to substantially larger datastores? Could the authors provide further analysis of the trade-off between memory size, computational cost, and performance improvement?
  2. Retrieval Quality and Domain Dependence. The paper already measures coverage, hit rate, and retrieval efficiency. Could the authors further investigate how retrieval quality and downstream performance vary with different degrees of domain mismatch between the datastore and the target corpus?
  3. Generalization to Standard Generation. Have the authors considered evaluating R-Joint in a standard text generation setting, where generation starts from a fully masked answer conditioned on a prompt? It would be interesting to understand whether the improvements observed in partially masked restoration also transfer to this setting.
5 comments

Thank you for the careful review, and for the kind words about the formulation and the matched joint-versus-marginal control.

Major 1 and Q3: standard generation

Since partially masked inputs provide ground-truth context that may facilitate retrieval, it remains unclear whether R-Joint would achieve similar improvements in general generation settings.

Thank you for raising this question. To address it, we have added comparative experiments evaluating R-Joint against Fast-dLLM and DAPD, alongside the baselines and controls in the submission. These experiments cover three standard generation benchmarks: GSM8K for mathematical reasoning, NQ-open for open-domain question answering, and RepoEval-API for repository-level API-call completion.

Essential setup. We evaluate frozen LLaDA-8B-Base and Dream-v0-Base-7B on GSM8K (1,319 test questions), NQ-open (3,610 test questions) and RepoEval-API (1,600 examples). All answer positions start masked; we use plain prompts and temperature 0. Within each model–task pair, methods share the base prompt, answer length and scorer. GSM8K reports final-answer accuracy; NQ-open and RepoEval-API report EM.

Methods and memory. We compare the submission's controls and R-Joint variants with Fast-dLLM and DAPD, using the external methods' released code. Rows 1–7 share an uncached confidence-threshold host (threshold 0.9, 32-token blocks). Rows 2–7 use the same permitted memory: training question–answer pairs for GSM8K and NQ-open, and other files of the same repository for RepoEval-API. The base host and the two external decoders use no memory. No test-set answer annotations are used for memory construction, prompts or calibration. Each row runs its own decoding trajectory.

Table R1. Accuracy on GSM8K and exact match (EM) on NQ-open and RepoEval-API, in %.

MethodLLaDA GSM8KLLaDA NQLLaDA RepoEvalDream GSM8KDream NQDream RepoEval
1. Base decoder (host)68.8424.4321.3873.0128.2527.63
2. B: token-wise retrieval68.4626.4522.1972.8630.7829.31
3. Mix: arithmetic fusion68.4626.3722.1972.5530.1729.13
4. Marg: fused marginals68.5426.3722.1372.8630.4429.13
5. Retrieved prefix69.6031.7725.4473.9236.6832.56
6. R-Joint68.6926.8122.4473.0131.1429.56
7. Retrieved prefix + R-Joint69.6032.2226.0074.0037.0933.00
8. Fast-dLLM67.1723.9319.6372.7127.6227.06
9. DAPD64.0622.4119.5069.2926.5426.31

What we observe. Differences are given as LLaDA / Dream, unless otherwise stated.

  1. Gains on NQ-open and RepoEval-API. R-Joint improves its host by +2.4 / +2.9 pp on NQ-open and by +1.1 / +1.9 pp on RepoEval-API.

  2. No improvement on GSM8K. R-Joint scores 68.69 against 68.84 on LLaDA (two questions fewer) and 73.01 against 73.01 on Dream.

  3. Comparison with the two decoders. On NQ-open, it achieves 26.81 / 31.14% EM, exceeding the best of the two decoders by 2.88 / 3.52 pp. On RepoEval-API, it achieves 22.44 / 29.56% EM, exceeding the best of the two by 2.81 / 2.50 pp.

  4. Joint values against shared-memory controls. Rows 2, 3 and 4 use the same memory and the same host decoder as R-Joint and differ in how the committed values are chosen; each row runs its own decode. R-Joint is the highest of the four in every column. On NQ-open and RepoEval-API it leads token-wise retrieval (B) by 0.25 to 0.36 pp and the product of its own marginals (Marg) by 0.31 to 0.70 pp.

  5. Improving the retrieved-prefix baseline with R-Joint. The retrieved-prefix baseline (row 5) retrieves text from the same external memory used by R-Joint and adds it to the host decoder's input as context. Row 7 keeps the same retrieved prefix and adds R-Joint's joint value rule during decoding. Compared with the retrieved-prefix baseline, adding R-Joint further improves EM by 0.4–0.6 pp on NQ-open and RepoEval-API across both backbones. The combination achieves the highest score, or ties for the highest, among all nine methods in each of the six columns. On these two tasks, R-Joint therefore improves answer accuracy even when the model already receives retrieved text as context.

Summary. In generation from fully masked answers, R-Joint improves on its host where the memory matches the task, is ahead of the shared-memory controls, and adds to the retrieved prefix. We will add Table R1 and its protocol to the revision.

The detailed experimental settings are provided in the next comment.

This comment provides the detailed experimental settings for the results reported in the preceding response to Major 1 and Q3. All row references below refer to Table R1 in that response.

Evaluation setup. All methods run on the frozen LLaDA-8B-Base and Dream-v0-Base-7B checkpoints with plain prompts (no chat template) and temperature 0. For each model and task, they share the base prompt, answer length and scorer.

  • GSM8K: 1,319 test questions, four fixed training demonstrations, 256 answer tokens; post-processing and flexible-extract exact-match scorer from lm-evaluation-harness.
  • NQ-open: 3,610 test questions, eight fixed training demonstrations, 32 answer tokens; normalized EM of the first output line against any reference answer.
  • RepoEval-API (Zhang et al., 2023): 1,600 test examples, prompted with the code preceding the target call in the same file, 96 answer tokens; official EM scorer.

Memory and calibration. GSM8K memory contains training question–solution pairs, excluding the demonstrations and 500 calibration examples. NQ-open memory contains training question–answer pairs, excluding the demonstrations; 3,000 development examples are used for calibration. Training records whose normalized question equals a test question are removed. RepoEval memory contains the other files of the same repository; the query file, its near-duplicates and the calibration files are excluded, using the submission's deduplication rule.

Each memory record is written in the task template. Its answer is masked from a given position onward, rather than at random as in Section 3.1, and clean blocks are stored at the first masked position. No test-set answer annotations are used for key construction, prompts or calibration. Rows 2–7 share this memory; the external decoders use none. Retrieved prefix is the control that supplies the same memory through the model's input context.

Base decoder and memory rows (1–7). These rows use an uncached confidence-threshold decoder in place of the fixed-step decoder of Section 4.5, so that they commit adaptively like the external decoders: within 32-token blocks, each pass commits the masked positions with confidence at least 0.9, and at least one. R-Joint keeps the submission's fusion, gate (5) and calibration grid. We retain the submission's 55% reveal threshold, counted over the last up to 512 positions of the prompt and answer, excluding retrieved-prefix tokens. Each pass commits at most three non-overlapping blocks of length 2–4 that pass the gate. A threshold decoder has no per-step budget, so these blocks are added to the positions it selects rather than debited as in Appendix G; where the two overlap, the joint values are used.

B, Mix and Marg are the controls of Table 5: B and Mix are calibrated separately, and Marg commits the product of the fused marginals with R-Joint's parameters. Prefix and Prefix + R-Joint share the context retrieved at the first pass (at most 512 tokens), attended from the second pass on, as in the submission. Each row runs its own decoding trajectory.

External decoders. Each is run from its released code. Where the authors give no setting for a task, we use that of the closest task.

  • Fast-dLLM (v1, a9b81e4): 32-token blocks, confidence threshold 0.9, prefix caching.
  • DAPD (05727b0): staged single-block decoder. For LLaDA, (tau_min, tau_max) is (0.01, 0.05) on GSM8K and NQ-open (the latter from the authors' TriviaQA analysis) and (0.01, 0.15) on RepoEval (their HumanEval/MBPP setting); for Dream, (0.005, 0.05) on all three tasks.

Reference: Zhang et al. (2023). RepoCoder: Repository-level code completion through iterative retrieval and generation. EMNLP.

Major 2 and Q2: memory quality and domain alignment

However, a more systematic quantitative analysis of this dependency would strengthen the paper.

Thank you for this suggestion. The submission includes several controlled comparisons of matched and mismatched memory:

  • same repository against other repositories: +1.14 / +2.99 pp against +0.04 / −0.03 pp block EM (Table 4, LLaDA / Dream);
  • same templates, different entities: about half of the gain is lost with the standard synthetic memory (Section 4.3);
  • wrong domain under a frozen gate (Wikipedia ↔ code, LLaDA): −0.20 and −0.01 pp (Appendix E);
  • out-of-distribution text added to an in-domain store: no consistent gain (Table 8).

We will discuss these results together in the revision so that the dependence is easier to see.

Minor points

How would the memory construction and nearest-neighbor retrieval costs scale to substantially larger datastores?

Construction is one frozen forward pass per window (about 700 windows per minute on one GPU), so it grows with the number of windows. Keys take 2.1 GB per 4.1M entries. Search is exact, so its cost grows with the datastore; the paper estimates 3.7 ms per call at that size (Table 21). On the benefit side, corpus coverage rises with corpus size (Table 9), while adding out-of-distribution text gave no consistent gain (Table 8).

It remains unclear whether the proposed approach can effectively scale to longer blocks or capture more complex dependencies between non-contiguous tokens.

Gains are generally smaller at b = 4, and not every cell is positive (for example Dream on Wikipedia at b = 3, Table 6). Longer tuples are harder to match exactly, and fewer of them occur in the reference corpus at all: about 40% of query 4-grams against 80 to 89% of bigrams (Table 9). The fusion rule (Propositions 2 and 3) does not require contiguous positions, while the current datastore construction and candidate selection use contiguous blocks.

Section 4.5 presents many experimental results and numerical comparisons in lengthy paragraphs, making it somewhat difficult to follow the main findings.

Agreed. We will reorganise Section 4.5 around a summary of the main findings, bring the key numbers closer to the text that discusses them, and keep each paragraph to one result.

These results raise questions about the practical significance of joint dependency modeling relative to the additional complexity introduced by the proposed framework.

At block length two, R-Joint improves EM over Marg by +0.71 pp on legal text and +0.42 pp on code with LLaDA (Table 5); on dense entity blocks the margin reaches 2.1–2.7 pp (Figure 2). At a fixed decoding state, the joint rule can use the same retrieved neighbours as token-wise voting, and its normalisation costs O(kb) (Proposition 3). In Table 5 it gives the lowest NLL and a higher EM than its own marginals in all thirteen cells. Table R1 agrees in direction: R-Joint is above B, Mix and Marg in every column.

Thank you again for your positive feedback and constructive suggestions. We would be happy to address any further questions or concerns.

Reviewer_WybjOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper addresses the factorization error arising from independently predicting token values during parallel decoding in masked diffusion language models. The authors propose R-Joint, a training-free inference method that retrieves joint token tuples from an external datastore and fuses their empirical distribution with the model’s factorized predictions. Experiments use LLaDA-8B-Base and Dream-v0-Base-7B on Wikipedia, Python code, legal opinions, and synthetic entity corpora. The paper reports improvements in block-level exact match and negative log-likelihood. In iterative masked-text restoration with matching memory, R-Joint also achieves higher token accuracy with 16 decoding steps than the base decoder with 22 steps.

Critical strengths

The idea of incorporating external memory to accelerate inference in masked diffusion models is novel and interesting.

Critical weaknesses

1. Missing comparisons with existing training-free decoding methods.

R-Joint is not empirically compared against existing training-free methods for accelerating masked diffusion language models, such as Fast-dLLM [1], DAPD [2], DepCap [3], and TACG [4]. Although these methods differ in how they coordinate parallel decoding, they are relevant baselines for assessing the quality–efficiency trade-off.

R-Joint also requires datastore storage, offline datastore construction, corpus-specific calibration, and online retrieval and fusion. The paper reports end-to-end latency including retrieval, but comparisons against the base decoder alone do not establish whether R-Joint offers a favorable trade-off relative to existing acceleration methods. Comparisons under matched backbone and evaluation settings, reporting both task quality and wall-clock inference cost, are needed. Offline construction and calibration costs, as well as datastore memory requirements, should be accounted for separately when assessing practical deployment costs.

2. Limited evidence for downstream generation performance.

Prior work on accelerated LLaDA and Dream decoding evaluates established downstream benchmarks, including GSM8K, MATH or MATH500, HumanEval, and MBPP [1–4], as well as IFEval [2]. In contrast, this paper evaluates masked-text restoration on Wikipedia, Python code, legal opinions, and synthetic entity corpora.

Block-level exact match and negative log-likelihood are useful for testing the proposed mechanism, and the paper additionally reports end-to-end restoration metrics. However, these experiments retain unmasked ground-truth text as context, which differs from generating a complete answer from a prompt. Consequently, the reported improvements do not establish whether R-Joint improves reasoning accuracy, functional code correctness, or instruction following under reduced decoding budgets.

The authors acknowledge this distinction in the limitations, but it remains a substantial gap in the evidence for practical generation benefits. Evaluation on established downstream generation benchmarks, with clearly specified external-memory construction and contamination controls, is needed to substantiate broader claims about generation quality and efficiency.

[1] Wu et al., Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding, 2025.
[2] Kim et al., DAPD: Dependency-Aware Parallel Decoding via Attention for Diffusion LLMs, 2026.
[3] Xia et al., DepCap: Adaptive Block-Wise Parallel Decoding for Efficient Diffusion LM Inference, 2026.
[4] Wang et al., TACG: Trajectory-Aware Commit Gating for Diffusion Language Model Decoding, 2026.

Questions for authors

Please check the above weaknesses.

4 comments

Thank you for your careful review and constructive suggestions. Our paper studies restoration and infilling in existing text or code, rather than complete-answer generation from a prompt. R-Joint recovers missing spans using observed context and editable memory. Infilling is a fundamental task supporting writing assistance and document completion (Donahue et al., 2020; Huang et al., 2020), and recovery of variable names and types in code (Raychev et al., 2015; Chen et al., 2022). Ithaca raised historians' restoration accuracy from 25% to 72% (Assael et al., 2022). Infilling is native to masked diffusion models: SEDD evaluates it, and Dream supports it without specialised training (Lou et al., 2024; Ye et al., 2025).

We will add standard generation experiments. We have also noted the baselines you mentioned; our submission already cites and discusses Fast-dLLM, DAPD, DepCap and TACG. Below, we present the experiments we have completed comparing R-Joint with Fast-dLLM and DAPD on standard generation tasks, together with the results and analysis.

Essential setup. We evaluate frozen LLaDA-8B-Base and Dream-v0-Base-7B on GSM8K (1,319 test questions), NQ-open (3,610 test questions) and RepoEval-API (1,600 examples). All answer positions start masked; we use plain prompts and temperature 0. Within each model–task pair, methods share the base prompt, answer length and scorer.

Methods and memory. Fast-dLLM and DAPD use released code. Rows 1–7 share an uncached confidence-threshold host (threshold 0.9, 32-token blocks). Rows 2–7 share training question–answer memory for GSM8K and NQ-open, or other files of the same repository for RepoEval-API. The base host and external decoders use no memory. No test-set answer annotations enter memory construction, prompts or calibration. Each row runs its own trajectory.

Table R1. Final-answer accuracy on GSM8K and exact match (EM) on NQ-open and RepoEval-API, in %.

MethodLLaDA GSM8KLLaDA NQLLaDA RepoEvalDream GSM8KDream NQDream RepoEval
1. Base decoder (host)68.8424.4321.3873.0128.2527.63
2. B: token-wise retrieval68.4626.4522.1972.8630.7829.31
3. Mix: arithmetic fusion68.4626.3722.1972.5530.1729.13
4. Marg: fused marginals68.5426.3722.1372.8630.4429.13
5. Retrieved prefix69.6031.7725.4473.9236.6832.56
6. R-Joint68.6926.8122.4473.0131.1429.56
7. Retrieved prefix + R-Joint69.6032.2226.0074.0037.0933.00
8. Fast-dLLM67.1723.9319.6372.7127.6227.06
9. DAPD64.0622.4119.5069.2926.5426.31

What we observe. Differences are given as LLaDA / Dream, unless otherwise stated.

  1. Gains on NQ-open and RepoEval-API. R-Joint improves its host by +2.4 / +2.9 pp on NQ-open and by +1.1 / +1.9 pp on RepoEval-API.

  2. No improvement on GSM8K. R-Joint scores 68.69 against 68.84 on LLaDA (two questions fewer) and 73.01 against 73.01 on Dream.

  3. Comparison with the two decoders. On NQ-open, it achieves 26.81 / 31.14% EM, exceeding the best of the two decoders by 2.88 / 3.52 pp. On RepoEval-API, it achieves 22.44 / 29.56% EM, exceeding the best of the two by 2.81 / 2.50 pp.

  4. Joint values against shared-memory controls. With the same memory and host, R-Joint leads B, Mix and Marg in all six columns. On NQ-open and RepoEval-API it exceeds B by 0.25–0.36 pp and Marg by 0.31–0.70 pp.

  5. Improving the retrieved-prefix baseline with R-Joint. Row 5 supplies retrieved text from the same memory as context; row 7 keeps this prefix and adds R-Joint's joint value rule. Adding R-Joint improves EM over row 5 by 0.4–0.6 pp on NQ-open and RepoEval-API across both backbones. The combination ranks highest or joint-highest among all nine methods in all six columns.

Settings follow in the next comment.

References

  • Assael et al. (2022). Restoring and attributing ancient texts using deep neural networks. Nature 603, 280–283.
  • Chen et al. (2022). Augmenting decompiler output with learned variable names and types. USENIX Security.
  • Donahue et al. (2020). Enabling language models to fill in the blanks. ACL.
  • Huang et al. (2020). INSET: Sentence infilling with inter-sentential Transformer. ACL.
  • Lou et al. (2024). Discrete diffusion modeling by estimating the ratios of the data distribution. ICML.
  • Raychev et al. (2015). Predicting program properties from "Big Code". POPL.
  • Wang et al. (2026). TACG: Trajectory-Aware Commit Gating for Diffusion Language Model Decoding. arXiv:2607.03236.
  • Xia et al. (2026). DepCap: Adaptive Block-Wise Parallel Decoding for Efficient Diffusion LM Inference. arXiv:2604.15750.
  • Ye et al. (2025). Dream 7B: Diffusion large language models. arXiv:2508.15487.

We provide further details on the experimental setup for the additional comparisons reported in our response to W1 and W2. All row references below correspond to Table R1 in that response.

Evaluation setup. All methods run on the frozen LLaDA-8B-Base and Dream-v0-Base-7B checkpoints with plain prompts (no chat template) and temperature 0. For each model and task, they share the base prompt, answer length and scorer.

  • GSM8K: 1,319 test questions, four fixed training demonstrations, 256 answer tokens; post-processing and flexible-extract exact-match scorer from lm-evaluation-harness.
  • NQ-open: 3,610 test questions, eight fixed training demonstrations, 32 answer tokens; normalized EM of the first output line against any reference answer.
  • RepoEval-API (Zhang et al., 2023): 1,600 test examples, prompted with the code preceding the target call in the same file, 96 answer tokens; official EM scorer.

Memory and calibration. GSM8K memory contains training question–solution pairs, excluding the demonstrations and 500 calibration examples. NQ-open memory contains training question–answer pairs, excluding the demonstrations; 3,000 development examples are used for calibration. Training records whose normalized question equals a test question are removed. RepoEval memory contains the other files of the same repository; the query file, its near-duplicates and the calibration files are excluded, using the submission's deduplication rule.

Each memory record is written in the task template. Its answer is masked from a given position onward, rather than at random as in Section 3.1, and clean blocks are stored at the first masked position. No test-set answer annotations are used for key construction, prompts or calibration. Rows 2–7 share this memory; the external decoders use none. Retrieved prefix is the control that supplies the same memory through the model's input context.

Base decoder and memory rows (1–7). These rows use an uncached confidence-threshold decoder in place of the fixed-step decoder of Section 4.5, so that they commit adaptively like the external decoders: within 32-token blocks, each pass commits the masked positions with confidence at least 0.9, and at least one. R-Joint keeps the submission's fusion, gate (5) and calibration grid. We retain the submission's 55% reveal threshold, counted over the last up to 512 positions of the prompt and answer, excluding retrieved-prefix tokens. Each pass commits at most three non-overlapping blocks of length 2–4 that pass the gate. A threshold decoder has no per-step budget, so these blocks are added to the positions it selects rather than debited as in Appendix G; where the two overlap, the joint values are used.

B, Mix and Marg are the controls of Table 5: B and Mix are calibrated separately, and Marg commits the product of the fused marginals with R-Joint's parameters. Prefix and Prefix + R-Joint share the context retrieved at the first pass (at most 512 tokens), attended from the second pass on, as in the submission. Each row runs its own decoding trajectory.

External decoders. Each is run from its released code. Where the authors give no setting for a task, we use that of the closest task.

  • Fast-dLLM (v1, a9b81e4): 32-token blocks, confidence threshold 0.9, prefix caching.
  • DAPD (05727b0): staged single-block decoder. For LLaDA, (tau_min, tau_max) is (0.01, 0.05) on GSM8K and NQ-open (the latter from the authors' TriviaQA analysis) and (0.01, 0.15) on RepoEval (their HumanEval/MBPP setting); for Dream, (0.005, 0.05) on all three tasks.

Reference: Zhang et al. (2023). RepoCoder: Repository-level code completion through iterative retrieval and generation. EMNLP.

Thank you again for the suggestions. We hope the additional comparisons and the clarified task scope address your concerns.

Thank you for the detailed response and the additional experiments. However, the new results do not resolve my main concerns.

My main concern is the practical value of the proposed method. The response clarifies that R-Joint targets restoration and infilling. Nevertheless, I framed it in the context of acceleration for two reasons. First, the factorization error it targets arises from committing several tokens per step, which most related work studies as the obstacle to few-step generation [1, 2, 3]. Second, the paper evaluates R-Joint against a base decoder given more steps. Comparisons with parallel decoding methods such as DAPD, DepCap, and TACG under matched practical cost, such as latency or FLOPs, therefore remain relevant.

1. The gains come from copying memory, not from better generation. R-Joint commits retrieved token tuples, so it helps mainly when the answer is already in the memory. On NQ-open, the memory holds training QA pairs, and 63.6% of test answers also appear among the training answers [4]. The rebuttal removes only exact duplicate questions. On GSM8K, where the answer cannot be copied, R-Joint gives no gain. Most of the gain also comes from retrieval itself (LLaDA NQ-open: host 24.43, retrieved prefix 31.77, prefix + R-Joint 32.22), leaving 0.4–0.6 pp for joint decoding.

2. The practical benefit is not established. Table R1 compares accuracy only, and the methods differ in caching and memory. The R-Joint rows run on an uncached host, which alone already beats the cached Fast-dLLM in every column, and they use memory that the baselines do not. No inference cost is reported, although these baselines report wall-clock throughput or latency and R-Joint adds retrieval, fusion, and a retrieved prefix, which the paper reports costs 36–40% more than the base decoder. The method also needs a task-specific datastore and calibration, and its gain disappears when the memory does not match the target (other repositories, GSM8K). For this added cost, the gain from joint modeling is small, as other reviewers also noted.

I have also read the other reviews and the authors’ responses. The other reviewers remain positive overall but point to the same limitations: Reviewer CCVn notes that the gains are task-dependent, with none on GSM8K, and Reviewer HrKv notes that the joint-specific benefit is not statistically established. Unlike them, I consider these limitations decisive. R-Joint is presented as a remedy for the factorization error of parallel decoding, yet its gains depend on the answer already being in the memory and come at an unreported inference cost.

For these reasons, I maintain my score.

[1] Liu et al., Discrete Copula Diffusion, 2024.
[2] Shariatian et al., Latent-Augmented Discrete Diffusion Models, 2025.
[3] Kim & Sung, Tensor-Train Joint Modeling for Few-Step Discrete Diffusion, 2026.
[4] Lewis et al., Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets, 2021.

Thank you for your follow-up.

We do not believe the concerns raised by the reviewer justify the lowest possible rating of Clear Rejection (Score: 1), as none of them identifies a fundamental flaw in our methodology or invalidates our central technical contributions and experimental findings.

We would like to reiterate the scope and central contribution of our paper. R-Joint addresses the limitations of independent token value selection in masked diffusion language models for restoration and infilling with a frozen model and relevant external memory. We have demonstrated the existence and significance of this problem through theoretical analysis and controlled experiments. R-Joint is a joint value-selection rule, not a new decoding schedule or accelerator. We do not agree that its controlled gains can be dismissed as copying or that its practical value remains unsupported. A quality–latency ranking against accelerators does not replace controlled evaluation of joint value selection.

1. DepCap and TACG. We have already cited and discussed both methods in our original submission, explicitly explaining why they differ from our work and are not comparable. Furthermore, both remain arXiv preprints with no identified peer-reviewed publication records. The ICLR 2027 Reviewer Guidelines state that “the lack of such comparisons cannot be a basis for rejection.” Nevertheless, we conducted additional comparisons with the published Fast-dLLM (ICLR 2026) and DAPD (ICML 2026). We do not agree that the absence of direct comparisons with these non-peer-reviewed arXiv papers justifies a Clear Rejection.

2. Restoration accuracy and cost. We do not agree that practical benefit is unsupported. In Table 14, R-Joint at 16 steps takes about 75% of the base decoder's runtime at 22 steps while improving token accuracy by 0.26–0.42 pp across five natural-corpus settings and by 0.95 pp with same-repository memory. All six paired, window-level 95% bootstrap intervals lie above zero.

3. Copying versus joint selection. We strongly disagree that the gains are merely copying. The calibrated copy rule (Table 7) recovers only 22–56% of R-Joint's gain on Wikipedia and legal text and 23–30% on synthetic text. The Marg control (Table 5) fixes retrieved neighbours and commit decisions: replacing the joint distribution with its product of marginals lowers block EM by 0.27–2.69 pp across thirteen cells. Document-clustered intervals confirm the advantage on legal text [+0.48, +0.95] pp and code [+0.30, +0.56] pp (Appendix F). These controls isolate the benefit of joint value selection.

4. NQ-open and GSM8K. We do not agree that limited gains on these benchmarks invalidate our core contribution. In Table R1, R-Joint exceeds B, Mix, and Marg with the same memory and adds about 0.4 pp EM over the prefix on NQ-open for both backbones. We do not claim unseen-answer generalization or improved mathematical reasoning; GSM8K shows no gain. The controlled evidence in point 3 does not depend on these benchmarks.

5. Retrieved-prefix overhead. We do not agree that the 36–40% overhead is attributable to R-Joint itself: it applies to prefix configurations, while standalone R-Joint needs no prefix. In restoration, adding R-Joint to the prefix adds at most 3% runtime and improves token accuracy in all thirteen rows, with twelve positive window-level intervals (Table 14). At 31 steps, the base decoder is at least as slow as prefix + R-Joint but 0.44–2.06 pp less accurate on six rows (Table 18).

6. Caching and memory in Table R1. R-Joint uses the same uncached host as the base row and the same memory as B, Mix, and Marg. Table R1 compares quality, not equal-latency performance. We do not present restoration timings from Tables 14 and 18 as timings for Table R1.

7. Datastore construction and reuse. Appendix H reports about six GPU-hours to build the 244K-window Wikipedia store and 2.1 GB for keys. Offline costs can be amortized through reuse; Table 14 includes online retrieval and fusion. With fully masked names, edited memory improves name EM over stale memory by 6.03 pp [+4.69, +7.45] (Section 4.3). R-Joint targets settings with relevant reference material.

We sincerely ask the reviewer to reconsider our paper's scope and central contribution. The assessment of practical value should account for the fixed-commit controls and the measured restoration costs, not only for the absence of an equal-cost ranking against accelerators. We therefore do not believe the remaining concerns justify a lowest possible socre Clear Rejection, as they do not identify a fundamental flaw in the proposed method or invalidate its central findings.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank the reviewers for their careful reading and constructive suggestions. The reviewers' main concern is the lack of baseline comparisons on standard generation tasks. We provide the following clarifications and additional experiments.

Clarifications

1. Scope of the paper. R-Joint is a decoding rule for filling masked spans when two things are available: visible text around the gap, and a reference corpus that may contain the missing span. We do not claim gains in reasoning, functional code correctness or instruction following. This scope is stated from the abstract onward, not only in the limitations section.

2. Our contribution. We introduce R-Joint, a training-free decoding rule that jointly selects masked token values by fusing a frozen diffusion model's predictions with retrieved block distributions. The fusion admits a closed-form solution, and the external memory can be updated without retraining.

3. Filling gaps in existing text is a task in its own right.

The infilling problem studied in our paper is fundamental and has practical value in its own right, including applications in writing assistance, text restoration and code recovery. The following studies illustrate these applications and the role of infilling in diffusion language models.

  • Donahue et al. (2020) and Huang et al. (2020) directly study text and sentence infilling, motivated by writing assistance and document completion.
  • Assael et al. (2022) fill textual gaps in ancient inscriptions to support historical research.
  • Raychev et al. (2015) and Chen et al. (2022) fill in missing information in existing code, recovering variable names and types to support code understanding.
  • Lou et al. (2024) evaluate text infilling with SEDD, and Ye et al. (2025) demonstrate this capability in Dream.

4. What the submission shows within this scope.

The experiments in our submission clearly demonstrate R-Joint’s effectiveness on its target restoration and infilling tasks.

Additional experiments

5. Generation from fully masked answers.

To address this concern, we added performance comparisons on standard generation tasks: R-Joint versus Fast-dLLM and DAPD on GSM8K, NQ-open and RepoEval-API. We summarise the findings below; full settings and results appear in our replies to Wybj (W1 and W2) and CCVn (Major 1 and Q3). Row and column references refer to Table R1 in those replies.

What we observe. Differences are given as LLaDA / Dream, unless otherwise stated.

  1. Gains on NQ-open and RepoEval-API. R-Joint improves its host by +2.4 / +2.9 pp on NQ-open and +1.1 / +1.9 pp on RepoEval-API; both tasks involve coherent multi-token answers (names, dates or API calls) and memory from the same source as the test items.

  2. No improvement on GSM8K. R-Joint shows no improvement over its host on GSM8K. Its memory contains solutions to other problems, consistent with the paper's observation that memory without matching content brings no gain.

  3. Comparison with the two decoders. R-Joint exceeds the best of the two external decoders by 2.88 / 3.52 pp on NQ-open and 2.81 / 2.50 pp on RepoEval-API. The external decoders differ from the host by −2.0 to −0.5 pp on NQ-open and −1.9 to −0.6 pp on RepoEval-API.

  4. Joint values against shared-memory controls. With the same memory and host decoder, R-Joint scores higher than B, Mix and Marg in all six columns. On NQ-open and RepoEval-API, it exceeds B by 0.25–0.36 pp and Marg by 0.31–0.70 pp.

  5. Improving the retrieved-prefix baseline with R-Joint. Adding R-Joint to the same retrieved prefix further improves EM by 0.4–0.6 pp on NQ-open and RepoEval-API across both backbones. The combination achieves the highest or joint-highest score among all nine methods in all six columns.

Summary. R-Joint's benefits extend beyond restoration and infilling to standard downstream generation: on NQ-open and RepoEval-API, it outperforms its host and shared-memory controls and further improves the retrieved-prefix baseline. Detailed results are provided in our responses to the individual reviewers below. We will incorporate these additional experiments and their findings into the revised manuscript.

References

  • Assael et al. (2022). Restoring and attributing ancient texts using deep neural networks. Nature 603, 280–283.
  • Chen et al. (2022). Augmenting decompiler output with learned variable names and types. USENIX Security.
  • Donahue et al. (2020). Enabling language models to fill in the blanks. ACL.
  • Huang et al. (2020). INSET: Sentence infilling with inter-sentential Transformer. ACL.
  • Lou et al. (2024). Discrete diffusion modeling by estimating the ratios of the data distribution. ICML.
  • Raychev et al. (2015). Predicting program properties from "Big Code". POPL.
  • Ye et al. (2025). Dream 7B: Diffusion large language models. arXiv:2508.15487.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.