1. Missing comparisons with existing training-free decoding methods.
R-Joint is not empirically compared against existing training-free methods for accelerating masked diffusion language models, such as Fast-dLLM [1], DAPD [2], DepCap [3], and TACG [4]. Although these methods differ in how they coordinate parallel decoding, they are relevant baselines for assessing the quality–efficiency trade-off.
R-Joint also requires datastore storage, offline datastore construction, corpus-specific calibration, and online retrieval and fusion. The paper reports end-to-end latency including retrieval, but comparisons against the base decoder alone do not establish whether R-Joint offers a favorable trade-off relative to existing acceleration methods. Comparisons under matched backbone and evaluation settings, reporting both task quality and wall-clock inference cost, are needed. Offline construction and calibration costs, as well as datastore memory requirements, should be accounted for separately when assessing practical deployment costs.
2. Limited evidence for downstream generation performance.
Prior work on accelerated LLaDA and Dream decoding evaluates established downstream benchmarks, including GSM8K, MATH or MATH500, HumanEval, and MBPP [1–4], as well as IFEval [2]. In contrast, this paper evaluates masked-text restoration on Wikipedia, Python code, legal opinions, and synthetic entity corpora.
Block-level exact match and negative log-likelihood are useful for testing the proposed mechanism, and the paper additionally reports end-to-end restoration metrics. However, these experiments retain unmasked ground-truth text as context, which differs from generating a complete answer from a prompt. Consequently, the reported improvements do not establish whether R-Joint improves reasoning accuracy, functional code correctness, or instruction following under reduced decoding budgets.
The authors acknowledge this distinction in the limitations, but it remains a substantial gap in the evidence for practical generation benefits. Evaluation on established downstream generation benchmarks, with clearly specified external-memory construction and contamination controls, is needed to substantiate broader claims about generation quality and efficiency.
[1] Wu et al., Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding, 2025.
[2] Kim et al., DAPD: Dependency-Aware Parallel Decoding via Attention for Diffusion LLMs, 2026.
[3] Xia et al., DepCap: Adaptive Block-Wise Parallel Decoding for Efficient Diffusion LM Inference, 2026.
[4] Wang et al., TACG: Trajectory-Aware Commit Gating for Diffusion Language Model Decoding, 2026.
Thank you for the positive assessment and for the precise question.
On Wikipedia the in-decoder margin of the joint tuple over Marg and B is +0.10 to +0.15 pp, the window-level intervals exclude zero, and the article-level intervals (59 articles) do not (Section 4.5, Appendix G).
1. Block level, clustered by document (Appendix F, LLaDA). At b = 2, with commit decisions held fixed, the joint-minus-marginal interval is [+0.48, +0.95] pp on legal text (249 opinions) and [+0.30, +0.56] pp on code (226 files). Both exclude zero. On Wikipedia it is [−0.04, +0.55].
2. Decoder with repository-local memory (new experiment). We evaluated generation from fully masked answers on frozen LLaDA-8B-Base and Dream-v0-Base-7B, using plain prompts (no chat template) and temperature 0. Within each model–task pair, R-Joint, Marg and B share the base prompt, answer length, scorer, memory and host decoder; each runs its own decoding trajectory.
Tasks and memory. RepoEval-API (Zhang et al., 2023) has 1,600 test examples, prompted with the code preceding the target call in the same file, with 96 answer tokens and the official EM scorer. Memory contains other files of the same repository, excluding the query file, its near-duplicates and calibration files under the submission's deduplication rule. NQ-open has 3,610 test questions, eight fixed training demonstrations and 32 answer tokens; scoring uses normalized EM of the first output line against any reference answer. Its memory contains training question–answer pairs excluding the demonstrations; 3,000 development examples are used for calibration. Training records whose normalized question equals a test question are removed.
Each memory record is written in the task template, with its answer masked from a given position onward and clean blocks stored at the first masked position. No test-set answer annotations are used for key construction, prompts or calibration.
Shared decoder and controls. The host is an uncached confidence-threshold decoder: within 32-token blocks, each pass commits masked positions with confidence at least 0.9, and at least one. R-Joint retains the submission's fusion, gate (5) and calibration grid. Its 55% reveal threshold is counted over the last up to 512 positions of the prompt and answer. Each pass commits at most three non-overlapping blocks of length 2–4 that pass the gate; these are added to the host's selected positions, with joint values used at overlaps. B is the token-wise retrieval control of Table 5 and is calibrated separately. Marg commits the product of the fused marginals with R-Joint's parameters.
RepoEval-API EM is reported in %:
R-Joint leads by 0.25 to 0.43 pp. On NQ-open the same comparison gives +0.36 pp over B and +0.44 / +0.70 pp over Marg (LLaDA / Dream). These generation results are reported as point estimates without document- or repository-clustered intervals.
Reference: Zhang et al. (2023). RepoCoder: Repository-level code completion through iterative retrieval and generation. EMNLP.
Thank you again for the helpful question. We would be happy to address any further questions or concerns.