- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents DIGAA, a training-free framework that separates information gathering from answer generation in retrieval-augmented diffusion language models. During information gathering, the model uses parallel token predictions to guide retrieval and collects information through a second forward pass. A separate generation stage then produces the final answer from the collected information. Experiments on Dream-7B-Instruct and LLaDA-8B-Instruct across six factual QA benchmarks report improvements over nine baselines. The paper also evaluates retrieval settings, inference latency, and the effect of applying the answer-generation stage to existing methods.
Critical strengths
- Separating retrieval-oriented intermediate predictions from the final answer is a practical design choice, and the framework requires no additional training.
- The evaluation covers two diffusion models, six QA benchmarks, sparse and dense retrievers, and multiple retrieval depths.
- Applying the same answer-generation stage to the baselines is a useful control. DIGAA retains improvements over these strengthened baselines, suggesting that its information-gathering procedure contributes beyond answer refinement alone.
Critical weaknesses
-
The information-gathering ablation does not isolate the proposed design choices. Although Table 4 controls the answer-generation stage, DIGAA changes both the documents used to construct retrieval queries and the number of forward passes per gathering step. It therefore remains unclear whether the gains arise from using fixed initial documents, additional computation, or their interaction. Separate component ablations would clarify this contribution. Retrieval recall or supporting-fact coverage would also help substantiate the claimed broader and deeper retrieval.
-
The quality and latency comparison uses different decoding budgets across methods. DIGAA is evaluated across several confidence thresholds, while SARDI is fixed at 0.90 and most other baselines use 32 or 64 forward passes. The stronger SARDI + AG baseline is also absent from the latency comparison. Although the reported wall-clock measurements are useful, comparable threshold sweeps and computational budgets are needed to support the claim that DIGAA offers a superior balance between quality and latency.
-
Several reported quantities are difficult to reconcile with the stated protocol. The no-retrieval Dream baseline scores 15.9 EM on NQ in Table 1, but 14.9 in Tables 6 and 7 and 14.8 in Table 8, despite the stated common decoding settings. Changing retrieval settings should not itself affect this baseline. Moreover, Algorithm 1 requires at least three forward passes and two retrieval calls per question, whereas Table 9 reports averages of 2.8 and 1.9 for Dream at τ = 0.10. These discrepancies require clarification before the experimental comparisons can be assessed confidently.
-
The answer-generation analysis does not clearly distinguish improvements in factual correctness from changes in answer format. Section 4.2 attributes 61% of the observed improvements to locating answers already present in the first-stage output. The remaining 39% groups inference with word recombination, answer completion, and other edits. Since the analysis only examines improved examples, it also omits cases where refinement damages a correct answer. A separate breakdown of factual corrections, formatting changes, and regressions would better establish the contribution of this stage.
Questions for authors
Please refer to the summary of weaknesses.
Response to Weakness 1.1 & Question 1.1: Technical differences and methodological novelty of DIGAA relative to IRCoT
Thank you for the comment. DIGAA's novelty lies in organizing relevant facts while retrieving documents using model outputs, separating predictions used to construct queries from those used to organize facts, and finally generating a direct, accurate answer independently from the organized facts. This allows the model to generate intermediate entities for broad exploration without worrying that this content will be mistakenly retained. Because facts are organized during exploration, the model can generate the answer directly without facing the large amount of irrelevant information in raw documents. This is an advantage unique to DIGAA. Therefore, describing DIGAA as "incorporating an established AR LLM-based RAG framework into an existing dynamic retrieval approach" is inaccurate.
W1.1&Q1.1: Technical differences and methodological novelty of DIGAA relative to IRCoT
Methodological novelty
Relative to IRCoT, DIGAA's specific contribution is to extend "retrieving documents" to "organizing relevant facts while retrieving documents." This allows the model to generate answers directly without facing large amounts of irrelevant information in raw documents or having to select and organize useful information.
Technical differences
We replaced the knowledge collection read by DIGAA's answer-generation stage with the retrieved document set, adopting an answer-input format similar to IRCoT's to test the benefit of organizing relevant facts while retrieving documents:
Reading the knowledge collection achieves higher EM and F1 on both models, supporting the benefits of collecting and organizing relevant facts before answer generation.