- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3(+1)
- Soundness
- 2
Read review
Paper summary
The paper argues that masked diffusion language models form an early, relation-dependent factual access state at visible entity positions. It supports this account with causal interventions in two models, then uses the proposed state for multi-view knowledge editing and per-entity evidence retrieval.
Critical strengths
The causal interventions give the proposed mechanism more support than attention visualizations alone. The editing and retrieval experiments also test whether the mechanism leads to useful applications; the reported improvements over the included baselines are substantial.
Critical weaknesses
The editing result remains limited on the natural question-transfer setting. On QAEdit, only 17.6% of LLaDA and 23.8% of Dream source-prompt answers are fully correct after Multi-View editing; performance on paraphrased questions is lower. This weakens the practical claim that a declarative edit can reliably be accessed through later questions.
The retrieval comparison does not fully isolate the benefit of the proposed representation. The proposed method uses known entity spans, makes one ranking per entity, and alternates results within a two-item budget. The principal baselines score the whole query. The entity-conditioned control is useful, but it still does not compare against a baseline given the same per-entity ranking procedure. In the natural evaluation, retrieval is also limited to each question’s supplied distractor context.
Questions for authors
How does the method compare with BM25 and dense retrievers when each is given the same entity spans, per-entity rankings, and alternating two-result budget? Would the retrieval gain persist with automatically detected spans and an open corpus? This comparison would materially affect my assessment of the retrieval contribution.
5 comments
Response to Reviewer cHKu (Part 2/2)
W2 + Q1. Does the retrieval comparison isolate the representation?
Thank you for this important point. We ran the three comparisons you asked for.
(1) Same entity spans, per-entity rankings, alternating budget of two. Each baseline is told where the two entities are, but not which relation belongs to which. For each entity, we form a sub-query: the original query with the other entity's name replaced by “the other”. Each baseline encodes it in its usual way and ranks all 5,528 cards once per entity; the two rankings then fill the budget of two in turn, exactly as for our method. Results are All@2 (%) on the controlled benchmark, with global search.
| Method | Query unit | Adjacent | Crossed |
|---|---|---|---|
| LLaDA access state, L31 (ours) | whole query, one encoding | 41.94 | 38.08 |
| Dream access state, L27 (ours) | whole query, one encoding | 38.28 | 21.78 |
| BM25 | whole query (Table 5) | 1.38 | 1.01 |
| BM25 | per-entity sub-query, gold spans | 11.83 | 10.75 |
| All baselines, range | whole query (Table 5) | 0.61–3.06 | 0.00–2.50 |
| All baselines, range | per-entity sub-query, gold spans | 4.0–14.2 | 1.9–13.5 |
Under this protocol, BM25 rises from 1.38 / 1.01 to 11.83 / 10.75. The per-entity procedure lifts the baselines by about ten points, but they stay below the access state. The remaining gap comes from the vector that is read at the entity span.
(2) Open corpus. In the natural setting, we now rank over all 106,329 sentences instead of each question's distractor context. Results are Both@2 (%): distractor context (Table 20) → full corpus.
| Method | 2WikiMultiHopQA | HotpotQA |
|---|---|---|
| Dream access state, L0 (ours) | 37.37 → 33.3 | 41.82 → 39.7 |
| LLaDA access state, L0 (ours) | 33.17 → 27.5 | 40.82 → 35.3 |
| BM25 | 26.93 → 25.7 | 25.15 → 22.8 |
| PromptReps, hybrid | 13.16 → 9.6 | 11.34 → 6.9 |
| DiffRetriever, hybrid (LLaDA) | 13.60 → 4.3 | 19.21 → 13.7 |
| Whole-query pooling (LLaDA) | 10.13 → 4.7 | 16.44 → 8.8 |
The dense baselines fall to 4.3–9.6 and 6.9–13.7. The access state remains the best method on both datasets.
(3) Automatically detected spans. We replaced the gold spans with spans from an off-the-shelf spaCy NER model, without tuning.
| Setting | Gold spans | spaCy spans |
|---|---|---|
| Controlled, LLaDA, adjacent / crossed (All@2) | 41.94 / 38.08 | 33.1 / 32.7 |
| Controlled, Dream, adjacent / crossed (All@2) | 38.28 / 21.78 | 31.1 / 18.8 |
| Natural, Dream L0, 2Wiki / HotpotQA (Both@2) | 37.37 / 41.82 | 31.6 / 31.9 |
| Natural, LLaDA L0, 2Wiki / HotpotQA (Both@2) | 33.17 / 40.82 | 28.1 / 30.7 |
The access state keeps 75–86% of its gold-span performance. On the controlled benchmark, it stays above the baselines even when those are given gold spans (at most 14.2 / 13.5), and in the natural setting it stays above BM25 (26.93 / 25.15).
In the revision: (i) Table 5 gains the per-entity baseline rows. (ii) In §4.2, the sentence “…which is what the state binds” is replaced by a more precise statement: the advantage is binding each entity to its relation in one encoding of the undecomposed query. (iii) The open-corpus and detected-span results are added to App. F.2 (Table R7) and App. F.1 (Table R6), with a pointer from §4.2. (iv) App. G no longer lists automatically detected spans and open-corpus retrieval as untested.
Thank you again for your time and for the valuable suggestions, which clearly improved the paper. We would be glad to clarify anything further.
Thank you for the authors' detailed responses, and I will keep my score.
Response to Reviewer cHKu (Part 1/2)
We sincerely thank the reviewer for the careful and fair assessment. We ran all the retrieval comparisons you asked for and report them below.
W1. Editing remains limited on QAEdit
We agree, and the paper states it as a limitation (Conclusion, App. G): free-generation transfer is improved, not solved. Two points of context: