- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper presents a systematic study on accelerating the inference for Hybrid Diffusion Language Models, where consists of both full attention and GDNs. Authors propose Exact Warm-State Reuse to cache both KV cache and GDN memory states. Exact Warm-State Reuse decides what regions need to be cached and only re-computes changed suffix. The experimental results show median speedup to 3.03x.
Critical strengths
Motivation
- The paper focuses on a practical problem for hybrid models, where KV Cache and hidden states in the GDN should be jointly considered.
Method
- The paper proposes a well-designed mechanism, Exact Region-Based State Reuse, as a solution.
Critical weaknesses
- Exactness under the native attention mask is confusing. It is the proposed Region-DAG mask provides the exact equivalence, instead of the native bidirectional attention from the model. The hidden states of stable tokens may change after each denoising step. Can you demonstrate whether the proposed caching mechanism saves the native inference dynamics from its native bidirectional attention or not?
Questions for authors
N/A.
Native inference fidelity and the localized row-geometry discrepancy
Thank you for identifying the central ambiguity. We agree that matching a declared attention mask and controlled prediction hashes does not, by itself, establish preservation of native hybrid inference dynamics.
Our supported claim is bitwise cached execution relative to a declared, shape-matched canonical execution contract. It is not a general claim of bitwise equivalence to native generation under every execution schedule.
The audited native configuration uses causal prompt prefill and a mixed clean/noisy self-speculative block. Matching attention edges is nevertheless insufficient to match the complete hybrid computation: the separate fixed-state native/declared diagnostic does not match across its four tested steps. GDN policy and finite-precision execution schedule must also be considered.
We separated four serving paths:
A: pristine native self-speculative generation. B: ordinary exact handoff without cross-edit transfer. C: fresh segmented prefix/suffix reconstruction. D: cross-edit joint-state restoration.
In the bounded diagnostic, A and B match on all eight observed outputs and all 563 aligned sampled tensor comparisons. C and D match on all eight outputs and all 696 aligned sampled comparisons. Native and segmented outputs differ. Thus, the discrepancy can arise without cross-edit restoration.
The latest raw-tensor experiment shows bitwise-identical embedding, normalization, and projection inputs, with identical recorded projection weights. All six own-shape projection replay checks reproduce their captures bitwise.
Changing projection execution from 946 rows to 11 rows reproduces the QKVZ discrepancy: 61 of 90,112 suffix values differ, with maximum absolute error 0.015625. Diagnostic 64-row padding reproduces the native projection output for this case.
This supports a row-geometry numerical explanation for the first observed causal-prefill divergence. It is not a full-model repair, does not explain every older mixed-GDN discrepancy, and does not establish preservation of denoising trajectories.
The sequential-edit benchmark records genuine restoration and invalidation, but native and restored-route final outputs match in 0/360 paired requests. We will retain this negative result and will not claim native-equivalent editing acceleration.
We will revise the manuscript to distinguish dependency eligibility, bitwise reuse against the canonical reference, and fidelity to native finite-precision generation. The new diagnostic explains the first observed divergence while leaving the broader native-equivalence limitation explicit. We will also avoid treating equality of completed answer tokens as proof that internal states or denoising trajectories are identical.