- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces DSpine, which is a parallel speculative decoding method based on depthwise causal conditioning. It establishes the token dependencies progressively across the depth dimension of the drafter. It has demonstrated the empirical observation clearly 1) token-predictive information appears in shallow layers 2) early predecessor conditioning improves successor prediction. Experiments are conducted fairly as all methods are trained from scratch using the same dataset and it shows significant improvement in both acceptance length and end-to-end inference speedup.
Critical strengths
- The paper is well motivated with clear empirical evidence on the how the depth and token dependency affect the final performance.
- The architecture for the drafter is well designed. The proposed layerwise feature injection, embedding supervision and final conditional decoding are sound.
- DSpine consistently outperforms strong parallel drafting baselines across different benchmarks under both greedy and sampling-based decoding and this gives pretty convincing results.
Critical weaknesses
- The ablation study section is quite short with brief mention of the results. I think the ablation study missing an important study on the effect of block-causal attention. DSpine changes the backbone attention from bidirectional to block-causal while simultaneously introducing adjacent causal injection. Both changes could improve predecessor-to-successor information propagation. However, the current ablations do not sufficiently isolate their individual effects.
- The gated adjacent injection resembles a lightweight residual feature communication mechanism. It remains unclear why the authors choose to project hidden states into a PCA-whitened output-embedding space.
- Does the
Questions for authors
- As DSpine is a depthwise technique, how will the acceptance-length and latency change when varying the number of injection layers?
Thanks for your reply. Beyond just the sequence length, I would be more interested in seeing a fair comparison of throughput across different batch sizes for Qwen-3.8-27B, after all, there may be additional costs involved here. Furthermore, it is my understanding that the recommended block size for DFlash V2 on Qwen-3.8-27B is 8; using a block size of 16 seems quite unfair, as the model was likely not trained with a block size of 16.