- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
In models with AR and diffusion modes, one network both drafts and verifies, but drafting still waits for verification, so the standard schedule needs two sequential model calls per iteration. To address this bottleneck, this paper introduces Packed-SS, a self-speculative decoding schedule that verifies and drafts in a single model call with input size linear in the speculative width, while preserving the target model's distribution. The key idea is to let the diffusion drafter run one call ahead: while verifying the current draft, the model simultaneously constructs the next draft from an older verified prefix and keeps its unresolved suffix for the next iteration.
Critical strengths
-
The motivation in the paper is reasonable.
-
The paper is clearly written.
Critical weaknesses
-
The proposed method has too narrow an applicability, it can only be used with models like Nemotron-Labs-Diffusion that support both AR mode and diffusion mode.
-
Although it reduces model calls, the final throughput improvement compared to Nemotron is limited.
-
Is there a way to apply the proposed method to external drafters like dflash?
-
The presentation of experimental results in the paper is unclear, it is strongly recommended to use tables to show the results.
-
The figures in the paper have significant room for improvement.
Questions for authors
Please see the weaknesses