- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes a parallel speculative decoding framework called Compass. The main contribution is to combine with two components: 1) Compact Layerwise Target Guidance compresses multiple target hidden states into a shared low-dimensional representation and injects separate shifts into the gate and up inputs of each draft MLP; 2) Reference-Aware Guidance Optimization uses a frozen initial drafter to distinguish previously accepted prefixes from prefixes requiring repair, applying different losses to these two groups.
Theoretically, the paper also derives a per-block lower bound on acceptance improvement under greedy verification. Experiments on vLLM report improvements in acceptance and decoding speed across three Qwen3 target models and six benchmarks.
Critical strengths
In general, the paper provides some practical findings while running parallel SD:
-
Some new findings ignored by previous works: improvements in draft acceptance can be offset by the computational overhead of target conditioning. The diagnosis are well-motivated with empirical results.
-
Proposal of Compact Layerwise Target Guidance and Reference-Aware Guidance Optimization to adress the new problems.
-
Extensive evaluations based on vLLM framework with existing schemes such as DSpark, Dflash, Domino, etc.
Critical weaknesses
While I appreciate the new findings and the proposed method, I think the paper has the following weaknesses:
-
Overall, the paper builds on the existing technique/architecture and the contribution on the architectural side is rather incremental. Multilevel target-feature fusion, target-dependent MLP steering, and low-dimensional projections are well-established ingredients. The usage of MLP projection is expected and does not conceptually differentiate this work from the existing ones. To this end, I don't see much that is fundamentally new in this work.
-
The design of acceptance-aware training objectives is not new either. The related-work section identifies several acceptance-aware objectives, including prefix-product objectives and adaptive position weighting. The main ablation compares RGO with KL+CE, but does not compare it against these closer alternatives.
Conceptually, RGO is similar to the existing re-weighting schemes -- I see "reference-aware protection/repair split" as a good packaging of the method rather than a fundamental novelty. Specifically, it should compare RGO against: (a) a plain KL regularizer toward the frozen reference, and (b) a uniform margin loss with triangular weights but no repair branch.
-
The theoretical part looks a bit forced, or AI-generated. I don't get much from it, neither by reading the proofs.
-
The cost of the proposed method is not analyzed or evaluated. While the paper argues the proposed method as low-cost, the inishgts from guidance latency, peak memory, total training cost, and end-to-end throughput against a matched baseline is missing in the manuscript.
-
Results other than Qwen are prefered.
Questions for authors
Please check the weaknesses part.