- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes MPFR, a speculative sampling algorithm based on a Poisson process that uses multiple drafts to improve sampling efficiency while preserving watermark strength. Its core contribution is to extend sampling to multiple drafts using the mapping theorem for Poisson processes and to ensure drafter invariance through a keyed Poisson source indexed by context.
Critical strengths
- The proposed method has theoretical value and may support future work combining watermarking with speculative sampling.
- The stopping-time drafter invariance has practical value and offers a reasonable compromise between strong invariance and conditional invariance.
Critical weaknesses
- The writing is difficult to follow. More intuitive explanations and illustrations would improve readability.
- The core contribution is incremental. For example, Sections 4.3 and 4.4 largely restate existing work, with the main new contribution limited to the extension to multiple drafts.
Questions for authors
- Section 4.3 explicitly acknowledges that PFR with a single draft is equivalent to MWS. Is the paper’s core contribution therefore entirely the extension to multiple drafts?
- The efficiency improvement of MPFR over GLS in Table 1 appears small. Does this suggest that increasing offers only limited gains for MPFR? Could the authors report the mean and standard deviation across three different seeds to present the results in more detail?
- The method achieves drafter invariance at a stopping time, as defined in Definition 4.3, rather than strong drafter invariance. Is this because the stronger property cannot be achieved, or because it is unnecessary?
- The paper reports that ANLPPT remains stable as changes . Is this because each token is still coupled as in the case of a single draft, making the watermark signal independent of by construction? Could the authors compare against an alternative in which the watermark signal naturally weakens when multiple drafts are used?
4 comments
Responses to drafter invariance and ANLPPT's stability, and Summary
Q3 — "invariance holds at a stopping time, not strongly. Is strong invariance unachievable, or unnecessary?"
In short, we consider it unnecessarily strong for us. However, we clarify that the right distinction is between a variable-length speculative block and the complete generated response, rather than a general impossibility of strong invariance. The stopping length depends on the drafter, so comparing the laws of without specifying whether position was emitted can be ambiguous. We do not claim that every coupling with a shortened block must fail a strong-invariance definition.
What we establish is a particularly operational property: for a fixed target, key, prompt, and shared context-indexed randomness, every emitted block is a prefix of the same target-keyed sequence , i.e. almost surely, regardless of the permitted draft choice. Conditional statements at a given block length are understood only on block-length events with positive conditional probability. When MPFR is iterated to a fixed total output length, the entire resulting output is pathwise identical across compatible drafters. Thus the drafter governs how many verification passes are needed, but not the target-keyed text that is produced. We now give the multi-draft proof in Appendix C (Proposition 5.2).
Empirically, extending the drafter-substitution ablation from to and varying the drafter in both scale (1B → 3B) and temperature (0.5 / 1.0 / 1.5): AATPS spans 3.47–4.39 and token rate 36.4–45.1 across the four conditions and both draft-count settings, while ANLPPT-U varies by and TPR@1%FPR by points. Efficiency responds strongly to the drafter; the detector-visible signal does not.
Q4 — "is ANLPPT's stability in true by construction, and hence not a result? Can you compare against a scheme whose signal weakens with more drafts?"
It is true by construction — and we now prove why: each emitted token's keyed pivot is a function of (key, context, target distribution) only, never of . So the reviewer's reading is correct, and we will present the stability as a consequence of a design decision rather than as a separate empirical finding.
But the construction is not free, which is what the requested comparison shows. Stability requires indexing the Poisson source by context. Index it by draft slot instead — the natural way to watermark an active-set multi-draft scheme, which is the structure the active-set baseline uses — and the emitted token is determined by a maximum over the realised active draft subset. The detector cannot recover that subset from the output text, since it depends on the accept/reject trajectory; it can only regenerate the maximum over all rows, which is not the pivot that produced the token.
In a synthetic setting that isolates the keying index (, Gaussian logits, 50% active fraction, 2000 tokens), with each scheme calibrated against its own correct null so that the comparison does not rest on mis-calibration, the per-token evidence is:
| context-indexed (ours) | draft-indexed | |
|---|---|---|
| 1 | 3.343 | 3.222 |
| 2 | 3.252 | 2.483 |
| 4 | 3.279 | 2.432 |
| 6 | 3.268 | 2.467 |
| 8 | 3.253 | 2.427 |
Ours is flat in ; draft-indexed keying loses about 25% as soon as and does not recover. At the two coincide, since the active subset is then necessarily the full set — a consistency check on the simulation. We will report this as a mechanism check and label it as such, not as a model-level result.
Separately, we have verified at that keying costs nothing in efficiency. On the Llama-3.1-8B/3.2-1B pair at , comparing MPFR against an otherwise identical decoder whose keyed Poisson source is replaced by unkeyed fresh noise (five cells over two datasets and ): AATPS lies in and TR in , while ANLPPT-U rises by 23–90×. The same holds in the single-draft regime (PFR 41.46 vs PFR-NoWM 41.72 tok/s at , and equally at every other ).
Summary
We sincerely hope that these explanations, corrections, and new results address the reviewer's concerns. We have revised the manuscript accordingly, with changes highlighted in blue. We appreciate the reviewer's time and constructive feedback, and would be grateful if the reviewer could kindly reconsider their evaluation in light of the strengthened theoretical and empirical evidence.
I thank the authors for their response. However, the extensive references to prior work in Sections 4 and 5 still make the paper’s actual contribution difficult to assess. Given its focus on watermark sampling efficiency, the experiments should offer more direct insight than the theory, yet those in the main text appear limited. I therefore remain inclined to keep my original score.
One More Clarification of Our Contributions in the Revised Manuscript
We sincerely thank the reviewer for the follow-up and for taking the time to consider our paper. We understand your concern and agree that our contributions could be presented more clearly.
We have therefore further revised the manuscript, with all new changes highlighted in , focusing on:
-
Section 4 (Background): We clarify that (1) this section is mainly for background, and hence it contains components that resemble existing work; and (2) the single-draft version can be viewed as a key representation that coincides with MWS, but it is precisely this representation that allows us to exploit the properties of Poisson processes to construct our multi-draft algorithm. A direct extension of MWS is unclear, and this representation serves as a key translation.
-
Section 5 (Contribution): In short, we design a novel multi-draft algorithm (Algorithm 4) with new theoretical analysis that achieves desirable sampling efficiency while preserving watermark strength. In contrast, the baseline GLS does not incorporate watermarking. More importantly, our key contribution is to introduce draft multiplicity as an additional dimension of the watermark–efficiency trade-off, improving efficiency while preserving watermark strength.
Moreover, we have also added the experimental results from our previous rebuttal to the revised paper, highlighting that MPFR achieves 48.30 tokens/s versus 33.62 for MWS, with comparable watermark strength.
We would be very grateful if you could take one more look at the -highlighted revisions, particularly at the end of Sec 1, the beginning of Sec 4, and Sec 5. Our primary intention is to make the paper's contributions easier to identify and assess, rather than to persuade you to change your judgment (of course, we would be extremely happy if you agree that our revisions based on your suggestions make the paper clearer to you).
We fully respect your assessment, whatever your final decision may be. Thank you again for your time, thoughtful feedback, and help in improving our paper!
Responses to presentation, contribution, and empirical gains
We have provided detailed responses to your points and substantially revised the manuscript using the permitted additional page, with all changes highlighted in blue.
Thank you for recognizing the theoretical value of our Poisson-process construction and the practical importance of stopping-time drafter invariance. We also appreciate the thoughtful questions about the novelty of the multi-draft extension, experimental variability, and watermark stability. We address weaknesses and four questions below.
W1: presentation
The revision adds a concise step-by-step PFR/MPFR workflow explanation, introduces the roles of lookahead L and draft multiplicity B with a simple two-draft example, adds a numerical comparison table against MWS/MSE and the multi-draft GLS baseline in Appendix D, and expands the captions of Figures 1 and 2 to make the intended trends explicit.
W2 / Q1 — "is the core contribution only the multi-draft extension?"
We respectfully argue that our contribution is more than a naive multi-draft extension. First, compared with GLS, our scheme relies heavily on the properties of Poisson processes, and the construction itself is nontrivial; moreover, it achieves better performance. Second, the key observation is that, although Hu and Huang (2024) identified an inevitable tradeoff between speculative sampling efficiency and watermark strength, we can use the multi-draft extension as a third dimension to address this fundamental tradeoff. Of course, achieving higher AATPS is straightforward; the key question is whether we can indeed achieve better wall-clock time without hurting watermark strength, and we answer this affirmatively. Moreover, technically, we also provide:
A context-indexed rather than draft-indexed keyed source. This is the design decision that makes the rest work; see Q4, where we show that the draft-indexed alternative loses detector evidence as B grows.
A proof that the construction is drafter-invariant at B>1. The submission defined multi-draft invariance (Appendix C.2) but proved it only for the single-draft Algorithm 3. We have closed that gap; the new result is stated in our response to Reviewer vB4D and holds for heterogeneous drafter sets, different temperatures and different B.
An acceptance lower bound strictly tighter than the corresponding bound for the existing drafter-invariant multi-draft scheme (Theorem 5.1, B.1–B.4). Following Reviewer vB4D we will state precisely that this orders the bounds, not the true acceptance probabilities.
Such constructions, theoretic guarantees, and practical performance on both sampling and watermarking, are nontrivial contributions.
Q2 — "the gains in Table 1 look small; can you report mean ± std over seeds?"
Table 1 averages 3 seeds × 1000 prompts per cell; the per-seed standard deviations are in Appendix Tables 6-9 but were dropped from the main table, which makes the result look like a single run. We will restore them.
The underlying evidence is stronger than the cell means suggest. Across all 48 paired seed-by-configuration comparisons (identical prompts and seed protocol, both decoders run sequentially in one process on matched hardware), MPFR has the higher token rate in 48/48, with per-cell gains up to +5.4%. Per-cell seed variability is small relative to the gap: the standard deviation of the MPFR − GLS token-rate difference across seeds has median 0.21 tok/s, while the difference itself ranges 0.34–2.17 tok/s, and its sign never flips.
On whether increasing B offers only limited gains — this is the more important half of the question, and the answer is the opposite: the gain grows with B, and Table 1 understates it because it fixes L=4, which is where the B-dependence is weakest. On a unified-hardware (L,B) sweep (see response to Reviewer ZXzX), the gain over the multi-draft baseline on the Llama pair shows a particularly pronounced gain at larger B, especially at small L (the intermediate values are not always monotone):
The same trend holds at scale and gets stronger, not weaker: on Qwen2.5-72B the paired t-statistic for token rate goes from −0.48 at B=2 to +9.94 at B=8 (and +10.85 with a 1.5B drafter), i.e. +11.62% and +10.47% at B=8. The mechanism is in the ZXzX reply: wider draft lists produce more duplicate prefixes, context merging collapses them (66% fewer expanded tree nodes at B=8 versus 52% at B=4), and the target's verification batch shrinks accordingly.
Increasing B is where the method pays off; L=4 with small B is the least favourable corner of the grid. We keep L=4 as the main configuration, since it matches the protocols of the prior work, and add the full grid alongside it so the dependence is more visible.