- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies speculative acceleration of Permute-and-Flip (PF) decoding while preserving its native watermark. SpecPF couples the draft and target models through a shared exponential random field, proposes multiple candidates at the root, extends each branch along a single path, and lets the target PF map traverse the precomputed tree. Theorem 2 establishes pathwise equivalence to direct PF under identical target logits, randomness, and the other stated conditions. For detection, the paper derives the alternative density of the PF pivot, proves single-pivot optimality of the native score, and applies five-fold cross-fitted bigram-signature filtering before Fisher/Gamma testing. Across Qwen and Vicuna configurations on three datasets, shallow SpecPF reports up to 1.32x measured speedup over direct PF. Longer lookahead raises aggregate accepted tokens per verification from 1.787 to 2.745. Signature filtering improves aggregate detection rates at 64 and 128 scored tokens. The approach is promising, but implementation-level exactness and gains under fully aligned comparisons remain insufficiently established.
Critical strengths
- A simple construction with a clear exactness objective. The target PF map always determines the output, while the draft only schedules contexts for advance computation. This supports a direct pathwise-equivalence argument and offers a principled route to retaining the native watermark. Root branching followed by single-path extensions limits tree growth. The same-run results in Table 1 show approximately 1.27-1.32x speedups on Vicuna, demonstrating practical value for the shallow configuration.
- A theoretically motivated detector that does not require target-model replay. The monotone alternative density motivates retaining the native single-pivot score and improving detection by excluding weak-evidence positions identified through observable bigrams. Cross-fitting, shrinkage, and minimum-retention safeguards make the procedure reasonably well specified. The aggregate TPR increase from 0.554 to 0.600 at 128 scored tokens in Table 2 is a useful empirical signal that merits further controlled evaluation.
Critical weaknesses
- The gap between theoretical pathwise exactness and implementation agreement remains unresolved. Table 21 reports whole-sequence agreement with L=1 of 0.909, 0.903, and 0.903 for L=2, 3, and 4, respectively, with the same pivot-exactness rates. Table 10 explicitly uses different context-label domains for Direct and Fused generation. These comparisons therefore do not directly validate the same-random-field, same-target-logits equivalence required by Theorem 2. The appendix appropriately acknowledges that similar detection rates do not prove pathwise preservation. Nevertheless, exact sampling and watermark preservation are central claims, so the discrepancies require a concrete explanation and a targeted implementation check. The reported mismatches do not by themselves refute the idealized theorem, but they limit its support for the current implementation.
- Efficiency and detection gains are not yet isolated under aligned operating conditions. At M=64 in Table 2, TPR rises from 0.455 to 0.481 while empirical FPR rises from 0.009 to 0.013; the gain at strictly matched empirical FPR is therefore unclear. For efficiency, a width-three SpecPF tree is compared with methods using one draft sequence and different target output distributions. Higher accepted-token counts do not establish better throughput at equal target-computation cost. Table 21's L=4 configuration achieves only 0.903x the geometric-mean throughput of the new L=1 baseline, and Table 17 also shows slowdowns for Qwen7. The manuscript discloses these limitations, but stronger evidence at matched computational budgets and empirical FPR is needed to substantiate the practical advantage.
Questions for authors
- What causes the approximately 9-10% sequence disagreement in Table 21? Please compare direct PF, shallow SpecPF, and longer-lookahead SpecPF token by token using identical context labels, randomness, and target-processing rules. A trace of the first divergence, including logits, support, noise, and cache execution, would help distinguish random-field addressing or execution differences from numerical effects. Can complete agreement be recovered when the theorem's conditions are enforced?
- How much does SigFisher improve detection on the same test population at matched empirical FPR? Please provide low-FPR ROC results or matched-FPR TPR estimates with uncertainty intervals, and clarify whether cross-fitting groups samples by prompt and key. Do the conclusions hold when filter-training and threshold-calibration uncertainty, excluded from Table 6's intervals, is included?
- Do the efficiency gains remain when target-context computation or measured runtime budgets are matched? A breakdown of drafting, target verification, and tree/cache management would clarify when longer lookahead improves token yield and when it improves actual throughput.
- Please specify the context-label function and treatment of repeated labels so that the Fisher/Gamma assumptions can be checked against the implementation. Which constructions or theoretical results are specifically new to PF relative to the cited shared-randomness methods and Hong et al.'s signature filtering?
4 comments
Responses to the questions (part 2).
Q2. Matched FPR, low-FPR sensitivity, prompt/key grouping, and bootstrap uncertainty
Scoring correction. A CPU audit reconstructed the frozen leave-one-dataset-out results and verified legacy_v1_reproduction = True. The legacy program thresholded a raw retained sum, although the retained number of pivots varies by text. The correct canonical score uses
With unchanged frozen tables and the original empirical calibration convention, Original PF and corrected SigFisher attain TPR 0.5508 and 0.5920 at (empirical FPR 0.0129 and 0.0139). This is a frozen cross-dataset transfer audit and does not independently establish matched-FPR improvement.
Main population reanalysis. We use the same held-out positive and null sample sets for Original PF and SigFisher within each model–dataset–budget cell. Five-fold splits group by prompt; each prompt has its own derived key. For each of 100 bootstrap repetitions per eligible cell, we resample paired prompt records, refit the five-fold bigram tables and recalibrate null thresholds. The stored generator outputs, original folds and keyed-field realizations remain fixed. All ten eligible cells reproduce the earlier canonical detector-suite output before resampling.
| Budget | Protocol | TPR Original → SigFisher | Measured FPR Original / SigFisher | TPR difference (pp) | Descriptive pooled 95% CI (pp) |
|---|---|---|---|---|---|
| 64 | Common analytical | 43.30% → 45.65% | 0.75% / 0.99% | +2.35 | [+0.95, +3.08] |
| 64 | Independent null calibration | 46.67% → 48.23% | 1.09% / 1.19% | +1.56 | [−1.35, +5.98] |
| 64 | Descriptive matched-FPR ROC | 45.96% → 47.38% | 1.12% / 1.12% | +1.43 | [−3.37, +5.16] |
| 128 | Common analytical | 59.41% → 62.89% | 1.40% / 1.34% | +3.47 | [+1.37, +4.39] |
| 128 | Independent null calibration | 56.12% → 60.88% | 1.34% / 1.04% | +4.75 | [−4.34, +13.14] |
| 128 | Descriptive matched-FPR ROC | 56.19% → 60.21% | 1.10% / 1.10% | +4.02 | [−1.66, +9.39] |
All six cells qualify at ( test), but at only four cells qualify (). Qwen14/CNN has just 14 calibration and 2 test texts; Vicuna/CNN has 94 and 45; both are excluded by the prespecified minimum of 100 in each split. The 128-token figures thus cannot be directly compared with the paper's original nine-cell or main six-cell aggregate. The intervals are descriptive test-count-weighted pooled bootstrap intervals, not hierarchical population-wide confidence intervals. Analytical cutoffs do not ensure identical empirical FPR, and matched-ROC thresholds were selected using evaluation nulls rather than an independent calibration split. Crucially, both independently calibrated 95% intervals include zero. We therefore do not claim statistically conclusive SigFisher superiority at strictly matched realized FPR, and we now provide low-FPR sensitivity plots with these qualifications.
Q3. Matched target work and runtime budget; drafting, verification and cache-management costs
The same-run direct-PF comparisons show the wall-clock regimes in which shallow SpecPF accelerates the same target law (Qwen14: 1.036–1.142×, Vicuna: 1.274–1.320×) and the Qwen7 slowdown (0.870–0.947×). The completed L1–L4 sweep also reports target contexts, draft/target calls, block latency, peak memory, and token rate; the average represented target contexts/block grows from about 4.0 at L1 to about 12.9 at L4. Although AATPS rises from 1.787 to 2.745, the geometric-mean L4/L1 throughput is only 0.903×. The L>1 rank-tree verifier is not optimized to precisely the same degree as the production L1 path, so this comparison includes implementation maturity as well as algorithmic overhead.
We now show actual token rate alongside AATPS and discuss represented target contexts as a work proxy, not measured FLOPs. We do not claim a controlled equal-compute, same-device comparison against all softmax-based baselines or separate per-component draft/verification/cache latencies; those measurements have not been completed. We also distinguish the U-score-only MSE-PSEUDO result from the unvalidated acceptance-aware detector. These revisions narrow the cross-method interpretation to observed operating points rather than a compute-matched Pareto-optimality claim.
Responses to the questions (part 3) and conclusion
Q4. Context labeler, repeated labels, Gamma-null assumptions, and novelty
The installed production shared-noise generator constructs
where .
using the domain separator specwater-shared-noise-portfolio-pf-v1|, an 8-byte unsigned big-endian length, and signed 8-byte big-endian token IDs. Prefixes at different output positions have different lengths and therefore cannot reuse the same label within a sequence. Keys are derived separately for different prompts. The null requires a text independent of the held-out key field, unique scored keyed context–token pairs, and a frozen text-only filter fitted without access to held-out-key randomness. Conditional on these assumptions in the ideal random field, retained pivots are i.i.d. uniform, giving the exact null. The implemented PRF is computationally pseudorandom rather than an ideal independent field; finite-sample empirical FPRs remain necessary.
We now explicitly credit generic shared-randomness decoding and Hong et al.'s signature filtering; the original contributions are the PF-specific bounded-width shared-exponential executor, the conditional fixed-operator pathwise theorem, the selected-PF-pivot density/monotonicity and single-pivot lower-tail optimality, and their application to the PF retained-count Fisher/Gamma test.
We remain confident in the value and contributions of our work, and would sincerely appreciate it if you could kindly reconsider your evaluation in light of the additional clarifications, corrections, and new experimental results. Thank you again for your time and consideration!
Thank you for the detailed responses
Thank you for the detailed responses and the effort devoted to the revisions and additional analyses. I have carefully considered the clarifications, corrections, and new results, which help clarify the scope of the claims and the interpretation of the experiments. However, after considering the response as a whole, my overall assessment remains unchanged. I therefore maintain my original score and recommendation of weak rejection. I appreciate the authors’ engagement throughout the discussion. score:2.
Responses to the questions (part 1).
Thank you so much for a precise evaluation of the main outstanding soundness and empirical-comparison issues! We have substantially revised the paper according to your comments. All the changed are highlighted in blue.
We performed additional numerical and statistical analyses specifically targeted at these questions. We answer the questions one by one as follows.
Q1. What causes approximately 9–10% L1/L>1 sequence disagreement? Can complete agreement be recovered?
Mathematical statement. Define a single deterministic processed-target operator UP(c)=(uc,SP(c)). With the same operator, labeler Λ, keyed field EΛ(c)(y), support, tie-breaking and stopping rule at every reached context, both direct PF and SpecPF emit Y(c)=argmaxy∈SP(c){Tuc(y)+EΛ(c)(y)}.
Induction over tokens gives exact pathwise equality, for any draft, B and L. This is a fixed-operator theorem; bitwise agreement between separately batched/cached FP16 forward passes is not assumed.
Physical experiment. The completed 6-cell lookahead sweep reports whole-sequence agreement with production L1 of 0.909, 0.903 and 0.903 for L2, L3 and L4. We extracted first disagreements and replayed six selected examples on the installed production shallow decoder and rank-tree executor. Each implementation independently reproduced its saved output in all six cases. Context labels, the processed top-50 support and keyed PF uniforms agreed; the numerical PF selection matched independent score calculations. The physical processed target logits nevertheless differed sufficiently to reverse the PF winner:
For example, on Qwen14 prompt 102, the processed logit of competing token 525 is 39.78125 in L1 versus 39.81250 in the rank tree, reversing the PF winner under the same keyed field. The two later-position traces were completed after correcting our diagnostic tracer's post-forward cache-length assumption; this was not evidence of a decoder cache bug.
These six examples identify a reproducible selected-case mechanism consistent with execution-geometry-dependent numerical differences. They do not establish that every population mismatch has this cause, that FP16 rounding alone is responsible, or that the current physical executors have achieved 100% agreement. Table 10's historical Direct/Fused experiments use distinct label domains, and we now explicitly label that comparison as a statistical rather than pathwise same-field check. A common-operator bitwise replay of the entire generation campaign has not been performed; we will not claim otherwise.