Trees from Marginals: Autoregressive drafting with factorized priors

On-device applications of autoregressive large language models (LLMs) require fast token generation for individual requests, yet sequential decoding limits generation speed while underutilizing available compute. Speculative decoding trades additional computation for more tokens per target-model forward pass. While factorized drafters have set new records in speculative decoding efficiency by predicting future-token marginals in parallel, their independence assumption causes acceptance rates to degrade sharply at later draft positions, limiting their scalability to large speculative budgets. In this paper, we analyze this limitation and introduce Weaver, a lightweight autoregressive adapter that constructs proposal trees from the top-$K$ predictions of a factorized drafter. Weaver scales to large tree budgets by restoring conditional dependencies while avoiding the cost of a full-vocabulary projection. To verify these trees efficiently in recent models with Gated DeltaNet layers, we derive a rollback-free tree-verification algorithm and implement GPU kernels in CUDA and Metal. By combining these model and systems contributions, we achieve average lossless speedups of $4.09\times$ over autoregressive decoding and $1.22\times$ over a DFlash baseline on NVIDIA B200 with Qwen3.6-27B. For local deployment with the same model, we achieve average speedups over autoregressive decoding of $3.11\times$ on Apple M5 Max and $6.29\times$ on NVIDIA DGX Spark, with corresponding speedups of $1.35\times$ and $1.51\times$ over DFlash.