- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
This paper proposes a three-stage hierarchical distillation strategy: progressive block-wise adaptation, block-wise distillation, and model-wise cross-scale distillation (from an 8B diffusion teacher to a 2B student). For driving understanding, visual perception, and motion planning tasks, the authors first fine-tune Qwen3-VL-2B on multi-task driving data to obtain an AR driving VLA. They then convert this model into a discrete diffusion VLA with block-causal attention. This approach achieves an approximately 2.8× decoding speedup with only a small loss in performance; when combined with system-level optimizations (FlashInfer and CUDA Graphs), the cumulative acceleration reaches 15.1×.
Critical strengths
- The method is simple. It does not require any task-specific projection heads or additional models, and the decoding block size can be adjusted at runtime, enabling a flexible trade-off between latency and accuracy.
- The evaluation is comprehensive. A single checkpoint is tested across three types of tasks under the unified multi-task protocol, and the planning component is further evaluated in both open-loop (NAVSIM v1/v2) and closed-loop (Bench2Drive) settings under the task-specific protocol.
- The ablation study is informative, covering each distillation stage, the divergence objective (forward KL, reverse KL, JSD), and the teacher paradigm (AR teacher vs. diffusion teacher).
Critical weaknesses
- FlashInfer and CUDA Graphs are applied only to the diffusion model, but these optimizations could also be used for the AR baseline (both are commonly supported in vLLM and SGLang). A fairer comparison would be between an “optimized AR” model and an “optimized diffusion” model.
- Accuracy results are evaluated on the Ascend 910C NPU, whereas the hardware-specific inference optimizations were benchmarked on a CUDA-compatible platform, so the latency-accuracy trade-off is not verified on a single platform.
- Speculative decoding with parallel block drafters (e.g., DFlash) and its application to driving VLAs (e.g., FlashDrive) are cited in the related work, but no comparison is provided for any of the tasks.
- For motion planning, a commonly used alternative is a continuous action head (such as the flow-matching trajectory decoder used in Alpamayo-R1), which could be much faster: in the Alpamayo-R1 paper, flow-matching trajectory decoding takes 8.75 ms (5 steps) versus 222 ms for AR trajectory decoding (127 tokens). A discussion of this design is missing.
- There is no end-to-end profiling or comparison. The paper reports only per-token decoding latency without specifying the number of tokens per trajectory, the prefill and visual encoding latency, or the proportion of end-to-end latency attributed to decoding; consequently, the actual benefits for real-world deployment cannot be assessed.
- To my knowledge, prior work already exists on converting AR models into block diffusion models (e.g., Fast-dLLM v2, SDAR). The paper does not discuss how its strategy differs from these existing approaches.
- Table 4 shows that direct AR-to-diffusion adaptation followed by block-wise distillation yields 87.7 PDMS, lower than the 88.1 achieved by the AR baseline, and the score only reaches 88.3 when model-wise cross-scale distillation from the 8B diffusion teacher is added. A fair comparison would be to distill the 2B AR baseline from an 8B AR teacher under an on-policy setting. Similarly, the analysis of exposure bias (which shows a 5.8% reduction in average per-waypoint L2 error) appears to compare a distilled diffusion model against an AR model before distillation; therefore, it is difficult to determine whether the gains come from the paradigm transition or from the 8B teacher.
Questions for authors
- What is the decoding latency of the AR baseline when FlashInfer and CUDA Graphs are also applied, measured on the same platform as the diffusion model?
- How many tokens does a trajectory contain for the main results? Could the authors provide an end-to-end latency breakdown (visual encoding, prefill, and decoding) for both the AR baseline and the diffusion model?
- How does the proposed approach compare with applying speculative decoding (e.g., DFlash and FlashDrive) to the AR baseline, in terms of both latency and accuracy?
- How does the three-stage hierarchical distillation strategy differ from existing AR-to-block-diffusion conversion methods such as Fast-dLLM v2 and SDAR?
- How does the 2B AR baseline perform when distilled from an 8B AR teacher under an on-policy setting?
10 comments
Response to Reviewer 5xz1 2/5
W5&Q2: End-to-end profiling&specifying the number of tokens per trajectory
W5. There is no end-to-end profiling or comparison. The paper reports only per-token decoding latency without specifying the number of tokens per trajectory, the prefill and visual encoding latency, or the proportion of end-to-end latency attributed to decoding; consequently, the actual benefits for real-world deployment cannot be assessed.
Q2. How many tokens does a trajectory contain for the main results? Could the authors provide an end-to-end latency breakdown (visual encoding, prefill, and decoding) for both the AR baseline and the diffusion model?
1.How many tokens does a trajectory contain for the main results?(W5/Q2)
Each trajectory consists of 8 future waypoints, serialized into 83.6 output tokens(mean count in Navtest).
2.Could the authors provide an end-to-end latency breakdown (visual encoding, prefill, and decoding) for both the AR baseline and the diffusion model?(W5/Q2)
Yes. We thank the reviewer for highlighting the importance of end-to-end latency evaluation. To provide a comprehensive latency analysis, we additionally profile visual encoding, LLM prefill, trajectory decoding, and total end-to-end (E2E) latency for both AR and WAM-Diff2 on a single NVIDIA H200 GPU with BF16 precision and batch size 1.
| Model | Vision Encode (ms) ↓ | LLM Prefill (ms) ↓ | Decode (ms) ↓ | E2E (ms) ↓ | E2E P99 (ms) ↓ |
|---|---|---|---|---|---|
| AR | 50.23 | 80.38 | 1817.79 | 1948.98 | 2254.88 |
| Diffusion | 49.02 | 82.97 | 670.11 | 802.10 | 892.14 |
| AR + System Opt. | 31.73 | 20.85 | 474.82 | 527.40 | 579.16 |
| Diffusion + System Opt. | 31.51 | 21.28 | 121.14 | 173.83 | 193.36 |
The results show that decoding constitutes the primary latency bottleneck, accounting for 93.3% of the original AR model's E2E latency. Under comparable system optimizations (FlashInfer and CUDA Graphs), WAM-Diff2 reduces decoding latency from 474.82 ms to 121.14 ms, achieving a 3.92× decoding speedup over the optimized AR baseline. More importantly, this translates into a 3.03× E2E speedup (527.40 → 173.83 ms), with a P99 latency of 193.36 ms, meeting the 200-ms latency budget corresponding to 5 Hz under our benchmark setting.
Response to Reviewer 5xz1 3/5
W3&Q3:Comparison with speculative decoding method on both latency and accuracy
W3. Speculative decoding with parallel block drafters (e.g., DFlash) and its application to driving VLAs (e.g., FlashDrive) are cited in the related work, but no comparison is provided for any of the tasks.
Q3. How does the proposed approach compare with applying speculative decoding (e.g., DFlash and FlashDrive) to the AR baseline, in terms of both latency and accuracy?
1.How does the proposed approach compare with applying speculative decoding (e.g., DFlash and FlashDrive) to the AR baseline, in terms of both latency and accuracy?(Q3/W3)
We thank the reviewer for suggesting this comparison. We applied DFlash to our 2B AR baseline, using the same hardware platform and system optimizations (FlashInfer and CUDA Graphs), and evaluated both latency and planning accuracy. FlashDrive also uses DFlash for speculative reasoning; our experiment evaluates this component within our VLA setting.
| Model | Vision Encode (ms) ↓ | LLM Prefill (ms) ↓ | Decode (ms) ↓ | E2E (ms) ↓ | E2E P99 (ms) ↓ | PDMS |
|---|---|---|---|---|---|---|
| AR | 50.23 | 80.38 | 1817.79 | 1948.98 | 2254.88 | 88.14 |
| AR + System Opt. | 31.73 | 20.85 | 474.82 | 527.40 | 579.16 | 87.99 |
| Diffusion + System Opt. | 31.51 | 21.28 | 121.14 | 173.83 | 193.36 | 87.28 |
| AR + System Opt. + Dflash | 31.73 | 20.85 | 134.96 | 187.64 | 279.53 | 87.99 |
Numerical differences under system optimizations. FlashInfer employs optimized attention kernels with different floating-point reduction orders, which may introduce minor numerical variations. These differences can propagate through autoregressive or confidence-based diffusion decoding, potentially leading to variations in the generated outputs and evaluation metrics.
Under these matched conditions, WAM-Diff2 achieves lower latency across all three measures: decoding latency decreases from 134.96 to 121.14 ms, mean E2E latency from 187.64 to 173.83 ms, and E2E P99 from 279.53 to 193.36 ms. WAM-Diff2 achieves 87.28 PDMS, compared with 87.99 for DFlash.
The results show that speculative decoding substantially accelerates the AR baseline, reducing E2E latency from 527.40 ms to 187.64 ms. However, WAM-Diff2 achieves both lower decoding latency (121.14 vs. 134.96 ms) and lower E2E latency (173.83 vs. 187.64 ms).
More importantly, WAM-Diff2 achieves 30.8% lower P99 latency (193.36 vs. 279.53 ms), demonstrating more predictable inference latency. In contrast, DFlash can suffer from repeated speculation rejections and low acceptance rates in challenging scenarios, leading to substantial tail latency. Notably, WAM-Diff2 maintains its P99 below the 200-ms budget of a 5 Hz driving cycle, highlighting its advantage for latency-sensitive autonomous driving.
Furthermore, WAM-Diff2 performs inference with a single 2B model, without an auxiliary drafter or a separate autoregressive verification step(DFlash introduces an additional 283M parameters approximately 14% additional parameters relative to the 2B baseline, making on-vehicle deployment more demanding.). These results highlight WAM-Diff2’s advantages in tail latency and inference architecture, which is particularly valuable for latency-sensitive autonomous driving applications.
W4:Discussion about continuous action head
W4. For motion planning, a commonly used alternative is a continuous action head (such as the flow-matching trajectory decoder used in Alpamayo-R1), which could be much faster: in the Alpamayo-R1 paper, flow-matching trajectory decoding takes 8.75 ms (5 steps) versus 222 ms for AR trajectory decoding (127 tokens). A discussion of this design is missing.
Thank you for highlighting continuous action heads. Methods such as Alpamayo-R1 and MeanFuser achieve efficient trajectory generation through specialized decoders. However, separating action generation from the VLM introduces distinct optimization objectives and additional challenges in reasoning–action alignment. Recent VLAs, such as AutoVLA and LinkVLA, instead explore unified token-based generation to more tightly couple reasoning and actions.
WAM-Diff2 follows this direction, focusing on accelerating the shared token decoder rather than introducing a separate action head. Our hierarchical AR-to-diffusion distillation enables efficient parallel generation of trajectories, driving VQA, and structured perception outputs while preserving the pretrained VLA's multi-task capabilities. We will clarify this distinction in the revision.
Response to Reviewer 5xz1 1/5
We sincerely thank the reviewer for the detailed feedback. The suggested comparisons have helped us better substantiate our results and clarify our contribution. We respond to each point below.
W1&Q1:Comparison between optimized AR model and optimized Diffusion model
1.What is the decoding latency of the AR baseline when FlashInfer and CUDA Graphs are also applied, measured on the same platform as the diffusion model?(Q1/W1)
We thank the reviewer for pointing out the need for a fair comparison. We applied the same system optimizations (FlashInfer and CUDA Graphs) to the AR baseline and evaluated both models on the same hardware platform.
The results show that WAM-Diff2 achieves a 3.92× decoding speedup over the system-optimized AR baseline (121.14 ms vs. 474.82 ms).
AR and block-diffusion models benefit differently from system optimizations because of their distinct attention execution pattern. PyTorch's native SDPA already provides efficient support for standard causal attention in AR models, whereas custom block-causal attention masks in diffusion models can introduce additional Python/CUDA dispatch overhead. FlashInfer and CUDA Graphs help eliminate these execution bottlenecks, allowing WAM-Diff2 to better realize the computational benefits of parallel decoding. We will update these result to the revise manuscript.
W2:Latency-accuracy trade-off on single platform
Thank you for highlighting the importance of evaluating latency and accuracy on the same hardware platform.We evaluated the AR and WAM-Diff2 checkpoints reported in Table 2 of the main paper on a single Ascend 910C NPU, measuring decoding throughput, latency, and NAVSIM PDMS. Both models use the original PyTorch implementation with BF16 precision and batch size 1, without additional system optimizations.
These same-platform results show a 2.77× decoding speedup while maintaining comparable planning performance (88.3 vs. 88.1 PDMS). The table summarizes representative operating points. Given the figure-upload limitations of this rebuttal response, we will included the full latency–accuracy trade-off curve in the revised manuscript.