- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper aims to accelerate inference in diffusion vision language models by jointly reducing the number of visual tokens and denoising iterations. It is motivated by the observation that the number of visual tokens are much bigger than that of prompt tokens and when visual context is compressed, using fewer generation steps can help preserve model accuracy. Based on these observations, this paper proposes PACE, which combines step-count adjustments with token pruning. PACE retains visual tokens that receive high attention scores from the prompt tokens and from responses tokens that are unlikely to become filler tokens. The remaining visual tokens are pruned, while if there is a remaining token that has sufficiently similar direction with the pruned one, it will be merged to the remaining token (spatial lever). PACE uses a different number of generation steps depending on the task. Experiments are conducted on benchmarks with output lengths of 16 tokens or fewer, and the results show that PACE preserves accuracy better than the baselines, reducing computational cost.
Critical strengths
- PACE preserves model accuracy better than other strong baselines even under aggressive token pruning, while also achieving fast inference.
Critical weaknesses
- Non-consistent setups for experiments and the results are only limited to short responses (16 or fewer).
- There are only r=0.75 results for LaViDa 8B model.
- No experiment with longer response tasks.
- Incomplete related work discussions.
- In terms of token pruning, the only difference between DVLMs and DLLMs is that visual tokens are concatenated in the front. State of the art DLLM inference acceleration methods should be considered as related works.
- accuracy score might improve with semi autoregressive decoding on K=16.
- when multiple tokens are generated per step, confidence aware parallel decoding (Wu et al. ICLR 2026) should be considered.
- Token pruning methods like Elastic-Cache (Nguyen-Tri et al. ICLR 2026) or DyLLM (Lee et al. ICML 2026) should also be studied.
Questions for authors
- The accuracy with lower K might be because there are less steps to be affected by the error caused by token pruning. Does PACE also work for tasks that require hundreds of output tokens?
- Is the trend shown in Figure 2 also valid in longer response generations?
- How could be the latency result of PACE-S and PACE-FULL same on generation tasks with r=0.5?
- Could you provide the accuracy results for LLaDA-V 8B with r=0.75 and LaViDa 8B with r = 0.5, 0.25, and 0.1?
- More breakdown of PACE's acceleration would be helpful to understand the results. For example, the lowest TFLOPs options were not always the fastest. It can happen because some operations are memory bounded, not computation bounded.
- Is PACE's temporal lever more efficient than confidence aware parallel decoding or other related methods that reduces the number of generation steps?
- Is PACE's spatial lever more efficient than DyLLM's salient token selection setting the cosine similarity threshold differently between text tokens and visual tokens? How about Elastic-Cache?