- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper investigates how multimodal (MM) and text-only (TXT) supervision affect draft–target alignment in multimodal speculative decoding. The authors decompose distributional disagreement into text-conditioned and visual-response risks, and demonstrate that these component risks can guide training data selection. To combine MM and TXT samples, they propose a mean-normalized risk selection strategy (MNRS-Joint) that enables joint selection without predefined modality quotas. Experiments on LLaVA-1.5-7B and Qwen2.5-VL-3B show improvements in accepted length over several baselines.
Critical strengths
- The proposed decomposition provides insights into how different supervision modalities affect draft–target alignment. The experiments further demonstrate that visual-response risk can serve as a effective selection criterion.
- The paper includes controlled comparisons across different supervision settings, risk-based sampling strategies, and normalization methods. The consistent improvements across multiple benchmarks and two vision-language models provide empirical support for the proposed approach.
Critical weaknesses
- The proposed mean normalization is relatively simple, and its advantages over other potential cross-modal selection strategies are not fully established. Comparisons with percentile-based ranking or stronger modality-ratio baselines would better justify the proposed design.
- The experiments primarily report accepted length rather than actual inference latency or throughput. Moreover, the computational overhead of estimating the proposed risk scores is not quantified, making the practical efficiency trade-off unclear.
- The relationship between total variation distance and token rejection probability is established under stochastic speculative sampling, whereas the experiments employ greedy decoding with dynamic draft trees. The theoretical connection to the actual evaluation setting therefore requires further justification.
- Although the method is evaluated on two vision-language models, both experiments rely on EAGLE-style draft architectures and the same training data sources. Furthermore, the lack of results across larger models makes it difficult to assess the stability of the reported improvements.
Questions for authors
- How does MNRS-Joint compare with selecting high-risk samples under different fixed MM/TXT ratios or using percentile-based cross-modal rankings?
- Could the authors provide actual inference throughput or latency measurements, along with an analysis of the computational overhead introduced by risk-based data selection?
- Are the reported improvements consistent across multiple random seeds, particularly the gain of MNRS-Joint over the Fixed 1:1 High baseline?