- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates whether discrete diffusion vision-language models need continuous access to visual information throughout the denoising process. The authors analyze how visual influence changes over time through visual KV removal experiments, further investigate the relationship between the generation state and future visual requirements, and propose a lightweight predictor to determine when visual computation can end. Experiments show that the method can substantially reduce vision-related computation while largely maintaining task performance, with some cross-model transferability.
Critical strengths
- The research question is clear, and the approach is coherent. The paper centers on the question of how long a model needs to see, progressing from intervention experiments that directly remove visual KV to representation analysis, and then applying the findings to visual exit decisions. This establishes a relatively complete connection between the analysis of the observed phenomena and the practical method.
- The experiment demonstrating the predictive ability of hidden states is interesting. Through experiments on representation changes, the authors show that visual information enters the hidden states involved in generation. Meanwhile, after controlling for the denoising step and current visual influence, the authors find that states with similar hidden representations have more similar future visual requirements. This indicates that the generation state provides predictive signals.
Critical weaknesses
- The choice of predictor input features lacks direct validation. The right panel of Figure 7 shows that using only the mean hidden state yields smaller differences in future visual requirements among neighbors than using the combination of the mean hidden state and the state difference. However, the final predictor still uses the mean hidden state, the state difference, and generation progress. Is this contradictory? In addition, the paper lacks corresponding ablations, making it difficult to determine how much improvement the state difference and generation progress each contribute.
- Issues with the central argument and figures. The authors interpret generation representations moving progressively closer to visual representations as the internalization of visual information, but the connection between this interpretation and changes in visual requirements over time is not fully developed. In particular, visual influence may first decrease and then increase again, and the authors need to explain how this phenomenon relates to the narrative of gradual internalization. The left and middle panels of Figure 1 also lack clear definitions of the axes and graphical elements, making it difficult to understand their specific meanings and the corresponding experimental evidence.
Questions for authors
- Could the authors provide ablations of the predictor input features, comparing generation progress alone, the mean hidden state alone, the mean hidden state combined with generation progress, and the full input? How do these choices affect prediction error, task performance, and the effectiveness of visual exit?
- Could the authors clearly explain the axes, curves, and magnified inset in the left and middle panels of Figure 1, along with the experimental evidence underlying them? Please also explain the relationship between generation representations moving closer to the visual representational region and the non-monotonic changes in visual influence.