- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
In Vision-Language Models (VLMs), a trainable connector bridges the gap between the frozen vision encoder and the language decoder. The paper investigates the mutual information of connector families using single Linar layer, two layers and query connectors applied in frozen encoder and decoder scenario.
Critical strengths
The paper analyzes information transfer from frozen vision encoders to language decoders across diverse VLM architectures like LLaVA, BLIP-2, Flamingo, and Qwen3-VL. Their study systematically compares how various connector types facilitate this process.
Critical weaknesses
- The experimental setup and key findings are hard to verify, as core definitions and parameters are insufficiently explained, see questions below.
- Figures are not clearly explained what to be seen. For instance Figure 1, ist firstly refered in the introduction without symbold explanation in text or caption. The symbols are eather not explained or only explained in later text passages or appendix. In the experimental section the textual description is inprecise, as only the overall figure is refered, while a specific subfigure is been meant.
- Equation 2 suffers from structural inconsistency. While X,Z, and Y represent features connected by functions (represented by arrows), T represents the language model, which is a function. Yet it is incorrectly rendered as a feature node.
- the training settings are not reproducably explained.
- The experimental results could be extended further. For example, in Figure 4a, the sample size is evaluated at only two data points, whereas Figure 4c omits the query-based approach.
Questions for authors
- how do you define the readable set U and how is it related to token pruning?
- How do you verify the LLM-decoder alignment given the mutual feature-subspace from the connector, as the decoder is non-retrainable. Simply maximizing the mutual information in Z^~ is no guaratnee for small channel decoder alignment.
- How exactly is the connector been trained? I assume only the caption loss form the teacher results are applied to Z^~, so before LLM-decoder?
- Regrading the training-settings: On what captions in specific do you train? How many samples are generated for training?
- How is the ortogonal Projector \PI_U been created? Is the projector a channel masking or is the feature been converted into leading subspace of PCA?
- How is the "usable Information"-function I(Z,Y) Page 4 been defined? Later on page 6 you define the mutual information function, but it is arguably not clear if these functions are the same.
- What is the difference between m and me in signal mass Eta? I cannot find a definition of me. Why do you have 2 definitions for signal mass eta (1. Sum over alpha and 2. formula to number of keys me, N and m)?
- in Theorem 4: What is meant by risk? Is it the computational complexity?