- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper asks whether a learned prompt-composition signal in a representation-level adapter actually influences a frozen VLM's output at inference time. The audited adapter is right before the LM head. The paper breaks the path from prompt to prediction into stages and measures the change at each stage under several interventions. In experiments, Dynamic-to-Uniform changes the conditioning state, but the router weights and accuracy barely change.
Critical strengths
None.
Critical weaknesses
- The main result is an almost null effect presented as a finding, and the experiments are too limited to support it. The experiments are small and narrow (just 900 training and 100 evaluation examples), Table 1 is a single seed, and there's one small VLM. Also, the evaluated model predicts option A for every example (Sec 5.6), so the "no prediction change" results are not meaningful. "Norm-matched random directions" are mentioned as a control in Section 6, but no results are reported.
- The writing and presentation of this paper are really hard to follow, so I could not verify the claims. The paper contradicts itself about prompt tuning (Sec 5.1 vs Appendix A.1.1), notation is used before it's defined (see questions), components and configurations are never explained (VEFT, KSL, KLAFT, PAAF, etc.), and Figure 1 does not match the experiments.
Questions for authors
- Did the authors check Figure 1 before submitting it? Is this really the final version you want to present?
- What is in Section 3? Did the authors ever define the variables before using them?
- This paper cites two anonymous papers under review. Can you provide more self-contained details of the two papers since they are not available to reviewers?
- What exactly is "Dynamic-to-Uniform composition"? I guess it is defined in Section 4.1 but why the authors use it without explaining it in the abstract?
- What is in line 148? Is it the defined in Eq.2?
- In Section 5.6, all predctions select option A. Are the same accuracies in Table 1, just because the model always answers A? Does the result still hold with a model that does not always predict the same option?
- Section 6 (line 477) says "norm-matched random directions provide an additional specificity control." but I don't think there are results for this... Where are the results?
- Section 5.1 says input-levle prompt tuning is omitted but Apendix A.a.a says role-aware prompts are prepended to the input and trained. What is true?