- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies how multimodal language models use retrieved knowledge during generation. It proposes Dynamic Contribution Steering (DCS), which adjusts the contributions of attention, feed-forward, and residual pathways using attention distributions and positive-activation ratios. Knowledge Layer Amplification (KLA) further combines a selected intermediate-layer representation with the final layer through entropy-based weighting. Experiments across several knowledge-intensive VQA benchmarks and model families report consistent accuracy improvements with near-baseline inference latency.
Critical strengths
-
Broad empirical evaluation. The method improves performance across multiple models, benchmark settings, and answer formats. Additional evaluations with newer MLLMs and different retrievers strengthen the evidence that its benefits are not restricted to one configuration.
-
Practical inference-time design. The approach requires no additional training and avoids a separate no-context backbone pass. Table 4 reports approximately baseline latency, making the combination of accuracy improvements and low measured overhead potentially useful.
Critical weaknesses
-
The adaptive weighting rules are not sufficiently validated. Although the weights vary with the input, their mapping is fixed: greater relative attention to retrieved text increases the attention branch’s weight, while a higher positive-activation ratio reduces the FFN branch’s weight. The paper does not directly establish that these statistics indicate whether the corresponding updates help or harm evidence-grounded answering. In particular, attention to text does not establish its reliability, and positive-activation counts do not measure harmful parametric interference. The component ablations demonstrate practical gains, but matched fixed-weight controls and an isolated KLA-only evaluation are needed to substantiate the specific adaptive design.
-
The evaluation leaves the method’s applicability and incremental benefit insufficiently established. Results are not separated by whether retrieved text contains the answer, is irrelevant, or conflicts with visual evidence. The retriever variations partially address retrieval quality but do not isolate these conditions, especially cases where preserving visual information is essential. Several gains over ALFAR are modest, including 0.55 and 0.72 percentage points on LLaVA-1.5 Human and Validation overall accuracy, making comparison consistency important. However, Section 4.1 describes baseline results as cited from ALFAR, whereas Appendix A6 describes re-implementation. The timing results are encouraging, but the measurement setup and inclusion of KLA-related computation also require clarification.
Questions for authors
-
Mechanistic validation: Do attention ratios and positive-activation ratios predict whether the corresponding pathway update helps or harms evidence-grounded answering? How does the adaptive method compare with matched fixed weights, and what is KLA’s contribution when evaluated alone?
-
Applicability: Can you separate results for answer-containing, irrelevant, and conflicting retrieved contexts? Does suppressing visually dominated updates remain beneficial when accurate answering depends on fine-grained visual evidence?
Kind Reminder to Reconsider the Review Score
Dear Reviewer,
Thank you again for your time and constructive feedback. We sincerely hope that our responses have adequately addressed your concerns.
We would greatly appreciate it if you could take another look at our rebuttal and reconsider your overall evaluation and score in light of our clarifications. Please let us know if any concerns remain, and we would be happy to provide further clarification.
Thank you for your consideration.