- Rating
- 2
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes an efficient inference strategy for multimodal contrastive decoding. The authors propose analyzing token-level changes based on the top-2 logit margin. Based on the margin, they selectively activate the auxiliary forward pass under a user-specified budget.
Critical strengths
- The design is reasonable, as tokens with a large vanilla top-2 margin are unlikely to be changed by MCD.
- The proposed budget is useful for practical systems.
- The paper honestly acknowledges that MCD does not always improve performance and that the proposed method only determines whether to activate MCD.
Critical weaknesses
- The proposed method is essentially a more budget-friendly version of MCD, without improving performance. Moreover, it relies only on the budget, and without a clear performance preference, it is difficult to determine the appropriate budget in practice. Therefore, the contribution is limited.
- The intuition behind the method is simply that low-confidence (small top-2 margin) tokens are more likely to change. This simple idea is expressed through complex mathematical definitions without theoretical contribution.
Questions for authors
- Can the method be improved to provide an indication of potential accuracy improvement, rather than only controlling the computational budget?
- Can this method be applied only to MCD, or can it be extended to other settings?