- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Completion-Aware Routing and Evidence Scaling (CARES), a lightweight framework designed to improve multimodal large language model (MLLM) inference under missing-modality conditions. The authors observe that directly incorporating retrieved or generated modalities may introduce inaccurate or irrelevant information, potentially degrading downstream performance. To address this issue, CARES introduces a shared, state-conditioned gating module that estimates the contribution of completed modalities using query representations, source-specific representations, and attention statistics. The predicted coefficients are then used to reweight attention allocated to completed modalities while keeping the MLLM backbone frozen. Experiments across four benchmarks, multiple MLLM backbones, and two completion approaches demonstrate performance improvements under various missing-modality settings.
Critical strengths
- The paper identifies a practically relevant limitation of modality completion.
The authors highlight that completed modalities do not necessarily improve downstream predictions and may even introduce misleading information. This observation is relevant to robust multimodal inference, particularly when retrieved or generated content is unreliable. The proposed framework follows a clear intuition: completed evidence should not necessarily receive the same treatment as directly observed information.
- The experimental evaluation is relatively comprehensive and demonstrates promising empirical results.
The experiments cover four datasets, multiple missing-modality configurations, several MLLM backbones, and both retrieval-based and generation-based completion methods. The authors also provide ablation studies on routing granularity and gating features, along with evaluations of cross-dataset and cross-completion transfer. These experiments offer useful empirical evidence for the effectiveness of the proposed framework.
Critical weaknesses
- The problem formulation and motivation are insufficiently justified, particularly regarding the necessity of modality completion.
Although the paper identifies a potential limitation of modality completion, it does not sufficiently establish why modality completion is necessary or preferable to alternative approaches for handling missing modalities. The Introduction primarily categorizes existing approaches and discusses their limitations, making much of the discussion resemble a Related Work section rather than a focused motivation for the proposed problem. More importantly, the paper does not clearly explain under which circumstances missing-modality completion provides indispensable information that cannot be effectively handled by the available modalities alone. Figure 1 demonstrates that directly incorporating completed modalities may degrade performance. However, showing that an existing completion-based method can fail does not necessarily establish the necessity of developing an additional mechanism to regulate completed evidence. A more fundamental question remains insufficiently addressed: why should an MLLM rely on potentially unreliable reconstructed modalities rather than directly adapting its inference process to the available inputs? Consequently, while the technical objective of regulating completed evidence is understandable, the practical significance and necessity of the overall problem setting remain insufficiently motivated.
- The methodological novelty is limited, as the proposed framework largely combines established gating and attention-reweighting mechanisms.
The core methodology consists of predicting a scaling coefficient through a lightweight MLP and applying it to the attention mass associated with completed modalities, followed by normalization. These components closely resemble established ideas in adaptive gating and attention regulation. In particular, Equation (8) shows that the proposed evidence scaling is mathematically equivalent to adding a learned bias to the attention logits of completed-source tokens. Although the conditioning features and parameter-sharing strategy are tailored to the missing-modality setting, the underlying computational mechanism does not appear fundamentally different from existing attention reweighting approaches. The paper discusses several related methods, including Flamingo, PAI, and VAR, but the distinction is largely based on the application scenario and the sources being regulated, rather than a substantially new methodological principle. Therefore, the contribution appears primarily incremental, and the paper does not sufficiently demonstrate why its particular formulation represents a meaningful methodological advance over existing gating and attention-based adaptation techniques.
Questions for authors
1-Insufficiently controlled comparisons. The evaluation lacks a supervised available-only baseline with a comparable parameter budget, making it difficult to isolate the benefits of modality completion from those of task-specific adaptation.
2-Limited comparisons with alternative mechanisms. Comparisons with standard gating or lightweight attention-reweighting methods under identical training conditions would better establish the advantages of CARES.
3-Insufficient analysis of completion quality. Figure 3(d) shows different routing coefficients across completion-quality levels, but the corresponding downstream performance is not reported.
4-Restrictive attenuation mechanism. Equation (4) only allows attenuation of completed evidence. Why is amplification excluded, particularly when completed evidence is more informative than the available inputs?
5-Unclear inference efficiency. Although CARES introduces few trainable parameters, token-level gating may incur additional computational overhead. Inference latency and memory comparisons would be helpful.
6-Limited generalization evaluation. Table 4 evaluates transfer under matched missing-modality configurations. Can CARES generalize to unseen configurations or different MLLM backbones?
7-Practical deployment concerns. Can a single Gate handle multiple missing-modality configurations rather than requiring configuration-specific adaptation?