acceptodds
Under review as a conference paper at ICLR 2027

CARES: How Can Missing-Modality Completion Truly Help Multimodal LLMs?

Abstract

Multimodal large language models (MLLMs) have advanced rapidly in recent years. However, their performance can degrade when input modalities are missing. Recent work has used modality completion to improve the robustness of MLLM inference. However, the completed modality is not always beneficial and can even be misleading, since modality completion may introduce inaccurate or irrelevant information. This raises a question: how can missing-modality completion truly help MLLMs? To this end, we propose Completion-Aware Routing and Evidence Scaling (CARES), a lightweight framework with two modules: completion-aware routing dynamically estimates the contribution of completed modalities at the token level, while evidence scaling adaptively regulates their influence during reasoning. Experiments across four benchmarks and multiple MLLM backbones validate the effectiveness, versatility, and generalization of the proposed CARES.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multimodal Large Language Models (MLLMs), Missing-Modality Inference, Adaptive Attention Regulation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_5ci9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Completion-Aware Routing and Evidence Scaling (CARES), a lightweight framework designed to improve multimodal large language model (MLLM) inference under missing-modality conditions. The authors observe that directly incorporating retrieved or generated modalities may introduce inaccurate or irrelevant information, potentially degrading downstream performance. To address this issue, CARES introduces a shared, state-conditioned gating module that estimates the contribution of completed modalities using query representations, source-specific representations, and attention statistics. The predicted coefficients are then used to reweight attention allocated to completed modalities while keeping the MLLM backbone frozen. Experiments across four benchmarks, multiple MLLM backbones, and two completion approaches demonstrate performance improvements under various missing-modality settings.

Critical strengths
  1. The paper identifies a practically relevant limitation of modality completion.

The authors highlight that completed modalities do not necessarily improve downstream predictions and may even introduce misleading information. This observation is relevant to robust multimodal inference, particularly when retrieved or generated content is unreliable. The proposed framework follows a clear intuition: completed evidence should not necessarily receive the same treatment as directly observed information.

  1. The experimental evaluation is relatively comprehensive and demonstrates promising empirical results.

The experiments cover four datasets, multiple missing-modality configurations, several MLLM backbones, and both retrieval-based and generation-based completion methods. The authors also provide ablation studies on routing granularity and gating features, along with evaluations of cross-dataset and cross-completion transfer. These experiments offer useful empirical evidence for the effectiveness of the proposed framework.

Critical weaknesses
  1. The problem formulation and motivation are insufficiently justified, particularly regarding the necessity of modality completion.

Although the paper identifies a potential limitation of modality completion, it does not sufficiently establish why modality completion is necessary or preferable to alternative approaches for handling missing modalities. The Introduction primarily categorizes existing approaches and discusses their limitations, making much of the discussion resemble a Related Work section rather than a focused motivation for the proposed problem. More importantly, the paper does not clearly explain under which circumstances missing-modality completion provides indispensable information that cannot be effectively handled by the available modalities alone. Figure 1 demonstrates that directly incorporating completed modalities may degrade performance. However, showing that an existing completion-based method can fail does not necessarily establish the necessity of developing an additional mechanism to regulate completed evidence. A more fundamental question remains insufficiently addressed: why should an MLLM rely on potentially unreliable reconstructed modalities rather than directly adapting its inference process to the available inputs? Consequently, while the technical objective of regulating completed evidence is understandable, the practical significance and necessity of the overall problem setting remain insufficiently motivated.

  1. The methodological novelty is limited, as the proposed framework largely combines established gating and attention-reweighting mechanisms.

The core methodology consists of predicting a scaling coefficient through a lightweight MLP and applying it to the attention mass associated with completed modalities, followed by normalization. These components closely resemble established ideas in adaptive gating and attention regulation. In particular, Equation (8) shows that the proposed evidence scaling is mathematically equivalent to adding a learned bias to the attention logits of completed-source tokens. Although the conditioning features and parameter-sharing strategy are tailored to the missing-modality setting, the underlying computational mechanism does not appear fundamentally different from existing attention reweighting approaches. The paper discusses several related methods, including Flamingo, PAI, and VAR, but the distinction is largely based on the application scenario and the sources being regulated, rather than a substantially new methodological principle. Therefore, the contribution appears primarily incremental, and the paper does not sufficiently demonstrate why its particular formulation represents a meaningful methodological advance over existing gating and attention-based adaptation techniques.

Questions for authors

1-Insufficiently controlled comparisons. The evaluation lacks a supervised available-only baseline with a comparable parameter budget, making it difficult to isolate the benefits of modality completion from those of task-specific adaptation.

2-Limited comparisons with alternative mechanisms. Comparisons with standard gating or lightweight attention-reweighting methods under identical training conditions would better establish the advantages of CARES.

3-Insufficient analysis of completion quality. Figure 3(d) shows different routing coefficients across completion-quality levels, but the corresponding downstream performance is not reported.

4-Restrictive attenuation mechanism. Equation (4) only allows attenuation of completed evidence. Why is amplification excluded, particularly when completed evidence is more informative than the available inputs?

5-Unclear inference efficiency. Although CARES introduces few trainable parameters, token-level gating may incur additional computational overhead. Inference latency and memory comparisons would be helpful.

6-Limited generalization evaluation. Table 4 evaluates transfer under matched missing-modality configurations. Can CARES generalize to unseen configurations or different MLLM backbones?

7-Practical deployment concerns. Can a single Gate handle multiple missing-modality configurations rather than requiring configuration-specific adaptation?

Reviewer_c2drOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies modality completion for multimodal models with incomplete inputs. It highlights that completed modalities may contain erroneous or irrelevant information and consequently interfere with predictions. To address this issue, the proposed method selectively incorporates completed evidence by attenuating its influence relative to the original inputs.

Critical strengths
  1. Clear motivation: The paper addresses the risk that modality completion may introduce misleading information and degrade predictions, shifting the focus from how to complete missing modalities to how to use the completed content.
  2. Useful insights into the use of completed evidence: Reweighting reduces the average attention allocated to completed sources while improving task performance. The results also suggest that evidence incorporation should depend on both the input sample and the local computational state.
Critical weaknesses
  1. Limited methodological novelty and insufficient comparison with existing gated attention methods. The core design consists of dynamic gating and source-level attention reweighting. The paper does not sufficiently establish what distinctive advantages its particular design offers over existing attention gating approaches.
  2. Supervised adaptation confounds the attribution of improvements. CARES is trained using downstream labels, whereas the available-only baseline, direct completion, and PE do not involve downstream training. Consequently, the improvements cannot be attributed entirely to the proposed evidence-regulation mechanism.
  3. The conclusions have a restricted scope, leaving broader generalizability insufficiently established. The main behavioral experiment constructs differences in evidence quality using texts with matching, neutral, or opposite sentiment. This supports task-relevance-based regulation but does not adequately distinguish factual correctness, semantic relevance, and label consistency.
Questions for authors
  1. How does the proposed method differ from existing gated attention approaches? Have direct comparisons been conducted? Does the proposed MLP-based attention-scaling architecture outperform existing gated attention methods under comparable conditions?
  2. CARES is trained using downstream labels, whereas the available-only baseline, direct completion, and PE are not. To what extent do the gains arise from access to supervised training data rather than the proposed architecture?
  3. Why is completed evidence only allowed to be attenuated? Could selectively amplifying useful completed information yield further improvements?
  4. Does the Gate learn evidence reliability, or primarily consistency with sentiment labels?
Reviewer_RN7dOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how frozen multimodal LLMs should use potentially unreliable missing-modality completions. CARES introduces a small shared gate conditioned on the current query, source-specific attention summaries, and relative attention mass. It attenuates completed sources and renormalizes attention while preserving within-source token ratios at each attention operation. The gate is trained with downstream task supervision. Experiments span four benchmarks, multiple missing-modality configurations, retrieval-based and generative completion, and several backbones, with granularity ablations and limited cross-dataset/provider transfer. The distinctive contribution is fine-grained, state-dependent regulation of completed evidence inside a frozen MLLM, rather than completion itself.

Critical strengths
  1. A concrete technical advance over completion-only pipelines and coarse selection. MissRAG (Pipoli et al., ICCV 2025) supplies retrieved missing-modality content, while DyMo (Du et al., ICLR 2026) already addresses the discard-versus-impute dilemma through adaptive selection of recovered modalities. CARES contributes a different intervention: a shared gate producing query-, head-, and layer-dependent source weights inside a frozen MLLM. The source-mass formulation and its equivalent logit bias make the intervention clear. Importantly, Table 2 trains coarse alternatives under the same budget: on Qwen/MissRAG, MOSI average accuracy rises from 70.77 for sample-level gating and 75.75 for head-level gating to 77.63 for full gating. Thus, the paper provides evidence for its specific granularity choice, beyond simply observing that unreliable completion should be filtered.

  2. Broad and largely consistent empirical gains. Table 1 evaluates both retrieval and generation providers across sentiment, intent, genre classification, and audiovisual QA. For example, Qwen/MOSI with missing text goes from 72.45 with available inputs to 49.27 after MissRAG completion, then 81.92 with CARES. Reusing identical completed content within each pipeline is a useful control. The additional backbone and transfer experiments further support usefulness beyond one model/provider, although they do not establish universal zero-shot deployment. These results support a weak acceptance despite the comparison limitations below.

References: MissRAG; DyMo.

Critical weaknesses
  1. The main comparison does not fully isolate completion regulation from downstream adaptation. CARES uses task labels, whereas direct completion and prompting do not. The parameter-matched tuning control is helpful, but it scales only the final hidden representation; matching parameter count does not make this an equally effective alternative to intervention throughout attention. Its MM-IMDb performance is also substantially below the available-only baseline. Consequently, the conclusion that the gains go beyond task supervision alone is stronger than this single control establishes. Relative to RAGPT (Lang et al., AAAI 2025), which combines retrieval/completion with supervised dynamic prompts, and DyMo's adaptive selection, the empirical advantage over alternative learned ways of handling unreliable completion remains uncertain. These methods are not drop-in equivalents, so I would prioritize a controlled adapted comparator on the same backbone and completed inputs, rather than demand an unfair cross-architecture leaderboard comparison. An equally supervised available-only baseline is also needed to quantify the incremental value of completion itself.

  2. Parameter efficiency is better established than practical efficiency and deployment generality. The roughly 3.5K-parameter shared gate is attractive, but it is evaluated at each attention query/head/layer and requires source-specific aggregation. Parameter count alone does not determine prefill/decode latency, memory usage, or compatibility with optimized attention kernels. Moreover, the main results train separate gates for dataset/backbone/provider/missing-pattern settings; the transfer experiments cover only a limited subset. This qualifies how broadly the proposed lightweight framework can be deployed without retraining. The concern does not negate the reported accuracy gains, but it limits their practical interpretation.

Reference: RAGPT.

Questions for authors
  1. Comparison that could affect the recommendation: With the same backbone, data, losses, and validation budget, how does CARES compare with a small attention-level adaptation or learned prompt baseline, both with completed inputs and with available inputs only? A representative subset would help establish whether completion-aware routing, rather than the placement of supervised adaptation, is decisive.

  2. Efficiency clarification: What are the measured prefill latency, decoding throughput, and peak memory overhead relative to direct completion at matched sequence lengths? Does the implementation retain optimized attention kernels, and what is the amortized cost when separate gates must be trained for different missing patterns?

  3. Interpretation of reliability: Figure 3(d) is suggestive, but the Correct/Wrong conditions also change agreement with the target sentiment. Can you provide variability and downstream accuracy for these conditions, and clarify whether the gate tracks factual/instance-level correctness beyond label agreement? The reduced completed-source mass in Figure 3(c) is expected from the attenuation rule itself and is not independent evidence of reliability estimation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.