acceptodds
Under review as a conference paper at ICLR 2027

Risk-Controlled Correction for Frozen Multimodal Models under Missing Modalities

Abstract

A deployed multimodal predictor may face missing modalities after its weights can no longer be changed. This raises a question that training-time repair does not answer: what part of the resulting error is still correctable from the information that remains? We show that this part is characterized by the Bayes gap, the difference between the model's prediction and the Bayes prediction given the observed modalities. For cross-entropy and squared error, excess risk measures this gap in the geometry of the loss; ambiguity in the observed modalities cannot be removed. Thus, a badly degraded prediction is not necessarily a correctable one. Estimating the Bayes prediction normally requires labels. However, when a model observing more modalities is conditionally correct, averaging its outputs conditional on the remaining modalities recovers this target without retrieval labels; with the same fixed blend and neighbors, it is no worse in expected cross-entropy or Brier risk than averaging hard labels. Some such assumption is unavoidable: otherwise the target is not identifiable from unlabeled data alone. GUARD (Gated Utility-Aware Recalibration for Deployment) leaves the deployed predictor unchanged. It estimates a candidate correction by retrieval, learns from deployment-matched labeled data when correction is likely to help, and calibrates a threshold on a held-out split by conformal risk control. Under exchangeability, the marginal probability that GUARD applies a correction whose loss increase exceeds a prescribed tolerance is bounded by a preset level, regardless of the learned score. In controlled experiments, correction gain follows headroom rather than raw error; across nine frozen model-dataset pairs spanning six domains, GUARD retains most of blanket correction's gain within harm budgets that blanket correction often exceeds.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
multimodal learning, missing modalities, post-hoc correction, conformal risk control, selective intervention, frozen models, uncertainty quantification

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_t4xYOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies post-hoc correction for frozen multimodal models under missing modalities and proposes GUARD. It characterizes correctable error as the Bayes gap, which distinguishes the model's overall error from the portion that can be corrected using only the currently observed modalities, and replaces retrieval labels with the outputs of a richer-view model when that model is conditionally correct. On this basis, GUARD constructs a candidate correction and then calibrates a threshold via conformal risk control to limit harmful interventions. The paper provides theoretical proofs and validates the performance of the proposed approach through experiments.

Critical strengths

The paper distinguishes between "correctable error" and "irreducible ambiguity," and on this basis establishes a risk-controlled post-hoc correction framework. The Bayes-gap analysis convincingly shows that the observed magnitude of performance degradation does not directly determine the model's correctable headroom; GUARD further decouples the construction of a candidate correction from the decision of whether to apply it, and uses conformal risk control to provide a finite-sample guarantee on the marginal joint probability of harmful intervention. This makes the method attend not only to average performance gains but also to explicitly constraining the negative risk that correction may introduce.

Critical weaknesses

The paper's core methodological increment relative to existing retrieval-based correction, selective prediction, and conformal risk control methods remains insufficiently clear. GUARD combines correction, a learned action score, and conformal calibration, and further controls joint harm through fit-split tightening. The authors have conducted several ablation studies, including comparisons with confidence scores, Learn-then-Test, different correction targets, and retrieval versus probe variants. However, these experiments still cannot fully distinguish whether GUARD's performance gains stem from its specific overall design or from a combination of existing correction methods and risk-calibration mechanisms.

Questions for authors

If the same correction candidate is fixed, and only GUARD’s learned action score together with the conformal threshold are compared against simpler selective or conformal gating baselines, do GUARD’s performance gains still hold? The authors are encouraged to further clarify the respective effects of the learned action score, conformal calibration, and other key components on the final performance gains and joint harm.

Reviewer_Z3gAOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies post-hoc correction of frozen multimodal models under missing modalities. It characterizes correctable error through the Bayes gap and proposes GUARD, combining retrieval-based correction, utility-based gating, and conformal risk control. Under exchangeability, GUARD bounds the marginal probability of harmful intervention. Experiments cover nine model–dataset pairs across six domains.

Critical strengths
  1. The problem is well motivated. Separating correction from intervention provides a coherent framework for improving frozen models while controlling harm.

  2. The theoretical analysis clarifies correctable error, conditions for transferring richer-view predictions, and the scope of the finite-sample risk guarantee.

  3. The evaluation spans diverse tasks and architectures, with extensive ablations on scoring, calibration, correction targets, data scale, and computational overhead.

Critical weaknesses
  1. GUARD requires deployment-matched labels for fitting and calibration, while its default retrieval target also uses labels. Although the appendix examines data scale and distribution shifts, effectiveness under a constrained total labeling budget and changing deployment conditions remains insufficiently established.

  2. The calibration comparison does not fully isolate GUARD’s advantage. Full GUARD includes threshold tightening that the baselines lack; without it, Learn-then-Test achieves lower empirical joint harm on all seven benchmarks with comparable gains (Table 3). Since the guarantees also differ, comparisons using matched scoring features and the same tightening procedure would better establish the benefit of GUARD’s calibration choice.

Questions for authors

The clear problem formulation, theoretically grounded design, and broad empirical evaluation support acceptance. However, the deployment supervision requirements and incomplete isolation of the calibration advantage limit the strength of my recommendation.

Reviewer_GBurOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates post-hoc correction of frozen multimodal predictors under missing modalities. It first distinguishes between error and correctable error through the Bayes gap. The paper then derives an exact relation between the Bayes gaps of nested modalities, providing theoretical justification for using richer-view predictions as retrieval targets. Based on these findings, the paper proposes GUARD, a framework that separates candidate correction from intervention selection. A candidate prediction is constructed using nearest-neighbor retrieval and blended with the temperature-scaled frozen output. A learned action score and conformal risk control are used to decide whether the correction is applied. The resulting guarantee bounds the marginal probability of applying a harmful correction. The experiments demonstrate improvements over frozen predictors while reducing harmful interventions compared with blanket correction in several settings.

Critical strengths

-Problem setting: Unlike existing methods that focus mainly on improving prediction performance, GUARD explicitly controls the risk of harmful corrections without frozen model. The experiments show that it can retain most of the performance gains while significantly reducing harmful interventions. -Theoretical analysis: The paper provides a clear theoretical distinction between error and correctable error. It also explains when predictions from richer modalities can be used to correct predictions under missing modalities, providing theoretical support for the proposed correction strategy.

Critical weaknesses

-Novelty: Although the proposed framework is valid, it is unclear to me which part constitute the substantial methodological contribution. It mainly combines existing techniques, including nearest-neighbor interpolation, a learned action score, and conformal risk control. It sounds to me more like combining and applying existing methods to the post-hoc frozen model getting. -The practical significance of joint harm: GUARD controls the joint probability of harmful interventions rather than the harm rate of all applied corrections. As shown in the IEMOCAP experiments, 83.15% of applied corrections can be harmful while the joint harm constraint is still satisfied. Although the authors acknowledge this limitation, it raises concerns about whether the method remains reliable in practical cases.

Questions for authors

-Reliability of applied corrections: In several IEMOCAP settings, a large proportion of applied corrections are harmful. Could the authors explain why the learned gate fails to identify usseful corrections? A comparison of conditional harm at matched intervention rates would help assess whether the learned score provides better intervention selection. -Labeled data: Although cross-mask transfer removes the need for retrieval labels, GUARD still requires labeled data. Could the authors clarify the practical advantage of cross-mask transfer with the same amount of labeled data required? The linear probe experiments in Tables 14–15 also show competitive or even better performance than retrieval-based correction on several datasets. Could the authors clarify the practical advantages of the retrieval-based approach over a simple supervised correction head trained on the same frozen representations, particularly when labeled data are limited?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.