- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes EASE, a method for handling missing modalities in multimodal large language models that learns sample-specific corrections from available inputs within a fixed low-dimensional space constructed from paired training representations. The authors conduct an analysis across different models and tasks and find that interventions in a low-rank subspace can approximate the label-score changes produced by restoring complete-input representations at the same intervention site. They evaluate the proposed method on three benchmarks and show that it outperforms most baselines across the evaluated settings.
Critical strengths
-
The authors analyze multiple MLLM backbones and show that low-rank interventions can preserve the decision effects of complete-input representations.
-
The experimental results are strong, and the proposed method outperforms most baselines across three benchmarks.
-
The paper is well-written and easy to follow.
Critical weaknesses
-
The method trains a separate editor for each missing-modality condition, which increases the training cost. However, the authors do not provide a comparison with jointly training a single editor across different missing-modality conditions. It is unclear whether training separate editors is necessary or provides any advantage over joint training.
-
Although the authors analyze interventions across four MLLM backbones, the proposed EASE method is only trained and evaluated on Emotion-LLaMA-v2. This makes it difficult to assess how well the method generalizes to other backbones.
-
The choice of intervention layers is not clearly justified. The authors use different layers for different missing-modality conditions but do not explain how these layers were selected or provide an ablation study on the effect of intervention depth. Since the effectiveness of representation editing may depend on the selected layer, this is an important missing analysis.
-
The text in Figure 2 is very small and difficult to read.
Questions for authors
-
Although the authors analyze low-rank interventions across four MLLM backbones, EASE is only evaluated on Emotion-LLaMA-v2. How well does the proposed method generalize to other backbones? Without evaluating EASE on additional models, it is difficult to determine whether the improvements are specific to the chosen backbone and intervention layers.
-
What is the motivation for training a separate editor for each missing-modality condition? Could the authors compare this approach with jointly training a shared editor across different missing-modality conditions to show whether separate training is necessary and justifies the additional training cost?
-
How are the intervention layers selected for each missing-modality condition? Could the authors clarify the selection criteria and investigate how sensitive the performance is to the choice of intervention layer?
-
Given that there is existing work that explores low-rank representation editing and knowledge transfer from complete to incomplete modalities, could the authors clarify how EASE differs from combining existing representation-editing methods with conventional knowledge distillation? In particular, what benefit does the proposed effect-aware supervision provide over directly distilling complete-input predictions?
4 comments
- Q4. Could the authors clarify how EASE differs from combining existing representation-editing methods with conventional knowledge distillation? In particular, what benefit does the proposed effect-aware supervision provide over directly distilling complete-input predictions?
EASE uses low-rank editing to recover task-useful modality effects. Its REP carrier is constructed from paired complete-minus-missing hidden-state displacements, providing a compact space of modality-related changes. The effect-aware objective then determines which reference effects the editor should learn to reproduce.
The key distinction from editing + KD is selective supervision. A conventional combination trains a low-rank editor to make incomplete-input predictions resemble complete-input predictions. Although ground-truth CE is also retained, the KD term still encourages agreement with the complete-input model, including on samples where that reference is unhelpful.
EASE instead uses the missing modality’s score effect as a reference. It matches the centered reference only when that correction yields lower CE than the current edit; otherwise, it penalizes excessive changes. Ground-truth CE remains active throughout. This balances recovery of modality effects with task correctness, since complete input does not always produce a better prediction.
We isolate this distinction using the same low-rank editor. Both methods use the same rank-6 REP editor, frozen backbone, data, and validation-selection protocol; only the training objective changes.
| Seed | Low-rank editing + KD | EASE | Paired gain |
|---|---|---|---|
| 2061 | 71.87 | 72.71 | +0.84 |
| 2062 | 71.89 | 72.87 | +0.98 |
| 2063 | 71.24 | 72.41 | +1.17 |
| Mean | 71.67 | 72.66 | +1.00 |
| Table 4. EASE versus low-rank editing + KD on IEMOCAP-4, using frozen Emotion-LLaMA-v2. |
EASE improves W-F1 at all three seeds, by 1.00 point on average. Because the editing architecture is identical, these gains support the benefit of effect-aware supervision over complete-input KD within the same low-rank editing framework.
- Q3. Layer sensitivity
We also retrained EASE at blocks 28 and 31 to test whether moving the editor makes task performance change.
| Dataset | Block 23 (default) | Block 28 | Block 31 |
|---|---|---|---|
| IEMOCAP-4 | 72.60 | 71.86 | 72.12 |
| MOSI | 79.58 | 79.76 | 79.71 |
| Table 5. Test W-F1 (%) with the same seed and hyperparameters at each layer, averaged over Missing-V/A/T and, for IEMOCAP-4, five folds.* |
Performance changes little across these tested late layers: W-F1 varies by 0.74 points on IEMOCAP-4 and 0.18 on MOSI. Higher replay recovery at deeper layers does not consistently improve task performance, as we proposed in Section 3.
Thank you for the additional experiments and clarifications. They address most of my concerns. The comparison with standard distillation also supports the benefit of supervision, although the gain is modest and has only been tested on one dataset. I will maintain my original rating of 3.
Response to reviewer TS9e
We thank the reviewer for their thorough and insightful feedback. Below, we address the main concerns and provide additional experimental evidence to substantiate our claims.
We now train and evaluate EASE on MiniCPM-o 4.5 and AffectGPT, extending the evidence beyond oracle replay on additional backbones.
MiniCPM-o 4.5 gains 13.85 mean W-F1 points across V/A/T. AffectGPT gains 22.18 / 9.32 / 31.33 points under Missing-T on IEMOCAP-4 / MOSI / MOSEI, respectively. These results show that EASE can be trained and deployed on additional frozen backbones using only the available modalities at inference.
The separate editors used in the paper are one implementation of EASE. They give each missing condition its own recovery basis and predictor. The same framework can also use a shared, condition-aware editor trained jointly across conditions. The two implementations achieve similar mean W-F1 (72.71 vs. 72.59), showing that EASE accommodates both separate and shared editing.
The intervention depth is preset and fixed across missing conditions. For Emotion-LLaMA-v2, we use the answer-position input of block 23 (relative depth about 0.75), unchanged across datasets and seeds. No test score was used to select this layer. Cross-layer replay on MELD provides supporting evidence:
Replay recovery depends strongly on depth: layer 23 is the earliest tested layer to recover most of the effect (91.8%), while leaving roughly the final quarter of the network to process the edit. This supports the preset choice.
Figure readability. We will enlarge the text and improve the layout of Figure 2 in the revision.