acceptodds
Withdrawn from ICLR 2027

Interaction Coordinate Distillation for Multimodal Sentiment Analysis

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multimodal sentiment analysis relies not only on modality-specific evidence but also on how linguistic, acoustic, and visual cues interact. Knowledge distillation can transfer the capabilities of a high-capacity multimodal teacher to a compact student, yet existing approaches primarily define supervision over final predictions, latent representations, or relational structures, leaving predictive interactions across modality combinations largely implicit. We propose Interaction Coordinate Distillation (ICD), which treats teacher and student predictions over the seven non-empty modality subsets as set functions and applies an anchored Möbius decomposition to obtain aligned first-, second-, and third-order interaction coordinates. The invertible transform reorganizes the same teacher subset responses into interaction coordinates, where coordinate-wise matching couples errors across related modality subsets through Möbius contrasts. This allows ICD to change the structure of the distillation constraint without changing the underlying teacher information. ICD jointly optimizes interaction-coordinate matching, task supervision, and full-modality distillation. Experiments on CMU-MOSEI and CMU-MOSI, together with controlled comparisons over raw subset supervision, interaction orders, and random orthogonal reparameterization, show that ICD effectively reconstructs teacher-defined interaction coordinates while achieving strong downstream performance. The additional subset computation required by ICD is absent at inference, introducing no inference-time overhead.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
multimodal sentiment analysis, knowledge distillation, multimodal interaction modeling, Möbius transform

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_2Z9AOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Interaction Coordinate Distillation (ICD) for multimodal sentiment analysis. It applies an anchored Möbius decomposition to teacher and student predictions over seven modality subsets, then matches the resulting first-, second-, and third-order interaction coordinates. Since the transformation is invertible, the contribution lies in restructuring the distillation objective while preserving the underlying teacher information. Experiments on CMU-MOSEI and CMU-MOSI include comparisons with direct subset distillation, interaction-order ablations, a random orthogonal transformation, and interaction-fidelity analyses.

Critical strengths
  1. A clear and potentially useful distillation formulation. ICD provides a concrete way to transfer modality-combination behavior through prediction-level interaction coordinates. It can be applied without adding inference-time branches or parameters, which makes the approach practically appealing.
  2. Relevant controlled comparisons. The Subset-7 baseline uses the same teacher responses as ICD, helping separate the benefit of additional supervision from that of the proposed coordinate system. The interaction-order and random-transformation comparisons further examine the design, and the paper openly reports mixed findings in its interaction analysis.
Critical weaknesses
  1. The gains attributable to the proposed coordinate system are small, and their reliability is unclear. In Table 2, ICD improves mean Acc-2 over Subset-7 by only 0.17 and 0.36 percentage points on MOSEI and MOSI, respectively; the improvements over Random Orthogonal are 0.16 and 0.15 points. These comparisons use only three seeds, without paired significance tests or confidence intervals for the performance differences. Meanwhile, Table 1 uses a single seed, and ICD does not consistently outperform the compared methods across metrics. The results suggest a possible benefit, but do not yet establish a convincing practical advantage for the central contribution.
  2. The reason for preferring Möbius-coordinate matching remains insufficiently established. In the quadratic region of Smooth-L1, the objective induces a metric proportional to (M^\top M) on subset-response errors. This changes both error coupling and weighting. A random orthogonal transformation does not control for the scaling and spectral properties of (M), so the existing ablations cannot isolate meaningful interaction structure from more general loss reweighting effects. Moreover, Figure 3 reports near-zero associations between interaction-fidelity gains and continuous prediction improvements, with confidence intervals including zero. This does not invalidate ICD, but leaves its central mechanism and broader significance unclear.
Questions for authors
  1. Statistical clarification: Can you provide paired per-seed results and confidence intervals for ICD versus Subset-7 and Random Orthogonal, including MAE and Pearson correlation? Do additional runs support a reliable advantage?
  2. Mechanism clarification: Have you compared ICD with normalized Möbius contrasts, weighted raw-subset losses, or transformations with matched singular values? Such comparisons would help determine whether the benefit comes specifically from the proposed interaction structure.
  3. Comparison clarification: Full KD uses a single probe, whereas ICD obtains its interaction targets from three averaged probes. How much of the improvement over Full KD remains when both methods use the same probe ensemble?
Reviewer_QhEZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ICD, which treats teacher responses over non-empty modality subsets as a set function, applies a shared anchored Möbius decomposition, and matches first-, second-, and third-order interaction coordinates during training.

Critical strengths

The set-function/Möbius-coordinate view departs from standard response distillation, and deployment remains a standard full-TAV student with no extra inference cost.

Critical weaknesses
  1. Since the anchored Möbius transform is invertible, it preserves exactly the same subset-response information. The paper does not sufficiently explain why matching Smooth-L1 losses on these particular linear contrasts should be better than direct subset matching, beyond asserting “Möbius contrast constraints.”

  2. First, second, and third-order coordinates are Möbius subtraction-based residual contrasts, not mutual information, redundancy, synergy, or information-theoretic interaction. The term “interaction” should be explicitly defined to avoid confusion.

  3. Main comparisons use a single seed-13 protocol; teacher probes use 2026/2027/2028; ablations report only three seeds. MSA commonly reports mean and maximum over five runs with standard seeds such as [1111, 2222, 3333, 4444, 5555]. The unusual choices may raise concerns about protocol rigor and even about AI automatically generated or insufficiently validated experimental reporting.

Questions for authors
  1. Please explain rigorously why coordinate-wise Smooth-L1 matching gives a better constraint than direct subset matching.

  2. Do you agree that these coordinates should be explicitly called Möbius residual contrasts rather than mutual-information or information-theoretic interactions?

  3. Full KD uses the fixed seed-2026 probe for its TAV target, while ICD uses the mean of three probes. Could this confound the comparison? Please report a version where Full KD and ICD use the same teacher target construction.

  4. Can you add at least one additional common MSA benchmark, such as CH-SIMS, IEMOCAP, or MELD, or otherwise justify the choice and soften generalizability claims?

Reviewer_N7yqOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes a multimodal distillation framework that explicitly models various combinations among modalities and performs distillation. It divides multimodality into multiple sub-representations and the multi-order interactions among them for distillation. Experiments show a certain effect.

Critical strengths
  1. The method is concise, and the mathematical definitions are clear.
  2. The paper is complete and easy to follow.
Critical weaknesses

• The effect shown in Table 1 is limited, and it is worse than many methods. • The ablation verification in Table 2 shows that the complementarity of different subsets and combination methods is very limited. • It is validated only on the three-modality datasets MOSEI and MOSI. Can this method be extended to the more widely studied bimodal setting? • Multiple subsets may be redundant, and no adaptive selection method has been designed. It also has not been proven that dividing into multiple subsets truly solves the specific problem that is not explicitly divided into subsets.

Questions for authors

How to achieve an ideal balance among those subsets during distillation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.