- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes Interaction Coordinate Distillation (ICD) for multimodal sentiment analysis. It applies an anchored Möbius decomposition to teacher and student predictions over seven modality subsets, then matches the resulting first-, second-, and third-order interaction coordinates. Since the transformation is invertible, the contribution lies in restructuring the distillation objective while preserving the underlying teacher information. Experiments on CMU-MOSEI and CMU-MOSI include comparisons with direct subset distillation, interaction-order ablations, a random orthogonal transformation, and interaction-fidelity analyses.
Critical strengths
- A clear and potentially useful distillation formulation. ICD provides a concrete way to transfer modality-combination behavior through prediction-level interaction coordinates. It can be applied without adding inference-time branches or parameters, which makes the approach practically appealing.
- Relevant controlled comparisons. The Subset-7 baseline uses the same teacher responses as ICD, helping separate the benefit of additional supervision from that of the proposed coordinate system. The interaction-order and random-transformation comparisons further examine the design, and the paper openly reports mixed findings in its interaction analysis.
Critical weaknesses
- The gains attributable to the proposed coordinate system are small, and their reliability is unclear. In Table 2, ICD improves mean Acc-2 over Subset-7 by only 0.17 and 0.36 percentage points on MOSEI and MOSI, respectively; the improvements over Random Orthogonal are 0.16 and 0.15 points. These comparisons use only three seeds, without paired significance tests or confidence intervals for the performance differences. Meanwhile, Table 1 uses a single seed, and ICD does not consistently outperform the compared methods across metrics. The results suggest a possible benefit, but do not yet establish a convincing practical advantage for the central contribution.
- The reason for preferring Möbius-coordinate matching remains insufficiently established. In the quadratic region of Smooth-L1, the objective induces a metric proportional to (M^\top M) on subset-response errors. This changes both error coupling and weighting. A random orthogonal transformation does not control for the scaling and spectral properties of (M), so the existing ablations cannot isolate meaningful interaction structure from more general loss reweighting effects. Moreover, Figure 3 reports near-zero associations between interaction-fidelity gains and continuous prediction improvements, with confidence intervals including zero. This does not invalidate ICD, but leaves its central mechanism and broader significance unclear.
Questions for authors
- Statistical clarification: Can you provide paired per-seed results and confidence intervals for ICD versus Subset-7 and Random Orthogonal, including MAE and Pearson correlation? Do additional runs support a reliable advantage?
- Mechanism clarification: Have you compared ICD with normalized Möbius contrasts, weighted raw-subset losses, or transformations with matched singular values? Such comparisons would help determine whether the benefit comes specifically from the proposed interaction structure.
- Comparison clarification: Full KD uses a single probe, whereas ICD obtains its interaction targets from three averaged probes. How much of the improvement over Full KD remains when both methods use the same probe ensemble?