- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes PIDFusion for multimodal sentiment analysis, which treats text and the joint audio–visual (AV) representation as two sources of a partial information decomposition (PID) with respect to the sentiment label. A BATCH-style neural probe with label-conditioned Sinkhorn coupling, trained on detached features with its own optimizer, estimates redundant, unique and synergistic quantities; a batch-centered local AV-unique proxy is calibrated by a one-time injection sweep and thresholded to decide which training samples receive a text-blind AV-only regression loss. Together with unimodal auxiliary heads and a detached gated fusion, the model is evaluated on CMU-MOSI, CMU-MOSEI and CH-SIMS within MMSA, with matched-random and uniform selection controls on MOSI.
Critical strengths
- The question of which samples need extra non-text supervision is well posed, and the matched-random and uniform controls are the right experiments to test whether sample identity matters.
- Gradient boundaries are clearly specified (detached probe, separate optimizer, detached selector), and the paper is candid that the atoms are operational estimates rather than exact PID.
- Hyperparameters, feature shapes and the efficiency profile (including the admission that FLOPs are the highest among compared models) are fully documented in Appendix C and E.
Critical weaknesses
- All results are single runs with no seeds, standard deviations or significance tests. The headline margins are within typical seed noise on these benchmarks: MOSI Acc-2 85.37 vs. 85.06 (Self-MM), MAE 0.705 vs. 0.710; MOSEI F1 85.32 vs. 85.27 (DLF) and Acc-2 below DLF; SIMS Acc-3 below TETFN/ALMT. The central ablation (full vs. matched-random: +0.92 Acc-2, +0.44 Acc-7; full vs. uniform: +0.77 Acc-2) is likewise a single-run difference of under one point.
- The PID machinery is largely decorative with respect to what the method uses. Only the local AV-unique proxy enters training, and by Eq. (16) it equals the AV probe's per-sample log-likelihood ratio log p_AV(c_i|z_AV,i)/π(c_i) shifted by a batch-level constant and clipped at zero; the Sinkhorn coupling only sets that constant. The natural control, selecting samples by AV-head confidence or loss without any PID estimator, is missing, as is any analysis of the estimated R, U_T, S atoms or validation of the local score (CH-SIMS provides modality-specific labels that would directly test whether selected samples are those where AV annotations disagree with text).
- The fusion gate in Section 3.5 is a sigmoid MLP whose parameters "remain at their initialization" because its output is detached and no objective trains it, so the main prediction mixes z_T and z_J through a random input-dependent gate. This is neither motivated nor ablated (learned gate, fixed 0.5, no gate), and it raises doubts about which part of the pipeline the reported numbers reflect.
- The baseline set is dated and mixed. Table 1 combines reproductions with numbers copied from DLF, and recent methods reporting substantially stronger MOSI results in the same MMSA setting (e.g., CICA, CaReFlow, EBMC with Acc-7 ≈ 49–51 and MAE ≈ 0.62–0.69) are absent, so "ranking first on all four metrics" does not hold against the current state of the art. On CH-SIMS only three baselines are compared, one of which (CENET, Corr 0.123) is a collapsed reproduction; standard SIMS baselines (Self-MM, MISA, MMIM, EMOE) are omitted.
- Ablations are restricted to MOSI; the threshold δ differs per dataset (0.05/0.10/0.02) with no stated selection procedure; there is no ablation of injection calibration, the unimodal-head weights, or the probe's coupling. The method roughly doubles FLOPs over Self-MM (16.05 vs. 8.60 G) for gains that cannot be distinguished from noise. Code is only promised upon acceptance.
The selective-supervision question is worth asking and the control design is sound, but the evidence is single-run and within noise, the effective selection signal reduces to AV-probe confidence, and the untrained fusion gate is unexplained. I would reconsider if the authors report mean±std over seeds for the main tables and ablations, add a confidence-based selection control, ablate or justify the frozen gate, and validate the local score against the CH-SIMS modality-specific annotations.
Questions for authors
see above