- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper addresses robust multimodal sentiment analysis under two perturbations: random data missing at the input and bit-level channel noise (BSC/BEC) during feature transmission in a distributed setting. The proposed DMDN consists of a Dynamic Information-guided Modality Correction module (DIMC), which estimates per-modality degradation via KL divergence, selects the least-degraded modality as dominant and corrects it with an adversarially generated proxy, and a joint source-channel coding pipeline in which features are encoded into spike trains by a Conv-BN-LIF sender with temporal-sequence attention, passed through a simulated binary channel, and decoded by a Mamba-based receiver before ALMT-style fusion. Experiments on MOSI, MOSEI and SIMS at a 0.5 missing rate report gains over LNLN and other baselines, with robustness curves, ablations and a sensitivity analysis.
Critical strengths
- Jointly considering input-level missing data and transmission-level corruption is a reasonable extension of the robust-MSA setting, and replacing a fixed language-dominant assumption with a per-sample choice is well motivated.
- The paper provides robustness curves over noise levels, a module ablation, a dominance-selection ablation, a loss-weight sensitivity study and a missing-rate sweep.
- The paper is concise and the overall pipeline is easy to follow.
Critical weaknesses
- The novelty is limited: the method is an assembly of existing components. DIMC reuses the dominant-modality correction of LNLN [3] (generator, effectiveness discriminator, completeness-weight loss, and even the same loss weights 0.9/0.8/0.1/1.0); fusion is ALMT's hypermodality learning; the spiking sender/channel/receiver follows the authors' cited spiking transmission frameworks [1, 2]; the TSA module is TCJA-SNN. The new element, KL-based dynamic dominance, gives marginal gains (Table 7: fixed-vision dominance matches the full model on Acc-7; MAE 1.084 vs. 1.067).
- The channel-noise setting is artificial and the comparison is not fair. Noise is injected into the model's own intermediate spike representation, and LNLN is evaluated through a 3-bit serialization interface built by the authors that it was never designed for. No conventional alternative (quantization + channel coding, DeepJSCC-style autoencoders, or plain repetition over the 8 timesteps) is compared, so the benefit of spikes + SSM is not established; Table 8 even shows higher inference time, and the FLOPs reduction is unexplained given the shared ALMT backbone.
- Several claims are contradicted by the paper's own results. The abstract says DMDN "consistently outperforms" baselines, yet Table 6 shows LNLN is better at missing rates ≤ 0.4 (e.g., 40.23 vs. 37.03 Acc-7 at 0.1), while the main tables report only the 0.5 operating point where DMDN starts to win. Training with channel noise, claimed to improve robustness, lowers MOSEI Acc-7 from 50.83 to 46.00/45.12 and MAE from 0.633 to 0.684/0.688 (Table 1).
- No standard deviations, seeds or significance tests are reported anywhere. Many margins (MOSI Acc-2 74.24 vs. 73.56, Corr 0.519 vs. 0.518, ~1-point differences in Table 3) are within typical seed noise on MOSI. Whether baseline numbers are re-runs or copied is not stated, and some entries look erroneous (MFMB-Net MOSEI Acc-2 47.64/58.11; MRUF MOSEI MAE 0.975).
- Technical and presentation issues: the inference-time proxy KL(R(X̃)‖X̃) measures how much the reconstructor changes its input rather than degradation, and the train/test mismatch is never validated; Section 3.3 says the sender emits binary spikes while Appendix A.1 quantizes "channel symbols" to 3-bit integers; Eq. (17) refers to 5×5/3×3/1×1 branches while Eq. (12) uses 1D kernels and Fig. 1 labels o^(1..3); H*_m and the supervision target for w (Eqs. 11, 19) are undefined; SIMS is described as "dialogues and interviews" although it is collected from movies and TV; robust-MSA baselines such as MMIN, GCNet, DiCMoR/IMDer and the semantic-communication literature are not discussed; no code is released.
The combined setting is worth studying, but in its current form the contribution is an assembly of prior modules, the evaluation is selectively reported without variance, and key claims are not supported. I would reconsider if the authors report mean±std over seeds across the full missing-rate range, add a fair channel baseline, and reconcile the contradictions above.
[1] Neuron-based Spiking Transmission and Reasoning Network for Robust Image-Text Retrieval (TCSVT 2023) [2] Multi-scale Spiking Pyramid Wireless Communication Framework for Food Recognition (TMM 2024) [3] Towards Robust Multimodal Sentiment Analysis with Incomplete Data (NeurIPS 2024)
Questions for authors
see above