- Rating
- 3
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a "TwoChannel" fusion rule, which stores missingness and unreliability separately in the support set and the conflict set, rather than handling them individually through channel weights. The prototype decomposition method reads the sub-views of each channel into a shared dictionary and separates the content that is inconsistent among the channel's own parts, so that each channel enters the fusion process in the form of "assertion" and "contradiction."
Critical strengths
1、This paper reveals that no matter what content the existing fusion family writes into the fusion vector, it always writes it onto a single non-negative coordinate of each channel, and that as the damage deepens, the gap between missing channels and damaged channels gradually narrows.
2、The authors propose TwoChannel, which carries missingness information on the support coordinate and unreliability on the conflict coordinate.
Critical weaknesses
1、The paper does not discuss how the specific thresholds for abstention/escalation are set. Even if the two types of failure are detected, how should thresholds be set to trigger different actions? How sensitive are these thresholds to the proportion of missing/damaged data and to cost asymmetry?
2、If core mechanism experiments (separation probes + task accuracy) could be added on the original real-world data of CMU-MOSEI or IEMOCAP, the strengths of this paper would be further enhanced.
3、This paper lacks evaluation under real-world missing or damaged patterns; in the synthetic cohort, missingness is implemented as "removal from existing records," and damage consists of three artificial perturbations. Further supplementation in this regard would make the work more meaningful.
4、This paper's reproducibility is relatively weak; releasing anonymized code and the corresponding data would enhance its reproducibility.
Questions for authors
The writing quality of this paper is relatively good; for specific comments, please refer to the weaknesses.
4 comments
(continued)
Summarized W3: Evaluation under real-world missing and damaged patterns
A3: Following your suggestion, we further evaluate our method under real-world missing and damaged patterns. Specifically, we use the real dataset CMU-MOSEI with its standard features, and take three patterns that occur naturally rather than by removal or artificial perturbation, namely one missing pattern (i.e., missing video when the face is not detected) and two damaged patterns (i.e., recorded background noise on audio and transcripts from speech recognition). We compare our method with the strongest baseline in Table 1 (i.e., Zero-fill) on two metrics (including separation and accuracy). Table R6 shows that our method keeps a separation above 0.96 under the three patterns against at most 0.6694 for Zero-fill, which indicates that our method holds under real-world missing and damaged patterns.
Table R6: Results under real-world patterns on CMU-MOSEI (Tables 1 and 2 under real-world missing and damaged patterns).
| Pattern | Zero-fill Separation ↑ | Our method Separation ↑ | Zero-fill Accuracy ↑ | Our method Accuracy ↑ |
|---|---|---|---|---|
| Missing video (face not detected) | 0.6694 | 0.9768 | 0.7926 | 0.7981 |
| Recorded audio noise | 0.6317 | 0.9712 | 0.8043 | 0.8011 |
| Speech recognition transcripts | 0.6152 | 0.9637 | 0.7738 | 0.7804 |
Following your suggestions, we have added these results to Appendix A.18 as Table 33 (Lines 1728–1743).
Summarized W4: Reproducibility without released code and data
A4: We respectfully clarify that this submission does not include a code repository, and the paper specifies the method for reproduction instead, namely the per-pass procedure with its training update in Algorithm 1 (Lines 1080–1114), the dictionary, graph, trust and training settings in Table 13 (Lines 1242–1263), the cohorts and degradations, the baselines, and the metrics and probe protocol in Appendices A.7 to A.10 (Lines 1066–1218), and the regeneration of the reported quantities in Appendix A.14 (Lines 1473–1478).
Best Wishes,
The Authors of Paper 52481
I am maintaining my original score.
The author addressed some of my concerns; given that I am not particularly familiar with this field, I am maintaining my original score.
Dear Reviewer bqtw,
Thank you for your timely response and for maintaining your positive assessment of our work.
We are glad that our rebuttal has addressed some of your concerns, with the threshold setting for the two actions and its sensitivity analysis, and the results on the original CMU-MOSEI and IEMOCAP data and under three real-world patterns. As suggested, we will keep these experiments in the final version.
Thank you again for your constructive feedback.
Best Wishes,
The Authors of Paper 52481
Response to Reviewer bqtw
Dear Reviewer bqtw:
Thank you for recognizing the novel perspective, the method design, and the clear presentation of our work. We have responded point by point to your concerns regarding the threshold setting for the two actions, the results on real datasets and under real-world missing and damaged patterns, and the reproducibility. In the marked copy appended to the revised PDF (pages 34–67), the revisions made in response to your comments are tagged with @Reviewer bqtw. We hope our responses resolve your concerns and look forward to further discussion.
Summarized W1: Threshold setting for the two actions and its sensitivity
A1: We respectfully clarify that the thresholds are set on the two scores our readout gives for the two failures, and each threshold is chosen on a validation split to minimize the expected cost of wrong actions. Table R1 lists the threshold setting for each failure.
Table R1: Threshold setting for each failure.
Following your suggestion, we further examine the sensitivity of the two thresholds to the proportion of missing and damaged data under three types of cost asymmetry (i.e., the cost ratio of a wrong acquisition to a wrong recalibration, including 1:1, 1:5, and 5:1). For the proportion of missing data, Table R2 shows that the missing threshold stays between 0.16 and 0.21, which indicates that it is insensitive.
Table R2: Missing threshold under missing rate and cost asymmetry.
For the proportion of damaged data, Table R3 shows that the corruption threshold stays between 0.58 and 0.66, which indicates that it is insensitive.
Table R3: Corruption threshold under damage level and cost asymmetry.
Following your suggestions, we have added these results to Appendix A.16 as Tables 26, 27 and 28 (Lines 1612–1655).
Summarized W2: Core mechanism experiments on the original CMU-MOSEI or IEMOCAP data
A2: As you suggested, we further repeat the two main experiments of Tables 1 and 2 on two real multimodal datasets (i.e., CMU-MOSEI and IEMOCAP) under realistic corruption. For the separation in Table 1, Table R4 shows that our method reaches an average separation of 0.972221, while the four baselines stay below 0.70 and remain nearer the 0.501730 of the random-feature control than the 1.000000 of the raw-input ceiling, which is consistent with the conclusion of Table 1.
Table R4: Results of Table 1 on real datasets.
For the accuracy at complete input in Table 2, Table R5 shows that our method reaches 0.8097 and 0.7389 on CMU-MOSEI and IEMOCAP, against 0.8124 and 0.7365 for plain fusion and 0.7731 and 0.7208 for gated fusion, which indicates that our method keeps the separation at a small accuracy cost (0.0027 below plain fusion on CMU-MOSEI and 0.0024 above it on IEMOCAP), consistent with the conclusion of Table 2.
Table R5: Results of Table 2 on real datasets.
Following your suggestions, we have added these results to Appendix A.18 as Tables 30 and 31 (Lines 1674–1709).