- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
Through a diagnostic experiment on MCubeS, the authors find that source participation at the feature level is nearly uncorrelated with the improvement that per-source experts bring to the prediction, suggesting that "participation does not equal contribution." This indicates that after multimodal feature fusion, the prediction stage still needs to account for how much each modality's source-specific information actually affects the prediction. EvidenceSAM therefore performs multimodal feature fusion in the CMF stage while retaining source information; in the RDEA stage, it uses source features to separately model corrective and degradation evidence with respect to the base prediction, so as to assess prediction-level source contribution; CGSR further decomposes the semantic states provided by RDEA on a per-class basis. On MCubeS / DeLiVER, the method improves over the strongest baselines by +2.58 / +2.33 mIoU.
Critical strengths
-
The main text is clearly structured.
-
The motivation is clear: the paper challenges the implicit assumption—adopted by many existing multimodal fusion methods for multimodal semantic segmentation—that source participation is roughly equivalent to its contribution, and proposes the EvidenceSAM framework to validate the necessity of source identity at the decision-decoding stage after multimodal fusion.
-
The method achieves state-of-the-art performance on multimodal semantic segmentation, with clear improvements.
Critical weaknesses
-
Insufficient comparative experiments. There is no comparison isolating TASR, so its actual effect remains unverified. Could TASR be replaced by a standard per-pixel spatial gating? What is the necessity of this design? TASR is also not shown in the architecture figure (Fig. 2).
-
The paper defines and uses a quantity called "semantic contribution," but prediction-level semantic contribution is not explored for the first time here; closely related concepts were studied earlier in Perceptual Score (NeurIPS 2021) and PID (Liang et al., NeurIPS 2023). The paper should discuss its relation to prior work on measuring semantic contribution / modality dependence, and avoid describing itself as the first to raise this question.
-
The figures and the notation system have clear defects. In the RDEA section, symbols are reused across different meanings and under-explained, which makes the method hard to follow; several variables are not clearly defined.
Questions for authors
-
TASR receives no explicit task supervision and is trained only end-to-end. How is "task alignment" actually achieved?
-
Please provide the construction of , , , and (formulas or code), and report the values of , , , and , together with a sensitivity analysis.
-
RDEA, as the core module of the method, is not described in sufficient detail to ensure reproducibility. Do the authors plan to release the source code to ensure the transparency of the method?
3 comments
Thank you for the extensive revisions. The revised version improves the readability and transparency of the paper, and substantially completes the description of the relevant modules and the evidence for their effectiveness. This is a substantive improvement, and I am willing to raise my score accordingly.
That said, readability and transparency should have been ensured before submission so that review effort is not spent on reconstructing the method. The lack of clarity in the submitted version made the paper substantially harder to understand, and this was avoidable.
Thank you for taking the time to carefully review our revised manuscript and for acknowledging the improvements. We appreciate your candid feedback and agree that the method should have been presented more clearly in the original submission. Your comments have helped us improve the clarity, technical presentation, and overall quality of the paper. We sincerely appreciate your thoughtful consideration.
Response to Reviewer ULn4: TASR, RDEA Construction, and Sensitivity Analysis
We thank the reviewer for the constructive suggestions concerning TASR, the relationship to prior work, and the mathematical transparency of RDEA. We have revised the method description and added controlled mechanism comparisons and parameter sensitivity analyses.
TASR is now explicitly illustrated in Figure 2, including pairwise scoring, bounded source exchange, and the updated fused representation.
Sections 3.2 and Appendix 6.1 describe the antisymmetric transfer mechanism. Each pairwise transfer increases one source's allocation while decreasing another's, preserving total support and keeping unsupported sources inactive.
Task alignment arises from the downstream segmentation objective. Appendix 6.1, Eq. (26), derives the gradient of the task loss with respect to the pairwise exchange. This relation connects the learned redistribution to task-dependent gradients through the difference between source features, without requiring an additional alignment target.
We have also added separately trained mechanism comparisons on both MCubeS and DeLiVER (Table 3). Replacing TASR with pixel-wise spatial gating yields 52.44%/62.56% mIoU, compared with 55.46%/67.71% for EvidenceSAM. The improvements are 3.02/5.15 points. Removing TASR while retaining support-normalized fusion yields 52.46%/62.63%.
We have expanded the relevant discussion in Section 2 by citing Perceptual Score (Gat et al., 2021) and the multimodal information decomposition framework of Liang et al. (2023).
These studies examine modality dependence and task-relevant multimodal information. EvidenceSAM addresses a complementary problem: evaluating source-associated semantic alternatives relative to a fused base prediction and incorporating their evidence into spatially varying semantic decisions. Our contribution is the use of this distinction in the prediction process, rather than introducing modality contribution as a new general concept.
We have clarified the construction of E_B, C_m, R_m^d, and H_m^d in Section 3.3 and Appendix 6.2.
Appendix 6.2, Eqs. (27)–(30), specifies the base and expert prediction statistics, coarse verifier features, class-belief weighting, and detail readouts. The four sigmoid evidence factors estimate base-error evidence, expert-correctness evidence, direct corrective evidence, and direct degradation evidence.
Eq. (9) combines these factors into corrective and degradation scores. Eqs. (10)–(11) define expert admission, ranking, and posterior aggregation. Appendix 7.2, Eqs. (48)–(50), further specifies the auxiliary training targets and losses.
The revised manuscript reports the RDEA parameters as lambda = 1.25, delta_adm = 0.01, T_s = 0.35, and kappa = 6.0.
Appendix 9.3 and Table 13 report a one-parameter-at-a-time sensitivity analysis on MCubeS using the trained checkpoint. Across 17 distinct inference configurations, the maximum decrease from the default performance is only 0.095 mIoU points.
The routing behavior remains responsive to the parameters. Increasing lambda from 0.625 to 1.875 changes expert admission from 70.79% to 17.89%, while final segmentation accuracy varies only slightly. Further analysis shows that 96.14%–99.92% of pixels with changed routing decisions retain their aggregated semantic class under the evaluated extreme settings.
The separately trained direct-utility-head ablation provides complementary evidence for the risk decomposition itself, with improvements of 3.16/5.49 points on MCubeS/DeLiVER.
The revised manuscript provides detailed expert inputs, evidence construction, selection rules, auxiliary objectives, gradient boundaries, and training configurations in Sections 3.3 and Appendices 6–7. We also plan to release the source code and corresponding experiment configurations publicly to support reproducibility.
We hope these revisions address the questions regarding TASR's mechanism and learning signal, the relationship to earlier research, and the construction and stability of RDEA.