- Rating
- 4(+1)
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes a novel framework for trustworthy multi-modal clustering. It integrates information bottleneck theory with evidential deep learning, which represents a fresh and reasonable direction that has not been fully explored before. The experiments are sufficient and cover multiple public multi-modal datasets. The evaluation is conducted on several benchmark datasets and compared against a broad set of recent baselines, which lends credibility to the reported results.
Critical strengths
- The research problem addressed in this paper has strong practical significance. Clustering multi-modal data in the presence of modality conflict is a challenging and widely encountered task, and the paper is well motivated in this respect.
- The modality weights in the proposed method are interpretable. Since they are derived from epistemic uncertainty, they directly reflect the reliability of each modality.
Critical weaknesses
- The necessity of Dempster-Shafer fusion is not justified. Why a simpler probability-level fusion would not be sufficient, and the authors should explain the advantage of this.
- Since pseudo-labels are generated from the model's own predictions, there is a risk that errors may propagate and accumulate during training. How is this error propagation handled?.
- The necessity of maintaining two fusion paths is not established. It is not clear what each path contributes to the final result and why both are required.
Questions for authors
- The key differences between the proposed method and existing multi-modal clustering methods are not presented clearly enough.
- Whether the proposed method can be transferred to other unsupervised learning tasks is not demonstrated. The evaluation is limited to multi-modal clustering, and its generalizability remains unclear.
- The reason for using Deng mutual information instead of conventional mutual information is not explained. The paper should clarify the effect that Deng mutual information brings in this task.
2 comments
Thanks for the response. This addresses my concerns, and I have no further questions. I will consider raising my score accordingly.
Response to Reviewer wCKL
We appreciate the reviewer’s constructive comments and suggestions, which have helped us improve the clarity and presentation of UNIT-MIB.
Weakness 1
D-S fusion is adopted to preserve the complete evidential information from different modalities. Unlike probability-level fusion, which primarily combines normalized class probabilities, D-S fusion operates on subjective opinions containing both class-wise belief masses and epistemic uncertainty. Thus, it preserves uncertainty information that may be lost through conventional probability averaging. Moreover, its combination rule incorporates the conflict coefficient to account for disagreements among modalities. These properties make D-S fusion suitable for constructing a global evidential consensus from modalities with different uncertainty levels and potentially conflicting evidence.
Revision:
Weakness 2
UNIT-MIB mitigates potential pseudo-label error propagation through confidence-based pseudo-label selection. Specifically, soft pseudo-labels are first generated from the global D-S consensus, which integrates evidential information from multiple modalities. Samples are then ranked according to their confidence scores derived from the D-S consensus, and only the pseudo-labels associated with the top-k most confident samples are retained for EDL supervision, while ambiguous ones are excluded. Furthermore, pseudo-labels are refreshed at the end of each epoch and remain fixed within each epoch to provide stable supervision. These mechanisms help reduce the influence of erroneous pseudo-labels and limit error accumulation during training.
Revision:
Weakness 3
The two fusion paths serve complementary roles. The feature-level path performs uncertainty-weighted semantic aggregation, while the decision-level path establishes an evidential consensus through D-S fusion and provides guidance via Deng-MI-based alignment. To validate their respective contributions, we conducted additional path-level ablation experiments. The results show that removing either path leads to decreases in both ACC and NMI across all five datasets, demonstrating their complementary contributions to the final clustering performance.
Revision:
Question 1
Compared with existing multi-modal clustering methods, UNIT-MIB explicitly incorporates EDL-derived modality-level epistemic uncertainty into IB optimization and feature-level fusion. Furthermore, DS-based evidential fusion and Deng-MI-based directed alignment connect uncertainty-aware representation learning with evidential decision-making, providing a unified framework for trustworthy multi-modal clustering.
Revision:
Question 2
The primary focus of UNIT-MIB is multi-modal clustering, and all experiments are conducted within this task. Therefore, the current experimental results do not establish its generalizability to other unsupervised learning tasks. Although the uncertainty regulation and IB-based representation learning have potential for broader applications, extending UNIT-MIB to other tasks would require adapting its task-specific components and learning objectives.
Question 3
In UNIT-MIB, conventional mutual information is retained in the information bottleneck objective for information compression and preservation, whereas Deng MI is specifically adopted for opinion-level alignment.
We adopt Deng MI because it accounts for both class-wise belief masses and uncertainty through Deng entropy, making it suitable for measuring discrepancies between complete subjective opinions. In contrast, alignment based solely on normalized class probabilities does not explicitly preserve the uncertainty information contained in subjective opinions.
In this task, Deng MI enables the feature-fused opinion MS to align with the D-S evidential consensus MDS in terms of both class beliefs and uncertainty. This provides an uncertainty-aware alignment constraint that helps reduce the semantic-logical gap..
Revision: