- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This manuscript challenges the popular paradigm of consistency learning in multi-view clustering. After summarizing existing consistency-based methods into two categories (strengthening cross-view consistency, and obtaining a global target for supervision), the authors provide an analysis of the effectiveness and limitations of consistency learning, thus proposing a consensus-oriented multi-view representation learning framework. The framework consists of three parts: relation preservation, self-attention and consensus-only clustering. Extensive experiments have shown the effectiveness of this framework.
Critical strengths
1.The method is well-designed and achieves strong performance on a wide range of benchmarks.
2.The authors have made efforts to give a theoretical analysis on the limitations of consistency learning.
Critical weaknesses
1.The definition of the word “consensus learning” is not clear, therefore the central contribution remains insufficiently defined. In my view, this study mainly works on maintaining complementary information across different views, which falls exactly into the perspective of complementary learning of MVC. However, the authors do not clarify on their relation with and differences from existing complementarity-aware MVC literature. Indeed, the authors mainly review consistency-based methods, while this work is substantially more related to complementary learning.
2.The analysis is rather simple.
Questions for authors
1.What is the definition of “consensus learning”? In what sense is your study different from existent complementary learning in MVC?
2.The quality of Figure 1 is not good. It is not immediately clear that the three images represent different views. Also, the supervision relation of the right part is not so clear.
3.The analysis seems to be an important part of your work as it is listed in your main contributions. However, it is altogether simple. Despite your complicated mathematical presentation, you do not present anything other than the two paradigms of consistency-based learning and their failure modes.
4.In the last paragraph of pg.4, you mention two principles: (1) preserve each view’s local relational structures during representation learning; and (2) apply direct clustering supervision only at the consensus level. In Table 2, you mainly ablate different consensus formation and relation preservation. There is insufficient direct empirical evidence on the correctness of these two principles.
8 comments
3, We thank the reviewer for the comment. We would like to further clarify the full scope and main findings of our theoretical analysis. First, due to the page limit, Section 3.1 provides a concise analysis of the relationship between consistency learning and consensus learning, primarily based on conditional mutual information. Specifically, we use the conditional mutual information (\Gamma_{ij}^{v}) to characterize the additional discriminative information that different views provide for the clustering relationship between a sample pair. This analysis identifies when consistency learning can approximate consensus learning: when clustering-relevant information is largely redundant across views. We then present an abstract formulation of consensus learning and establish two basic principles for consensus-oriented representation learning. Second, the underlying mechanisms and limitations of the two consistency-oriented paradigms are further examined in Appendix A.
Appendix A.1 examines how cross-view consistency constraints affect discriminative structures. We first consider a consistency constraint based on squared Euclidean distance and reformulate it as a common-center optimization problem. This explains why discriminative information supported by only a subset of views may be weakened. We then analyze consensus formation using a general consensus function, showing how the clustering objective can guide the integration of multi-view information without requiring discriminative information to be present in every view.
Appendix A.2 analyzes the information mismatch in global-to-local supervision. Specifically, we rewrite the conditional mutual information as a difference between conditional entropies and explain, from an information-theoretic perspective, that when other views contain clustering-relevant information unavailable from an individual view, the global objective may involve clustering semantics beyond what that view can capture independently. Directly requiring local predictions to match the global objective may therefore lead to supervision mismatch. Furthermore, through the gradient propagation of the consensus-level clustering objective, we explain how each view participates in clustering optimization through its contribution to the consensus representation, without having to independently reproduce the complete global clustering semantics.
Overall, our analysis goes beyond summarizing the two consistency-oriented paradigms and their potential failure cases. It characterizes the relationship between consistency learning and consensus learning, including the conditions under which consistency learning can approximate consensus learning. More importantly, these analyses explain why consensus-oriented representation learning should preserve the local relational structure of each view and apply direct clustering supervision at the consensus level.
4, We thank the reviewer for the comment. We verify the two principles by corresponding experiments.
First, the principle of preserving each view's local relational structures during representation learning is implemented through the relational preservation loss. The ablation study on this constraint in the original Ablation Study subsection provides a direct empirical evaluation of this principle. The results show that incorporating relational preservation consistently improves performance under different consensus formation strategies, indicating that maintaining the local relational structures of individual views benefits multi-view clustering.
Second, to evaluate the necessity of applying clustering supervision only at the consensus level, we considered three supervision strategies: consensus-only, view-only, and joint supervision. Joint supervision was examined in the original Empirical Analysis of Consistency-Oriented Learning subsection by varying the balance between the supervision terms. The results showed that joint supervision did not lead to better clustering performance. In addition, we conducted a new ablation study with supervision applied only to view representations. By keeping all other settings unchanged and varying only the location of clustering supervision, we directly compared consensus-only and view-only supervision. As shown in the new Table 3, consensus-only supervision outperforms view-only supervision on all evaluation metrics across the three datasets. These results are consistent with the theoretical analysis in Section 3.1 and Appendix A.2. When other views provide additional clustering-discriminative information that is absent from a particular view, requiring each view to match the global clustering target may lead to a global-to-local supervision mismatch. In contrast, consensus-level supervision first integrates discriminative information from different views and then optimizes the clustering structure at the consensus level, without requiring each view to independently reproduce the complete global clustering semantics. The new supervision-position ablation study and its results have been added to Table 3 of the revised manuscript.
| Supervision | Animal | Animal | Animal | Animal | AWA-7view | AWA-7view | AWA-7view | AWA-7view | Xmedia | Xmedia | Xmedia | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| acc | nmi | pur | ari | acc | nmi | pur | ari | acc | nmi | pur | ari | |
| View-only | 0.592 | 0.677 | 0.651 | 0.491 | 0.203 | 0.333 | 0.239 | 0.103 | 0.136 | 0.384 | 0.156 | 0.065 |
| Consensus-only | 0.648 | 0.722 | 0.681 | 0.56 | 0.558 | 0.635 | 0.595 | 0.43 | 0.919 | 0.921 | 0.919 | 0.85 |
We hope that our clarifications, additional experiments, and corresponding revisions adequately address your concerns. We would appreciate your reconsideration of the overall assessment in light of these responses.
1, We thank the reviewer for the question.
First, we define Consensus Learning as a multi-view representation learning paradigm that integrates clustering-relevant evidence across views at the consensus level and directly learns the clustering structure. Its central idea is that each view provides structural evidence consistent with its own observations, while the consensus representation integrates this evidence and carries the final clustering semantics. Therefore, an individual view does not need to independently possess complete clustering-discriminative information; the clustering structure is determined jointly by the integrated multi-view evidence.
Second, our distinction from existing complementary learning methods lies in how inter-view information relationships and their optimization objectives are treated. Existing complementary learning methods primarily aim to explore and exploit differences across views to obtain richer sample representations and clustering-discriminative information. However, differences across views are not necessarily relevant to clustering and may also contain information unrelated to the final clustering structure. In contrast, Consensus Learning does not prescribe whether views should be consistent or complementary, but treats both as manifestations of the relationships among multi-view evidence during the clustering process.
Specifically, in our method, the relational preservation loss constrains the local structures within each view, with the aim of preserving the fidelity of each view to its own observations, rather than prescribing the information relationships between views. Accordingly, we do not explicitly constrain inter-view consistency or complementarity. We then construct the consensus representation through sample-wise adaptive cross-view interactions and apply clustering supervision only at the consensus level. This allows the contributions of different views to the final clustering structure, as well as their representation learning, to be guided by the clustering objective. Consequently, both redundant and complementary clustering-relevant evidence can contribute to consensus formation without requiring their information relationships to be specified in advance. We have added the relevant discussion to Section 2.2 of the revised manuscript.
2, We thank the reviewer for the suggestion. We have redrawn Figure 1 to clearly indicate that the three images represent different views, and revised the schematic on the right to better illustrate the relationship between the consensus representation and clustering supervision. These changes are reflected in Figure 1 of the revised manuscript.