acceptodds
Under review as a conference paper at ICLR 2027

IBEC-MVC:INFORMATION BOTTLENECK EHANCED CONTRASTIVE MULTI-VIEW CLUSTERING VIA PROGRESSIVE MATRIX FACTORIZATION

Abstract

Multi-view clustering has emerged as a fundamental problem in machine learning, aiming to discover hidden patterns by leveraging complementary information across multiple data representations. However, existing methods suffer from information loss during dimensionality reduction and fail to effectively balance view-specific characteristics with cross-view consistency. In this paper, we propose IBEC-MVC (Information Bottleneck Enhanced Contrastive Multi-View Clustering), a novel framework that integrates three key innovations: (1) Information Bottleneck-guided Progressive Matrix Factorization (IB-PMF) that systematically compresses representations while preserving clustering-relevant information through theoretically grounded constraints, (2) Entropy-Regularized Self-Supervised Contrastive Learning (ER-SCL) that enhances discriminative structure in latent representations while preventing cluster collapse, and (3) Mutual Information Maximization for View Consistency (MIM-VC) that optimally fuses multi-view information through principled information-theoretic objectives. Our theoretical analysis provides information preservation guarantees for the progressive factorization process, while extensive experiments on benchmark datasets demonstrate significant improvements over state-of-the-art methods.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Multi-view Clustering, Information Bottleneck, Progressive Matrix Factorization, Contrastive Learning, Mutual Information Maximization, Entropy Regularization

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_aaSvOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes IBEC-MVC, an Information Bottleneck Enhanced Contrastive Multi-View Clustering framework. It integrates three components: Information Bottleneck-guided Progressive Matrix Factorization (IB-PMF), Entropy-Regularized Self-Supervised Contrastive Learning (ER-SCL), and Mutual Information Maximization for View Consistency (MIM-VC). The method aims to preserve clustering-relevant information during progressive dimensionality reduction, prevent cluster collapse through entropy regularization, and adaptively fuse heterogeneous views via mutual information maximization. Theoretical results include information preservation, contrastive convergence, consensus mutual information bound, and global convergence. Experiments are conducted on five benchmark datasets against 17 baselines, with reported improvements in ACC and NMI.

Critical strengths

The paper proposes a principled integration of information bottleneck, contrastive learning, and mutual information maximization for multi-view clustering.

IB-PMF introduces progressive matrix factorization with information-theoretic compression, which is a reasonable way to preserve clustering-relevant information.

ER-SCL addresses the important cluster collapse problem through entropy regularization.

MIM-VC provides an adaptive view fusion mechanism based on mutual information and view quality.

The paper reports competitive performance on five benchmark datasets and compares against many recent baselines.

Critical weaknesses

The proof of Lemma3.1 in AppendixA.1 appears incorrect. From Eq.~(17), the derivation gives

I(Z1;Y)≥(1+β)I(X;Y)−βH(Y).I(\mathbf{Z}_1;\mathbf{Y}) \ge (1+\beta)I(\mathbf{X};\mathbf{Y}) - \beta H(\mathbf{Y}).
Since $I(\mathbf{X};\mathbf{Y}) \le H(\mathbf{Y})$, this does not imply
I(Z1;Y)≥(1−β)I(X;Y)I(\mathbf{Z}_1;\mathbf{Y}) \ge (1-\beta)I(\mathbf{X};\mathbf{Y})
or
I(Z1;Y)≥I(X;Y).I(\mathbf{Z}_1;\mathbf{Y}) \ge I(\mathbf{X};\mathbf{Y}).

The claimed information preservation guarantee therefore needs to be re-proved or the assumptions must be clarified.

The experimental section is incomplete in the provided version. Table~2 only shows the header, so the main quantitative comparison cannot be verified. Complete results, standard deviations, and significance tests should be provided.

The paper claims average improvements over state-of-the-art methods, but the exact average improvements and statistical significance are not clearly reported.

Ablation studies are mentioned but not shown in detail. The individual contributions of IB-PMF, ER-SCL, and MIM-VC need to be quantified.

Hyperparameter sensitivity is not adequately discussed. Important parameters include LL, τ\tau, β\beta, λ1\lambda_1, λ2\lambda_2, λ3\lambda_3, λ4\lambda_4, and α\alpha. The selection strategy and robustness across datasets should be clarified.

Some notation and formulas need correction. For example, the definition of pc(v)p_c^{(v)} in Eq.(7) uses arg⁡max⁡jhji(v)\arg\max_j h_{ji}^{(v)}, but the index/dimension convention is unclear. The consensus constraint in Eq.(12) also needs clearer explanation.

Reproducibility details are insufficient: data preprocessing, train/test splits, random seeds, number of runs, and code availability are not fully described.

The paper should better distinguish its contributions from existing information bottleneck, NMF-based, and contrastive multi-view clustering methods.

Questions for authors

Can you correct or clarify the proof of Lemma~3.1? Under what assumptions does the information preservation bound hold?

Can you provide the complete Table~2 with all baseline results, standard deviations, and statistical significance tests?

How are the hyperparameters selected? Can you provide sensitivity analysis for β\beta, τ\tau, λ1\lambda_1--λ4\lambda_4, and α\alpha?

Can you report the full ablation study showing the effect of removing IB-PMF, ER-SCL, and MIM-VC?

Please clarify the notation in Eq.(7) and Eq.(12), especially the cluster assignment and consensus regularization terms.

Can you provide computational complexity analysis and runtime comparisons with the baselines?

Will code and implementation details be released to support reproducibility?

Reviewer_DWHrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes IBEC-MVC. The main contributions and experiments are summarized as follows: (1) IBEC-MVC integrates three components: information bottleneck-guided progressive matrix factorization to compress representations while preserving clustering-relevant information; entropy-regularized self-supervised contrastive learning to enhance discriminative structures and prevent cluster collapse; and mutual information maximization for view consistency to fuse multi-view representations. (2) The author provides theoretical analyses of information preservation and convergence. (3) Extensive experiments are conducted on six datasets, comparing IBEC-MVC with 17 methods. The results demonstrate competitive clustering performance.

Critical strengths
  1. This paper addresses challenges in multi-view clustering, including information loss, discriminative representation learning, and cross-view information fusion.
  2. The proposed algorithm integrates progressive matrix factorization, contrastive learning, and mutual information maximization into a unified framework.
Critical weaknesses
  1. In Eq.(7), minimizing the entropy regularization term encourages imbalanced cluster assignments rather than balanced clustering, potentially leading to cluster collapse.
  2. The mutual information term in Eq.(12) contradicts the objective of mutual information maximization. Moreover, according to Eq.(8), the final term in Eq.(12) should be 0. These inconsistencies raise my concerns about the correctness of the objective function.
  3. Several theoretical derivations appear incorrect. For example Eq.(23), Eq.(55).
  4. The variable Y in Eq.(1) is not clearly explained. If Y denotes the ground-truth labels, how to get it in an unsupervised clustering setting? Furthermore, how can the author guarantee that information compression removes noise rather than clustering-relevant information?
  5. In Eq.(6), all sample except the sample itself are included in the contrastive denominator. Such false negatives may damage the underlying clustering structure.
  6. The optimization algorithm does not clearly explain how all variables in the objective function are optimized.
  7. The IBEC-MVC involves O(n^2) complexity, which may limit its applicability to large-scale datasets. However, this paper does not provide runtime comparisons with existing methods.
  8. The related work and background sections are relatively simple.
  9. Although the paper provides ablation and parameter sensitivity studies, the experiments do not verify whether progressive compression preserves clustering-relevant information, and entropy regularization prevents cluster collapse.
Questions for authors

Although the consensus representation H* is used to obtain clustering assignments, the constraints imposed on H* and the final label extraction procedure are insufficiently described. The author should clarify whether H* is directly treated as a cluster membership matrix and how valid cluster assignments are ensured. Because traditional algorithms often run clustering method on H* to get final clustering outcomes.

2 comments

Although some minor issues have been addressed, the core concerns remain unresolved: the original formulation contains explicit sign errors and vanishing terms, and the theoretical justifications are post-hoc rather than causal. Therefore, I maintain my original score.

Thank you for the follow-up. Below is the complete effective objective after substituting Eq. (8), followed by additional training interventions on four datasets.

Effective objective.

Writing EvE_v for positive marginal entropy and MvM_v for the MINE estimate, the minimized global objective is

\left\\|X^{(v)}-\left(\prod_lW_l^{(v)}\right)H^{(v)}\right\\|_F^2 +\lambda_1L_{\mathrm{CL}}^{(v)}-\lambda_2E_v -\lambda_3M_v\\!\left(H^{(v)},\sum_uw_uH^{(u)}\right) \right].$$ The negative entropy and MI coefficients implement their maximization within the joint objective. Substituting the exact consensus eliminates its discrepancy penalty, while the MI terms retain the view-to-consensus coupling shown explicitly above. The representation-gradient paths are specified in W6 of our first response. **Additional training interventions.** We extended the component-removal comparison beyond HW and MSRC-v1 to YALE, BBC, BBCSport and Reuters. Each configuration was trained separately from initialization, retaining the final cluster-score dimension and the remaining settings. ACC values are percentages. | Configuration | YALE | BBC | BBCSport | Reuters | |---|---:|---:|---:|---:| | Without IB-PMF | 67.9 | 84.1 | 88.5 | 87.7 | | Without ER-SCL | 68.8 | 84.8 | 89.2 | 88.5 | | Without MIM-VC | 69.6 | 85.2 | 89.8 | 88.9 | | Full method | 71.2 | 86.7 | 91.4 | 90.3 | Removing IB-PMF uses one-layer MF with zero rate weight. Removing ER-SCL disables contrastive learning and entropy regularization. Removing MIM-VC disables MI and uses equal fusion weights. Each intervention reduces ACC on all four datasets. These controls change the training computation and measure the contribution of the corresponding component. The frozen-representation and marginal/conditional entropy diagnostics in W4 and W9 of our first response address compression and cluster balance. This added table extends the evidence to component interventions across the remaining benchmark datasets.
Reviewer_rjmMOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors propose an Information Bottleneck Enhanced Contrastive Multi-View Clustering framework, where three key innovations are considered, i.e., Information Bottleneck-guided Progressive Matrix Factorization (IB-PMF), Entropy-Regularized Self-Supervised Contrastive Learning (ER-SCL) and Mutual Information Maximization for View Consistency (MIM-VC).

Critical strengths

S1. This paper is well-written, and easy to follow. S2. Theorectical analysis is well provided with the description of method. S3. The flowchart figure is clear.

Critical weaknesses

W1. Depsite the clear presentation, the effectiveness of this method needs further validation, where the maximum dataset size used in this experiment is 2000, and the most recent baselines (about information bottleneck and contrasting multi-view clustering) are missing. W2. How about the time complexity of the proposed method? The smaller size of dataset brings out the concern about substantial time cost of the method. W3. The literatures about contrasting multi-view clustering are missing in the introduction of related work. W4. How to capure the complementary information in MIM-VC, and how about the dimension of H^{(v)}.

Questions for authors

See weakness.

Reviewer_YkGuOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes IBEC-MVC (Information Bottleneck Enhanced Contrastive Multi-View Clustering) as a novel framework that integrates: (1) Information Bottleneck-guided Progressive Matrix Factorization (IB-PMF) that systematically compresses representations while preserving clustering-relevant information through theoretically grounded constraints, (2) Entropy-Regularized SelfSupervised Contrastive Learning (ER-SCL) that enhances discriminative structure in latent representations while preventing cluster collapse, and (3) Mutual Information Maximization for View Consistency (MIM-VC) that optimally fuses multi-view information through principled information-theoretic objectives. The theoretical analysis is used to provide information preservation guarantees for the progressive factorization process, while extensive experiments on benchmark datasets demonstrate its improvements over state-of-the-art methods.

Critical strengths

The problem motivation and modular design are reasonable. The three fundamental challenges of multi-view clustering (information loss during dimensionality reduction, view inconsistency and discriminative structure learning, lack of principled view fusion mechanisms) are mapped to three separable modules. The idea is clear, and the flow in Figure 1 is intuitive, so that subsequent researchers can learn from or reuse a module.  The experiments have relatively wide coverage and are targeted towards three experimental questions: 6 datasets, 17 baselines, and ACC/NMI dual indicators for main results (RQ1), ablation studies (RQ2), parameter sensitivity analysis (RQ3).

Critical weaknesses
  1. The proofs of the core theoretical claims contain substantive errors or are trivialized. If the theory does not hold, the positioning of the main contribution is undermined, which is the primary reason for rejection.
  2. Key implementation details are missing, which weakens the credibility of the empirical results and makes the conclusions difficult to verify independently.
Questions for authors
  1. Lemma 3.1 I(Zl(v);Y)≥(1−βl)I(X(v);Y)I(\mathbf{Z}_l^{(v)};\mathbf{Y}) \ge (1-\beta^l)I(\mathbf{X}^{(v)};\mathbf{Y}) is a fundamental error. According to β=0.01\beta = 0.01 in Implementation Details, more layers ll mean better information preservation bound and infinite layers construct a sufficient statistic, which obviously conflicts with the data-processing inequality. Similary, when the number of layers is fixed, smaller β\beta means better information preservation bound and β\beta should be zero, which negates IB by deleting the I(Zl(v);X(v))I(\mathbf{Z}_l^{(v)};\mathbf{X}^{(v)}) term in the IB objective. Hence, the proof in Appendix A.1 has to fail at multiple independent steps--Authors should restate Lemma 3.1 and re-prove a correct bound instead of retaining the current derivation.
  2. The entropy regularization term in Eq. 7 contains a non-differentiable indicator function. How is its gradient computed during training? Is it relaxed? If so, the Lipschitz constant in Theorem 3.2 should be re-derived based on the relaxed objective.
  3. Is the submodularity property of mutual information in Eq. 37 correct? Please provide a rigorous justification for this step.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Scope of our response on β, information preservation, and fusion

Thank you for reading our responses. We understand your concern about the scope of the rebuttal. We would like to address the specific example you gave, because the role of β in the submitted method and its use in the original proof are distinct.

In the submission, Eq. (1) already uses β to balance input compression against retention of task information. Eq. (5) already contains the corresponding practical ingredient, a rate penalty with a layer-specific weight. Our response supplied a detail missing from the paper: how that weight scales with β. We agree that this relationship should have been explicit. The quantity being weighted is the same rate term that appears in the submitted factorization objective.

Your initial review also explicitly asked us to “restate Lemma 3.1 and re-prove a correct bound.” We followed that request. The original proof did not establish a retention guarantee from β and depth alone. The revised argument uses the chain rule to show what follows under explicit bounds on the task information lost at each layer. Our answers to the other two questions were similarly specific: for Eq. (7), we gave the soft cluster proportions, their entropy gradient, and a smoothness bound for that objective. For Eq. (37), we replaced the disputed inequality with an identity that accounts for the information lost by the weighted fusion in Eq. (8).

These corrections narrow the theoretical guarantees, and that matters when evaluating the paper. We nevertheless disagree that the example of β establishes that the rebuttal changed the core method. The progressive factorization, contrastive learning, and weighted fusion were all specified in the submission and evaluated in its ablation study. Our response identifies both what the original analysis failed to establish and what can be established for those components.

Authors admitted that "the role of β\beta in the submitted method and its use in the original proof are distinct." and "These corrections narrow the theoretical guarantees, and that matters when evaluating the paper." Good. That is an important reason why I think the respone greatly changed the core of the paper. Since there remains some time. I can discuss Q1 a little bit. In the original paper, the lower bound of the remaining information strictly monotonically rises via the rate 1−βl1-\beta^l. In particular, ll is not bounded in the theory, wihch naturally prompts readers to increase layers for better even perfect information preservation. Of course, at the begining, (1) the rising of the lower bound does not conflict with the decreasing of the upper bound from DPI. However, the upper bound should be no less than the lower one. While (2) the final values of the lower bound 1 is larger than that of the upper one 0 (or a small value near 0), which indicates a cross at a finite layer. What will happen when the two bound inevitably meet because of the theorially infinite layers? This future almost straightforwardly pointed out in Q1 was not directly answered and authors only emphasized (1). In fact, due to DPI, any strictly monotonically rising lower bounds are not unrealistic. Thus, in the response, authors have to represent the bound via a set of ϵ\epsilon in [0,1], which makes the bound become a “correct nonsense”, not only "narrow the theoretical guarantee". In addition, β\beta unralated to the bound become also meaningless.This is a core and fatal question for a paper grounded in the information theory.

Thank you for clarifying the finite-depth question. Yes, a crossing occurs if a layer has already lost some task information and the chain is continued far enough. Suppose that layer retains a fraction r of the input’s task information, with r below one. By the data-processing inequality, every subsequent layer retains at most r. For any fixed β between zero and one, the lower bound in the submitted lemma approaches one, so it exceeds r at a finite depth.

This is why our earlier replies express the retention bound in terms of the loss at each layer. If each layer retains at least a specified fraction of the task information it receives, the product of those fractions gives a lower bound on the fraction retained across the chain. A numerical guarantee therefore requires bounds on the individual layer losses.

The submitted paper introduces β as the coefficient of the input-information term in Eq. (1) and reports how clustering accuracy varies with β in Fig. 2. Eq. (5) writes the layerwise rate penalty with its own weight. Our earlier replies specify the connection between the two. With the base weight fixed at one, β = 0.01 gives the rate penalty unit weight, while β = 0 removes it. Thus β controls how strongly training penalizes rate relative to reconstruction error. The finite-depth argument explains why this training choice alone cannot certify how much task information the learned representations retain.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.