acceptodds
Under review as a conference paper at ICLR 2027

Missing-View Self-Distillation Imputation for Incomplete Multi-View Clustering

Abstract

Real-world multi-view data often contain missing views due to failures during acquisition, transmission, or storage. This makes it difficult to directly apply multi-view clustering methods that require complete samples. Existing methods typically recover missing information through cross-view prediction, neighborhood structure, or relation propagation. However, some graph-based methods rely on shared relations that may not capture specific structure of target view, potentially introducing bias into the recovered representations. To address this limitation, we propose Missing-View Self-Distillation Imputation for Incomplete Multi-View Clustering (MVSI-IMVC). Specifically, we construct a target-view self-distillation predictor. We use full-information and pseudo-missing states from complete samples and use the observed target view to provide direct supervision. Teacher predictions further guide target-view latent recovery for incomplete samples. This allows the model to learn predictive relationships specific to each target view and missing pattern. We also introduce Henze–Penrose divergence-guided fused representation learning. Together with a cross-view instance contrastive loss, it organizes completed latent representations at the instance and distribution levels and connects target-view latent recovery with fused representation learning and clustering. Experiments show that MVSI-IMVC achieves competitive clustering performance on multiple benchmark datasets.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
incomplete multi-view clustering, missing-view completion, multi-view representation learning, cross-view latent prediction, self-distillation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_BnHrOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The manuscript proposes MVSI-IMVC (Missing-View Self-Distillation Imputation for Incomplete Multi-View Clustering). It establishes a target-view self-distillation predictor. The idea is to exploit full-information and pseudo-missing states from complete samples as well as the observed target view as supervision information. Then teacher predictions are used to guide target-view latent recovery for incomplete samples. The following shortcomings are severe. The paper have no very solid contributions.

Critical strengths

A couple of methods are used to conduct experimental comparison.

Critical weaknesses

First, it seems that there have no enough evidences for that the proposed scheme can work well. It is unconvincing that self-distillation is competent in extracting effective information. Second, a distillation network can perform well only under the condition that there are a large amount of samples. However, I see that the used datasets do not satisfy this point. Third, four datasets from different fields are selected for experiments. Moreover, the used handWritten dataset (Duin, 1998), the Scene-15 dataset (Lazebnik et al., 2006), the LandUse-21 (Yang & Newsam, 2010) dataset, and the ALOI-100 dataset were constructed a long time ago. At present many large-scale datasets are available and more suitable.

Questions for authors
  1. Why is the desigined self-distillation scheme effective for missing-views? And how many samples are enough for the experiments? I hope to see convincing explanations and evidences.
  2. It is not known how to properly set the parameters of the self-distillation scheme.
Reviewer_RgJjOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a incomplete multi-view clustering method. It constructs pseudo-missing states from complete samples and uses target-view supervision and self-distillation to recover missing-view latent representations. Cross-view instance contrastive learning and Henze Penrose divergence are further employed to optimize the fused representations, followed by K-means clustering.

Critical strengths
  1. The method constructs pseudo-missing states from complete samples, enabling direct target-view supervision for learning view-specific recovery relationships.

  2. The framework combines target-view latent recovery, cross-view contrastive learning, and Henze Penrose divergence-based representation fusion for incomplete multi-view clustering.

Critical weaknesses
  1. The contribution of self-distillation is not sufficiently demonstrated. In Eqs. (5)–(8), both teacher and student predictions are supervised by the target-view representation, and the teacher uses the same predictor with complete-view inputs. It is therefore unclear what additional benefit self-distillation provides over direct target-view prediction. The authors should separately evaluate the contributions of recovery supervision, teacher grounding, and self-distillation.

  2. The Henze–Penrose divergence-based association loss needs further justification. Eq. (16) combines Henze Penrose divergence with marginal entropy regularization, but their individual contributions are not evaluated. Moreover, the variance-based weights in Eq. (13) measure feature concentration rather than necessarily reflecting view reliability. Additional ablations and comparisons with alternative association losses and fusion strategies would help justify these choices.

  3. The cross-view contrastive loss formulation requires clarification. Eq. (12) excludes positive pairs from the denominator, unlike the standard InfoNCE objective. The authors should explain the motivation for this design and compare its performance with standard InfoNCE.

Questions for authors
  1. Could the authors provide separate ablations for the recovery, teacher grounding, and self-distillation losses? In particular, how much does self-distillation improve over direct target-view prediction without teacher guidance?

  2. How much does the Henze Penrose divergence contribute compared with marginal entropy regularization? How does the proposed variance-based fusion compare with uniform weighting or other fusion strategies?

  3. Why are positive pairs excluded from the denominator in Eq. (12)? How does this formulation perform compared with standard InfoNCE?

Reviewer_F1MXOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

Aiming at the problem of incomplete multi-view clustering, this paper proposes MVSI-IMVC: a self-distillation-based missing view imputation method for incomplete multi-view clustering. The method constructs a target view self-distillation predictor based on the full-information teacher branch and the pseudo-missing student branch, combined with target view self-supervision and distillation guidance to learn the view-specific missing latent representation imputation ability, while combining cross-view instance contrastive loss and Henze–Penrose divergence-guided adaptive fusion mechanism to optimize the view imputation, feature fusion, and clustering tasks end-to-end. The method is evaluated on four datasets and multiple missing rates, and the experimental results show that the method can significantly improve the clustering performance, especially for the high missing rate cases.

Critical strengths

This paper has a clear research motivation. The proposed self-distillation completion strategy takes the observations of the target view itself as the supervision anchor and incorporates teacher guidance from the full-view context. It not only exploits the complementary information among multiple views but also ensures that the completion results conform to the true distribution of the target view. Through extensive experiments, the authors verify that the proposed method can significantly improve clustering performance in most cases, especially under high missing rates.

Critical weaknesses

This method designs a separate predictor network PvP_v for each view. The number of parameters and computational cost increase linearly with the number of views VV. For scenarios with an extremely large number of views (more than dozens of views), the training and inference costs rise significantly. Meanwhile, the Henze–Penrose divergence requires computing a d×dd\times d joint distribution matrix, which introduces extra computational overhead when the latent dimension dd is large.

Pseudo-missing masks are sampled from the empirical missing distribution, which essentially assumes that missingness is completely at random. However, missingness in real-world scenarios is often missing not at random. For instance, certain types of samples are more prone to missing specific views, and the missing probability is correlated with sample features. In this case, the self-distillation completion relationships learned by the model under random missingness may not adapt to the real non-random missing patterns, and this scenario is not validated in the paper.

Questions for authors
  1. The teacher branch takes the latent representations of all views as input, while incomplete samples may retain only one observed view during real inference. There exists a huge gap in the amount of input information between the student and the teacher. Especially under a high missing rate of 0.7, most samples have very few available views. Can the distillation signals from the teacher’s predictions still effectively guide the completion task under this circumstance? Is there a risk that excessively strong teacher information may mislead the student instead?

  2. Each view corresponds to an independent predictor PvP_v. The model parameters will expand significantly when the number of views is large. Has the idea of sharing the backbone network of predictors and only adopting lightweight view-specific heads to adapt to different target views been considered? How much impact does this lightweight design have on model performance? Compared with other baselines, what are the GPU memory usage and single-epoch training time of the proposed method on this dataset? Are there statistics related to computational efficiency?

  3. The paper adopts Henze–Penrose divergence to measure the statistical correlation between views and the fused representation, instead of more commonly used metrics such as JS divergence and the InfoNCE lower bound of mutual information. What are the core advantages of this divergence in the scenario of multi-view clustering fusion? Have ablation and comparison experiments with other divergence/correlation metrics been supplemented?

  4. Some distillation-based IMVC methods have been proposed. Please supplement them in the related work section and elaborate on the differences and advantages compared with these methods.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.