acceptodds
Under review as a conference paper at ICLR 2027

Diagnosing and Preventing MSE-Alignment-Induced Representation Collapse in Unaligned Multi-View Clustering

Abstract

Fully unaligned multi-view clustering aims to recover unknown sample correspondences across views and uncover a shared cluster structure from unaligned representations. A common strategy is to construct learnable cross-view alignment matrices and optimize them using the mean squared error (MSE) loss for sample alignment and cross-view fusion. However, our empirical observations reveal that directly optimizing the alignment matrices and latent representations under the MSE objective can lead to a scale-shrinking shortcut, termed MSE-alignment-induced representation collapse, which compresses inter-sample distances and distorts the underlying cluster structure. To address this issue, we first theoretically analyze the mechanism underlying this collapse and then propose Representation Dispersion-Enhanced Unaligned Multi-view Clustering (RDE-UMC), which effectively mitigates MSE-alignment-induced representation collapse by enhancing the dispersion of latent representations while preserving discriminative cluster structures. Specifically, guided by the analyzed collapse mechanism, we first design a representation dispersion enhancement operator that quantifies dispersion across principal directions to mitigate representation collapse while maintaining the separability of representations in each view. We then incorporate this operator into the MSE objective to formulate Representation Dispersion-Enhanced MSE (RDE-MSE) to perform cross-view alignment on separable representations. Building on the aligned representations, we further integrate the operator into contrastive learning to formulate Representation Dispersion-Enhanced Contrastive Learning (RDE-CL), which further facilitates cross-view consistency fusion among the aligned representations. Experiments on benchmark datasets demonstrate that RDE-UMC effectively mitigates MSE-alignment-induced representation collapse, preserves representation dispersion and discriminability, and improves clustering performance.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Multi-view clustering, Unaligned data, MSE-based alignment, Representation collapse

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_3Py6OpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents RDE-UMC, a method for unaligned multi-view clustering that uses representation dispersion to mitigate feature collapse induced by MSE-based alignment. They introduce a dispersion enhancement operator based on the determinant of the joint feature covariance to measure the spread of representations along the principal directions. This operator is used to normalize the MSE alignment loss, making it invariant to common feature scaling, and to adjust the temperature for contrastive fusion. When dispersion decreases, the temperature is lowered to strengthen the contrastive signal between aligned representations. Experimental results on multiple benchmark datasets demonstrate the effectiveness of the proposed method.

Critical strengths
  1. The dispersion operator is well designed to capture directional collapse in the joint representation space. The geometric mean of the covariance eigenvalues in Eq. (5) approaches zero if any eigenvalue tends to zero. Using this quantity to normalize the alignment loss helps discourage collapse along any direction.
  2. Adapting the temperature to feature dispersion is well motivated. When features shrink to some extent during training, Eq. (9) lowers the temperature to amplify similarity differences, helping preserve discrimination between positive and negative pairs.
Critical weaknesses

Firstly, the paper does not compare the proposed loss with MSE alignment on L2-normalized features. Applying L2 normalization to both the mapped and reference features before computing MSE would also prevent the loss from decreasing solely through uniform feature shrinkage. Without this comparison in Table 4, the additional benefit of dispersion-based normalization remains unclear. Secondly, the advantage of temperature adaptation over a tuned fixed temperature remains unclear. Table 4 includes variants with and without adaptation, but does not state whether the fixed temperature was tuned. The gains may come from using a better temperature value, rather than from adjusting it based on dispersion.

Questions for authors
  1. How much do the clustering results change when a different reference view is used? Section 3.2 selects the view with the largest input dimension, but a higher input dimension does not necessarily imply better learned representations. 2. Why lower the temperature rather than increase the contrastive loss weight when dispersion is low? Increasing λC\lambda_C in Eq. (11) would also strengthen the influence of the contrastive loss on feature updates. 3. How are the mapping matrices PvrP^{vr} initialized before joint training? Algorithm 1 does not specify this. 4. How would clustering performance change if the aligned features were concatenated instead of averaged? Section 3.4 uses an unweighted average, which may lose complementary information from individual views.
2 comments

For Weakness 1

The following table compares RDE-MSE and MSE with L2 normalization and shows that RDE-MSE achieves better clustering performance, since L2 normalization reduces differences in feature magnitude, potentially discarding useful information. For example, two distinct representations, (1,0)(1,0) and (2,0)(2,0), both become (1,0)(1,0) after normalization, making them indistinguishable in the normalized feature space. RDE-MSE instead regulates the representations through the designed objective function, without directly forcing each feature vector to have unit norm.

DatasetMetricMSE with L2 normalizationRDE-MSE
Caltech-5V-7ACC0.33140.9071

For Weakness 2

The fixed temperature was tuned, but even its best setting yielded lower clustering performance than our adaptive temperature. The following table reports the tested fixed-temperature values and their corresponding clustering results, showing that our adaptive scheme outperforms all tested fixed settings. These results show that our adaptive mechanism dynamically adjusts the temperature in response to changes in representation, thereby further improving clustering performance.

Caltech-5V-7Fixed setting 1Fixed setting 2Fixed setting 3Fixed setting 4Adaptive temperature
Temperature setting0.250.50.751.0τbase=0.5\tau_{\mathrm{base}}=0.5
ACC0.85930.86140.86290.85790.9071

For Question 1

The following table shows how clustering performance ACC changes with the reference view on Caltech-5V-7, Aloi-100, and MSRCv1. The view with the largest input dimension generally yields the best clustering performance. We select this view as the reference because the rule is straightforward to apply, requires no additional optimization, and typically gives the best empirical results.

Dataset (view with largest input dimension)Reference view 1Reference view 2Reference view 3Reference view 4Reference view 5
Caltech-5V-7 (3)0.45710.48710.90710.69790.5586
Aloi-100 (4)0.56820.62970.61650.6311N/A
MSRCv1 (2)0.34760.78570.69050.64760.5667

For Question 2

We lower the temperature because it amplifies differences between the similarity scores used by the softmax, helping distinguish samples even when dispersion is low; increasing λC\lambda_C alone cannot produce this effect. Specifically, a lower temperature sharpens the softmax distribution, which allows the contrastive objective to focus on separating easily confused samples when dispersion is low. Increasing λC\lambda_C only raises the overall weight of the contrastive loss and cannot provide the same adjustment.

For Question 3

We use a simple randomized initialization for each mapping matrix PvrP^{vr} before joint training. This is straightforward to implement and performs well in our experiments. We will specify this initialization in Algorithm 1.

For Question 4

Replacing averaging with concatenation changes ACC from 0.9071 to 0.9014 on Caltech-5V-7 and from 0.6311 to 0.6006 on Aloi-100, as shown in the following table. Our method uses contrastive learning to extract cross-view consistent information for clustering, while attenuating noise and complementary components that are difficult to distinguish. For these consistent features, averaging forms a compact consensus representation in the shared feature space, better supporting clustering based on cross-view consistency. The experimental results also support this choice.

DatasetMetricAveragingConcatenation
Caltech-5V-7ACC0.90710.9014
Aloi-100ACC0.63110.6006

I thank the authors for their response, which has addressed my concerns. I recommend accepting this paper.

Reviewer_5rteOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes RDE-UMC for unaligned multi-view clustering, motivated by a limitation of MSE-based alignment: MSE loss can be reduced via feature shrinkage without improving cross-view correspondences. The method first learns view-specific representations with autoencoders and computes a dispersion measure from the joint covariance of two views. Then, this measure is used to normalize the MSE alignment loss, canceling the effect of common feature scaling, and to adapt the contrastive temperature for cross-view contrastive fusion. After that, the fused representations are clustered with K-means. The paper provides a scaling analysis, reports improved clustering performance on eight datasets, and includes component ablations.

Critical strengths
  1. The paper identifies a limitation of MSE alignment: the loss can decrease simply because the features shrink, without any improvement in sample matching. The authors also show that adding reconstruction loss does not rule out this behavior. To improve alignment, the proposed method removes the effect of feature shrinkage on the alignment loss, encouraging the model to reduce the loss through more accurate alignment.

  2. The experimental results provide strong support for the method's effectiveness and its main design choices. Across eight datasets, RDE-UMC achieves the best results on nearly all reported metrics. The ablation studies compare different loss combinations and isolate the contributions of dispersion-based loss normalization and temperature adaptation.

Critical weaknesses
  1. The claim that feature collapse harms clustering performance lacks direct experimental evidence. Figures 1 and 5 show that collapse occurs, but do not demonstrate that it reduces clustering performance.

  2. How much additional training time does the dispersion operator introduce? Table 4 reports its effect on clustering performance, but not the cost of computing the covariance matrix and its determinant at every iteration.

Questions for authors
  1. How does the implementation handle a zero or near-zero denominator in Eq. (6)? The denominator is computed from the covariance determinant. When the covariance determinant approaches zero, the denominator also approaches zero, potentially making the loss numerically unstable.

  2. How would clustering performance change if you used average variance, tr⁡(G)/d\operatorname{tr}(G)/d, instead of the proposed dispersion enhancement operator? Since the average variance also seems to make the loss invariant to uniform feature scaling. Comparing the two measures would therefore clarify whether the operator provides additional clustering benefits. Table 4 does not include this comparison.

  3. What value of the base temperature τbase\tau_{\mathrm{base}} was used in the experiments? This parameter is used to compute the effective temperature in Eq. (9), but its value is not reported in the paper.

1 comment

For Weakness 1

The following table provides experimental evidence that feature collapse harms clustering performance. On Caltech-5V-7, ordinary MSE and RDE-MSE are compared under the same experimental setting. The ordinary-MSE variant replaces the anti-collapse RDE-MSE alignment with ordinary MSE. During training, this variant undergoes gradual feature collapse, accompanied by degraded clustering performance. RDE-MSE retains the anti-collapse mechanism, allowing the alignment objective to focus on learning sample correspondences. These results show that the mse-alignment induced feature collapse dose harm clustering performance.

MethodMeasurementEpoch 0Epoch 30Epoch 60Epoch 90Epoch 120Epoch 150
Ordinary MSEMean feature norm2.70620.09550.08140.07540.07190.0696
Ordinary MSEACC0.70210.19640.19070.24070.26430.2857
RDE-MSE-onlyMean feature norm2.70622.32422.34522.37232.40852.4500
RDE-MSE-onlyACC0.70000.84790.86210.87140.88070.8971

For Weakness 2

The following table compares training times with and without the dispersion operator, showing a low overhead of at most 2.307%. For each view pair, the operator is D(Zvr)=det⁡(G(Zvr))1/dD(Z^{vr})=\det(G(Z^{vr}))^{1/d}. Its dominant cost is constructing G(Zvr)=Zcvr(Zcvr)⊤/(2N)G(Z^{vr})=Z_c^{vr}(Z_c^{vr})^\top/(2N), where Zcvr∈Rd×2NZ_c^{vr}\in\mathbb{R}^{d\times 2N} denotes the centered joint representation matrix and computing G(Zvr)G(Z^{vr}) requires O(Nd2)O(Nd^2) operations. Since d≪Nd\ll N, the operator's computational cost scales linearly with the number of samples NN.

Training time on Caltech-5V-7Without operatorWith operatorOverhead (%)
Per epoch (s)0.70210.71832.307
Total training (s)105.31107.742.307

For Question 1

When the covariance determinant in Eq. (6) is zero or extremely small, we set the denominator to no less than 1×10−121\times10^{-12} before computing the normalized loss. This prevents division by zero, overflow, and NaN values during training, thereby improving numerical stability during training.

For Question 2

The following table compares the clustering performance of the two measures and shows that using tr⁡(G)/d\operatorname{tr}(G)/d leads to lower performance, since it may fail to detect collapse in any individual direction and cannot reliably prevent such collapse. For example, when G=diag⁡(2−δ,δ)G=\operatorname{diag}(2-\delta,\delta), the average variance remains 1 as δ→0+\delta\to0^+, whereas the proposed dispersion (2−δ)δ\sqrt{(2-\delta)\delta} approaches zero. This shows that our operator detects this collapse and helps prevent it. The results in the table further support the benefit of this property for clustering.

DatasetMetricAverage variance tr⁡(G)/d\operatorname{tr}(G)/dProposed operator det⁡(G)1/d\det(G)^{1/d}
Caltech-5V-7ACC0.85640.9071

For Question 3

The base temperature was set to τbase=0.5\tau_{\mathrm{base}}=0.5. It remains fixed during training. We will explicitly report the base-temperature value in the paper.

Reviewer_E2j9OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper examines an optimization problem in MSE-based multi-view alignment: the model can lower MSE by shrinking its features, reducing its incentive to uncover sample correspondences. The authors document this behavior by tracking feature norms during training and use a scaling analysis to show that adding a reconstruction loss does not rule it out. RDE-UMC addresses the problem by using representation dispersion to adjust the alignment objective, limiting loss reduction induced by feature shrinkage and encouraging the model to improve sample matching. Experiments report improved clustering performance on eight datasets. Training curves, loss-weight sensitivity analysis, and component ablations further assess convergence and how the loss weights and individual components affect the results.

Critical strengths

The controlled experiments are well designed and provide clear evidence that MSE alignment induces feature shrinkage. In Figures 1(b)–(d), the authors compare training with and without the standard MSE alignment loss while keeping the autoencoder architecture and all other training settings unchanged. Across all three datasets, feature norms remain relatively stable with reconstruction loss alone but decrease steadily when MSE alignment is added.

Critical weaknesses
  • The paper does not provide sufficient experimental evidence that collapse is mitigated in each view. Figure 5 does not report separate feature-norm curves for individual views.

  • The advantage of the proposed normalization over existing anti-collapse regularizers has not been established empirically. Appendix A.3 discusses VICReg's variance and covariance regularizers, but Table 4 only compares different uses of the proposed operator. It does not include a baseline that combines ordinary MSE with these regularizers in the same framework.

Questions for authors
  • Would feature norms continue to shrink if only the alignment loss in MVC-UM were replaced with RDE-MSE? Figure 1(a) shows feature shrinkage in MVC-UM. This comparison would test whether the proposed loss modification also mitigates collapse in an existing method.

  • Why normalize each alignment loss by the dispersion of its view pair rather than a shared estimate computed from all views?

  • Why is the temperature in Eq. (9) directly proportional to dispersion? The paper explains why the temperature is lowered as dispersion decreases, but not why this particular linear relationship is used.

  • What values of the loss weights λA\lambda_A and λC\lambda_C were used for each dataset in Table 2? Figure 4 presents a sensitivity analysis, but the paper does not specify the values used in the main experiments.

2 comments

For Weakness 1

We provide additional per-view representation-norm analyses, demonstrating that RDE-UMC effectively mitigates representation collapse across individual views. Ordinary MSE drives representation norms toward zero, with an average reduction of 98.82% by epoch 150. In contrast, RDE-UMC maintains non-vanishing norms on the order of 1, ranging from 0.3065 to 2.3329 at epoch 150. These per-view results provide direct evidence that RDE-UMC mitigates norm collapse consistently across all five views.

SettingE0E30E60E90E120E150Norm reduction from E0 to E150 (%)
MSE-V12.40310.03520.01340.00970.00950.015399.36
MSE-V22.10900.04180.01210.00980.00870.008799.59
MSE-V31.88000.03310.01260.00950.00830.008599.55
MSE-V42.65030.03300.01300.01090.00990.010099.62
MSE-V54.30100.69830.23480.19390.17300.173895.96
RDE-V12.40310.47810.51070.58490.60500.626273.94
RDE-V22.10900.85971.11971.07761.10381.074449.06
RDE-V31.88000.25490.42880.39090.39810.387979.37
RDE-V42.65030.68500.31200.30730.30830.306588.44
RDE-V54.30102.17471.95982.13002.34602.332945.76

For Weakness 2

We directly compare RDE-MSE with ordinary MSE augmented with VICReg’s variance and covariance regularizers within the same framework. As shown below, RDE-MSE consistently outperforms this baseline across all four clustering metrics on both datasets. These results provide direct empirical evidence for the advantage of our normalization over adding separate anti-collapse regularizers to ordinary MSE.

DatasetMethodACCNMIARIPUR
Caltech-5V-7MSE + VICReg0.88430.77470.76540.8843
Caltech-5V-7RDE-MSE0.90710.81390.81050.9071
Aloi-100MSE + VICReg0.22220.48570.12280.2606
Aloi-100RDE-MSE0.63110.76300.48710.6508

For Question 1

Replacing MVC-UM's ordinary MSE alignment loss with RDE-MSE also mitigates feature-norm shrinkage in the existing multi-view clustering framework. We keep the original framework and all other experimental settings unchanged. As shown below, standard MVC-UM reduces the mean representation norms to near-zero values (0.0065–0.0281), whereas MVC-UM + RDE-MSE retains enough norms (3.9299–4.3827) at the end of training. This controlled comparison demonstrates that the alignment-loss replacement alone mitigates norm collapse in an existing method, and that its benefit is not limited to the full RDE-UMC model.

DatasetMethodInitial Mean NormFinal Mean Norm
Caltech-5V-7MVC-UM5.46880.0281
Caltech-5V-7MVC-UM + RDE-MSE5.46883.9299
BBCSportMVC-UM5.48120.0080
BBCSportMVC-UM + RDE-MSE5.48124.3827
REUMVC-UM5.47030.0065
REUMVC-UM + RDE-MSE5.47034.3006

For Question 2

The alignment objective is intrinsically pairwise: each non-reference view vv is aligned with the reference view rr, and the complete objective sums these pairwise terms:

LRDE-MSE=∑v≠r∥PvrZv−Zr∥F2ND(Zvr)\displaystyle \mathcal{L}_{\text{RDE-MSE}}=\sum_{v\neq r}\frac{\lVert P^{vr}Z^v-Z^r\rVert_F^2}{N\mathcal{D}(Z^{vr})}

Here, ZvrZ^{vr} denotes the joint representation of views vv and rr, and PvrP^{vr} maps view vv to the reference view. We use pair-specific dispersion to match this pairwise loss design. A shared global estimate can not adequately reflect the scale of each view pair and can not provide appropriate normalization for every pairwise loss term.

For Question 3

We use the linear mapping because it establishes the most direct proportional relationship between temperature and representation dispersion while also performing well in our experiments. We compare fixed-temperature, square-root, linear, and quadratic mappings on Caltech-5V-7. Among the three adaptive mappings tested, the linear mapping achieves the highest ACC on Caltech-5V-7.

MappingACC
Fixed temperature0.8614
Square-root0.9043
Linear (Ours)0.9071
Quadratic0.8900

For Question 4

We use λA=0.1\lambda_A=0.1 and λC=1.0\lambda_C=1.0 for every dataset. We will explicitly report the values in the paper.

The authors have addressed most of my concerns, so I have decided to increase the score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.