- Rating
- 4(+1)
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents RDE-UMC, a method for unaligned multi-view clustering that uses representation dispersion to mitigate feature collapse induced by MSE-based alignment. They introduce a dispersion enhancement operator based on the determinant of the joint feature covariance to measure the spread of representations along the principal directions. This operator is used to normalize the MSE alignment loss, making it invariant to common feature scaling, and to adjust the temperature for contrastive fusion. When dispersion decreases, the temperature is lowered to strengthen the contrastive signal between aligned representations. Experimental results on multiple benchmark datasets demonstrate the effectiveness of the proposed method.
Critical strengths
- The dispersion operator is well designed to capture directional collapse in the joint representation space. The geometric mean of the covariance eigenvalues in Eq. (5) approaches zero if any eigenvalue tends to zero. Using this quantity to normalize the alignment loss helps discourage collapse along any direction.
- Adapting the temperature to feature dispersion is well motivated. When features shrink to some extent during training, Eq. (9) lowers the temperature to amplify similarity differences, helping preserve discrimination between positive and negative pairs.
Critical weaknesses
Firstly, the paper does not compare the proposed loss with MSE alignment on L2-normalized features. Applying L2 normalization to both the mapped and reference features before computing MSE would also prevent the loss from decreasing solely through uniform feature shrinkage. Without this comparison in Table 4, the additional benefit of dispersion-based normalization remains unclear. Secondly, the advantage of temperature adaptation over a tuned fixed temperature remains unclear. Table 4 includes variants with and without adaptation, but does not state whether the fixed temperature was tuned. The gains may come from using a better temperature value, rather than from adjusting it based on dispersion.
Questions for authors
- How much do the clustering results change when a different reference view is used? Section 3.2 selects the view with the largest input dimension, but a higher input dimension does not necessarily imply better learned representations. 2. Why lower the temperature rather than increase the contrastive loss weight when dispersion is low? Increasing in Eq. (11) would also strengthen the influence of the contrastive loss on feature updates. 3. How are the mapping matrices initialized before joint training? Algorithm 1 does not specify this. 4. How would clustering performance change if the aligned features were concatenated instead of averaged? Section 3.4 uses an unweighted average, which may lose complementary information from individual views.
2 comments
I thank the authors for their response, which has addressed my concerns. I recommend accepting this paper.
For Weakness 1
The following table compares RDE-MSE and MSE with L2 normalization and shows that RDE-MSE achieves better clustering performance, since L2 normalization reduces differences in feature magnitude, potentially discarding useful information. For example, two distinct representations, (1,0) and (2,0), both become (1,0) after normalization, making them indistinguishable in the normalized feature space. RDE-MSE instead regulates the representations through the designed objective function, without directly forcing each feature vector to have unit norm.
For Weakness 2
The fixed temperature was tuned, but even its best setting yielded lower clustering performance than our adaptive temperature. The following table reports the tested fixed-temperature values and their corresponding clustering results, showing that our adaptive scheme outperforms all tested fixed settings. These results show that our adaptive mechanism dynamically adjusts the temperature in response to changes in representation, thereby further improving clustering performance.
For Question 1
The following table shows how clustering performance ACC changes with the reference view on Caltech-5V-7, Aloi-100, and MSRCv1. The view with the largest input dimension generally yields the best clustering performance. We select this view as the reference because the rule is straightforward to apply, requires no additional optimization, and typically gives the best empirical results.
For Question 2
We lower the temperature because it amplifies differences between the similarity scores used by the softmax, helping distinguish samples even when dispersion is low; increasing λC alone cannot produce this effect. Specifically, a lower temperature sharpens the softmax distribution, which allows the contrastive objective to focus on separating easily confused samples when dispersion is low. Increasing λC only raises the overall weight of the contrastive loss and cannot provide the same adjustment.
For Question 3
We use a simple randomized initialization for each mapping matrix Pvr before joint training. This is straightforward to implement and performs well in our experiments. We will specify this initialization in Algorithm 1.
For Question 4
Replacing averaging with concatenation changes ACC from 0.9071 to 0.9014 on Caltech-5V-7 and from 0.6311 to 0.6006 on Aloi-100, as shown in the following table. Our method uses contrastive learning to extract cross-view consistent information for clustering, while attenuating noise and complementary components that are difficult to distinguish. For these consistent features, averaging forms a compact consensus representation in the shared feature space, better supporting clustering based on cross-view consistency. The experimental results also support this choice.