acceptodds
Withdrawn from ICLR 2027

Relative Semantic Representation Learning for Partially View-Aligned Clustering

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Partially view-aligned clustering (PVC) aims to cluster multi-view data with partially known cross-view correspondences. However, existing PVC methods still face two limitations: i) view-specific representations may not sufficiently capture the relational semantics required for reliable cross-view correspondence estimation; and ii) a uniform association strategy may underutilize known correspondences and yield unreliable associations for unaligned samples. To address these issues, we propose Relative Semantic Representation Learning for Partially View-Aligned Clustering (RSR-PVC). Specifically, each sample is represented by a normalized distribution of its relations to the cross-view aligned samples, forming a relative semantic representation for cross-view comparison. Subsequently, a hard-to-soft relation-space association mechanism is developed to handle aligned and unaligned samples differently. Known aligned correspondences guide bidirectional matching, while associations for unaligned samples are modeled with soft distributions. Finally, the resulting associations are incorporated into relational consistency learning to enhance cross-view representation consistency. Extensive experiments on eight benchmark datasets against seven state-of-the-art methods demonstrate the effectiveness of RSR-PVC.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
multi-view clustering, partially view-aligned clustering, representation learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_5cbvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RSR-PVC for partially view-aligned clustering. The method constructs relative semantic representations using known aligned samples as shared references to facilitate cross-view comparison. It further introduces a hard-to-soft association mechanism that combines bidirectional alignment for known correspondences with soft association and relational consistency learning for unaligned samples. Experiments on eight benchmark datasets report competitive clustering performance against seven baseline methods.

Critical strengths
  1. The proposed framework integrates relative semantic representation learning, cross-view association, and multi-view fusion into a unified pipeline.
  2. The method explicitly distinguishes between known and unknown cross-view correspondences and adopts different association strategies for each.
  3. The experiments cover eight benchmark datasets with different unaligned ratios. Ablation studies and additional analyses are provided to examine the contributions of individual components.
Critical weaknesses
  1. The paper identifies limited cross-view semantic comparability and unreliable correspondence estimation as two major limitations of existing PVC methods. However, prior works such as [A] [B] [C] have already addressed these challenges through relational matching, distribution alignment, confidence estimation, and soft association. Although these methods are discussed in the Related Work section, they are not included in the experimental comparisons, making it difficult to assess the advantages of the proposed method over closely related approaches.
  2. he proposed relative semantic representation relies heavily on the aligned samples used as semantic references. However, the experiments only examine varying unaligned ratios, without considering the representativeness of the reference set. Even at a fixed alignment ratio, different class distributions or limited class coverage may substantially affect the quality of the resulting relation space. This dependency remains insufficiently investigated.
  3. The cross-view association matrix is constructed within each mini-batch, restricting candidate correspondences to samples in the same batch. The paper does not clarify how such cross-batch correspondences are handled or how this limitation affects association quality.

[A] Wang et al. Contrastive Graph Distribution Alignment for Partially View-Aligned Clustering.

[B] Peng et al. SMART: Semantic Matching Contrastive Learning for Partially View-Aligned Clustering.

[C] Xu et al. Partial Contrastive Learning for Partially View-Aligned Multi-View Clustering.

Questions for authors
  1. Could the authors further clarify the key differences between RSR-PVC and the closely related methods? It would also be helpful to include direct experimental comparisons with these methods to better demonstrate the advantages of the proposed approach.
  2. How sensitive is the proposed relative semantic representation to the representativeness of the aligned reference samples? Could the authors evaluate its robustness under different reference class distributions or varying class coverage while keeping the alignment ratio fixed?
  3. How does the proposed method handle cases where the true cross-view counterparts of unaligned samples fall into different mini-batches? Could the authors clarify the impact of batch-wise association on the quality of learned correspondences, particularly for larger datasets?
Reviewer_bQQ1OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates the Partially View‑Aligned Clustering (PVC) task. To address the drawbacks of existing methods — that cross‑view matching directly performed in the original feature space is vulnerable to view‑specific distribution shifts, and uniform association strategies fail to differentially exploit known aligned samples and unaligned samples — the authors propose the RSR‑PVC method. RSR‑PVC constructs relative semantic representations using aligned samples as shared references, and further develops a hard‑soft hybrid association mechanism. It applies hard bidirectional‑matching constraints for known aligned samples, models soft association distributions for unaligned samples, and completes multi‑view representation fusion and clustering via relational consistency loss. The paper conducts ablation studies and robustness tests under different unaligned rates across eight benchmark datasets, and experimental results are reported to validate the effectiveness of the proposed method. However, this method heavily relies on the quality and quantity of aligned reference samples; the template view must be manually predefined; it involves numerous hyper‑parameters that impose heavy tuning burdens; furthermore, high computational complexity raises scalability concerns.

Critical strengths

(1) The hard‑soft hybrid association mechanism designed in this paper avoids false pseudo‑correspondences introduced by one‑hot matching. It is a relatively original technique and provides a new insight for the partially view‑aligned clustering task.

(2) This manuscript contains extensive experiments, including comparison experiments, ablation studies, robustness evaluations, runtime overhead analysis, computational‑complexity analysis and other comprehensive tests, forming a complete chain of experimental evidence.

Critical weaknesses

(1) Equation (12) contains four loss terms and three balancing hyper‑parameters. Besides, there are additional hyper‑parameters such as temperature coefficients and neighbor counts. In my opinion, this makes the proposed RSR‑PVC extremely difficult to tune, which limits its practical deployment in real‑world scenarios.

(2) Except for ReutersEF and Cifar10, the remaining datasets are too small to guarantee convincing and solid experimental conclusions.

Questions for authors

(1) I argue that the performance of the proposed RSR‑PVC strongly depends on the quality and quantity of aligned reference samples. Relative semantic representations fully rely on in‑batch aligned samples as references. If the number of aligned samples within a batch is very small, or those aligned samples themselves contain noise, the reliability of the entire relation space will be directly undermined. The paper lacks systematic experimental evaluation for this risk. I expect the authors to provide explanations on this issue. (2) The template view is manually predefined in the paper, which introduces potential bias. The authors fix the first view as the template view, whereas there exists no natural "template view" in real‑world scenarios. Different template‑view selections may lead to performance fluctuations, yet this issue is barely discussed in the main text. (3) The proposed method incurs high computational cost and exhibits poor scalability. Computing the relation‑similarity matrix brings O(B2)O(B^2) complexity. When dealing with extremely large‑scale datasets together with a large number of aligned reference samples per batch, time and memory overhead will inflate rapidly. Although runtime comparisons are provided in the appendix, the paper does not test performance and GPU‑memory bottlenecks under ultra‑large‑dataset settings. (4) Other questions can be found in Critical Weaknesses.

Reviewer_ptS7OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a multi-view clustering framework, RSR-PVC, for partially view-unaligned data. The authors' main idea is to use aligned samples to provide a suitable reference space for unaligned samples, thereby addressing the problem of incomparable representations of unaligned samples. This approach is reasonable. In addition, the paper designs a "hard-to-soft" cross-view association mechanism, and the learned association matrix directly serves the subsequent multi-view fusion and clustering. The paper validates the proposed method on multiple datasets and achieves improved performance.

Critical strengths
  1. This paper provides a very good perspective to address the cross view incomparability of misaligned samples, as they are in different representation spaces.
  2. The proposed method has been validated on multiple datasets. Despite the relatively small dataset sizes, the experimental results are encouraging, and the method exhibits significant advantages over competing approaches, particularly under higher misalignment rates.
Critical weaknesses
  1. This method fixes the first view as a template view and postpones flexible template selection to future work, but template views are an important structure of RSR-PVC, so the author should at least validate the impact of template view quality by iterating each view as a template experiment, rather than treating it as future work.
  2. In the early stages of training, even aligned views have not yet aligned their representation space, so the relative semantic representations constructed from aligned references may be unreliable at the time, leading to systematic errors in early associations and even being solidified by subsequent training. The paper did not analyze this early unreliability and its impact on the final correlation quality.
  3. The author should provide the hyperparameter search range of the compared methods, as experimental results show that some methods perform very poorly on certain datasets.
  4. The paper uses the Bhattacharyya coefficient to measure the similarity between two relative semantic distributions in eq. (6) and directly uses it without discussing why this metric was chosen over other distribution metrics.
Questions for authors
  1. Can the method still be executed normally if there are no aligned samples in a batch? How do you ensure that there are aligned samples in each batch?
  2. The stop-gradient operation is applied to the association matrix during aggregation, but should the similarity terms used in the alignment loss also be detached from the gradient?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.