acceptodds
Under review as a conference paper at ICLR 2027

Reliable Correspondence Learning for Incomplete and Unaligned Multi-View Clustering

16 views

Abstract

Incomplete and unaligned multi-view clustering (IUMVC) has emerged as a challenging paradigm for discovering shared cluster structures from multiple views, where observations may be missing and cross-view sample correspondence is unknown. However, the unknown correspondence makes reliable cross-view matching difficult, as missing observations can result in multiple plausible matches, while erroneous correspondences may propagate errors to subsequent representation learning and multi-view consensus. To address these challenges, we propose a reliable correspondence learning framework, termed CAMVC, for robust clustering of unaligned incomplete multi-view data. Specifically, we first establish a canonical coordinate system without using paired samples and develop a cluster-routed local sparse transport mechanism to identify plausible correspondences, avoiding both premature hard matching and exhaustive global matching. Furthermore, we exploit correspondence-induced reconstruction consistency to quantify sample-level reliability and feature-dependent residual variation, allowing unreliable correspondences to exert less influence on subsequent representation learning. Finally, we integrate cross-view geometric consistency with reliability-aware consensus and high-order tensor regularization to capture complementary and higher-order dependencies across views, followed by reliability- and coverage-aware fusion for clustering. Extensive experiments on six benchmarks under varying levels of missingness and misalignment demonstrate that CAMVC improves clustering performance and effectively learns cross-view correspondences, even as the common sample-order assumption is progressively relaxed.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Multi-view clustering

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_vdwtOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes CAMVC for incomplete and unaligned multi-view clustering. It establishes canonical coordinates using an automatically selected reference view and learns sparse soft correspondences through cluster-routed local sparse transport (CLST). Experiments cover six datasets under different missingness and unalignment settings.

Critical strengths

The paper addresses a practically relevant problem in multi-view clustering, where missing observations and unknown cross-view correspondences occur together. Learning without known sample pairs makes the setting particularly diffculty . Second, CAMVC provides a coherent framework that connects correspondence learning with sample- and view-level reliability. The integration of correspondence uncertainty into representation learning and structural integration is a meaningful aspect of the method. Experiments on six benchmarks under different missingness and unalignment settings provide encouraging evidence of the method’s effectiveness.

Critical weaknesses

The paper treats reconstruction consistency as a reliability proxy, but its interpretation in the presence of feature-dependent residual variation needs further explanation. The ablation experiments are conducted only at a 10% missingness rate, which is the easiest case. Can the performance be maintained at other missingness rates? Questions:

  1. How should the sample-specific reliability weights be interpreted when the observed reconstruction residual is affected by feature-dependent residual variation?
  2. On Synthetic3d at 50% missingness, CAMVC achieves an ACC of 0.5867. On Synthetic3d at 30% missingness, CAMVC achieves an ACC of 0.7183. With a decrease in the missingness rate, the results differ so greatly. What drives such a large improvement in performance?
  3. What motivates the baseline 1/n1/n? How does it account for an assignment that is uniform over the retained candidates?
Questions for authors

See Weakness

1 comment
AuthorsOpenReview ↗

Response to vdwt

Thank you for your detailed review. We summarize our responses below.

Q1. Sample reliability under feature-dependent residual variation.

A: The weight qi=exp⁡!(−∥R:,i∥222σmcc2)q_i=\exp\\!\left(-\frac{\lVert R_{:,i}\rVert_2^2}{2\sigma_{\mathrm{mcc}}^2}\right) controls a sample’s influence on learning; it is not a probability that its correspondence is correct. A large residual may reflect correspondence errors, feature noise, or missingness-induced representation errors. The covariance term serves a different role by modeling variation across feature directions. As shown in Eq. (21), the sample and covariance terms contribute additively to the residual update.

Nevertheless, the Euclidean definition of qiq_i remains sensitive to feature-dependent residual variation after standardization. We will make this limitation explicit in Section 3.3 and Appendix B.2 and evaluate a covariance-normalized alternative using R:,i⊤(Σ(v))−1R:,iR_{:,i}^{\top}(\Sigma^{(v)})^{-1}R_{:,i}. This comparison will assess the sensitivity of the reliability mechanism to the residual metric.

Q2&W1. Higher-missingness ablations.

A: We agree that the component analysis should cover higher missing rates and extend the ablations to 30% and 50% missingness.

Table R1: Ablation results (%) at 30% missingness.

DatasetMetricFullGlobalHardUniSampleUniGraph
Synthetic3dACC71.83063.33360.50067.33367.500
NMI23.87021.54325.62724.23224.363
PUR71.83063.33360.50067.33367.500
F-score50.71049.77751.40752.09452.197
ReutersACC44.23039.41742.08341.75039.417
NMI39.37013.58415.35715.84414.725
PUR47.42041.75042.33342.50041.667
F-score28.73026.31227.11227.01626.692
MfeatACC55.80047.55045.45045.95044.950
NMI44.59032.19531.26331.66832.638
PUR59.80047.70048.05048.50048.150
F-score34.67032.73631.61331.43432.327
HandwrittenACC53.75040.70044.95046.15046.500
NMI58.05032.43033.83033.41433.408
PUR53.75043.70048.05048.95049.300
F-score33.47031.26432.14432.14432.139

Table R2: Ablation results (%) at 50% missingness.

DatasetMetricFullGlobalHardUniSampleUniGraph
Synthetic3dACC58.67053.16750.83355.66757.333
NMI45.35012.12215.62815.21114.732
PUR58.83053.16750.83359.66759.333
F-score50.96043.47843.07045.10444.701
ReutersACC38.17035.08333.91739.00034.250
NMI28.8807.3979.73510.0136.935
PUR49.50035.08333.91739.00034.250
F-score37.12021.96022.62123.37321.703
MfeatACC43.30032.30034.40035.60035.850
NMI41.79019.33419.20919.48419.751
PUR43.30032.95036.35035.75036.150
F-score26.51022.64421.79022.27322.266
HandwrittenACC48.45035.60032.35036.75036.750
NMI46.71019.42418.43418.92918.928
PUR48.45037.40033.90038.35038.350
F-score22.72022.06420.82321.83721.854

Increasing missingness from 30% to 50% reduces the available observations and cross-view overlap, leaving fewer observed constraints for correspondence estimation and representation learning. The ACC on Synthetic3d decreases from 71.83% to 58.67% under this change.

Q3. The 1/n1/n baseline and uniform assignment over retained candidates.

A: The baseline 1/n1/n is the maximum entry of a uniform row over all nn canonical columns. Thus, the current measure quantifies concentration relative to global uniform assignment. For a row uniform over si<ns_i<n retained candidates, its peak is 1/si1/s_i, and the measure is positive. It therefore reflects restriction of the candidate set as well as concentration within that set.

To distinguish these effects, we will compare the original measure with a support-normalized alternative. For an observed row ii, let si=nnz⁡(Pi,:(v))s_i=\operatorname{nnz}(P_{i,:}^{(v)}) denote its support size. We define

hi=max⁡jPij(v)−1/si1−1/si,si>1,h_i = \frac{\max_j P_{ij}^{(v)}-1/s_i}{1-1/s_i}, \qquad s_i>1,

with hi=1h_i=1 for singleton support. Missing rows with zero support are excluded. This alternative assigns zero to rows that are uniform over supports of size greater than one. We will report its effects on fusion weights and clustering performance and clarify the distinction between global and conditional concentration in Section 3.5 and Appendix B.6.

Reviewer_exeYOpenReview ↗
Rating
3(−1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The manuscript proposes CAMVC for incomplete and unaligned multi-view clustering without known cross-view sample pairs. It constructs canonical coordinates using an automatically selected reference view and learns sparse soft correspondences through cluster-routed local sparse transport (CLST). Correspondence-dependent reconstruction residuals provide sample-specific reliability, while observed-data reconstruction quality regulates graph fusion.

Critical strengths
  1. The framework considers both feature-dependent residual variation and higher-order dependencies across views. Covariance-aware residual modeling and high-order tensor regularization provide complementary ways to account for noise and cross-view structure.

  2. The appendix provides relatively comprehensive experiments covering multiple missing rates, with favorable results.

Critical weaknesses

Appendix B.1 does not clearly explain how independently obtained view clusters are matched to the reference clusters when initializing non-reference transports. The experimental protocol needs clearer reporting of parameter selection and result variability. Appendix A.1.5 provides search grids for λ1\lambda_1, λ2\lambda_2, and λ3\lambda_3, but does not specify the selection criterion. It also reports 20 k-means replicates without stating the number of independent runs of the complete procedure or how the reported results are aggregated.

Questions for authors

(1) How are the view clusters matched to the reference clusters without known cross-view sample pairs? Please explain the matching criterion.

(2) What criterion is used to select λ1\lambda_1, λ2\lambda_2, and λ3\lambda_3 from the grids in Appendix A.1.5? Please report the selected values and whether they vary across datasets and missing rates.

(3) Please provide the update rule and implementation settings for the clipped column-balancing steps.

2 comments
AuthorsOpenReview ↗

Response to exeY

We thank the reviewer for identifying these omissions. We clarify the matching criterion, parameter-selection protocol, and column-balancing procedure below.

Q1. How are independently obtained view clusters matched without known sample pairs?

A: We thank the reviewer for this question. Initialization matches each non-reference view to the selected reference view by comparing their cluster-relation graphs, without requiring known cross-view sample pairs.

For each view vv, we independently apply unsupervised k-means to its observed samples to obtain cc pseudo-clusters. Here, cc is the prescribed number of clusters, assumed known in our experiments. These pseudo-clusters are estimated from the input features; no ground-truth cluster memberships are used.

We then construct a c×cc\times c relation matrix GvG_v. Each entry Gv(a,b)G_v(a,b) is the rank of the squared distance between cluster centers aa and bb, centered and standardized within its row. These profiles describe relative inter-cluster relations and reduce dependence on absolute distance magnitudes. Their use assumes sufficiently consistent cluster relations across views.

The matching criterion is agreement between the source and reference relation matrices. Specifically, we seek a one-to-one mapping π\pi from source clusters to reference clusters by approximately solving

\sum_{a=1}^{c}\sum_{b=1}^{c} G_v(a,b)\\,G_1\bigl(\pi(a),\pi(b)\bigr),$$ where $\Pi_c$ is the set of cluster permutations and $G_1$ denotes the reference-view relation matrix. We use pairwise-swap search with 64 starts: one identity mapping and 63 random permutations. Each start performs at most 10 sweeps, accepting only strictly improving swaps; the best solution across starts is retained. Neither ground-truth class memberships nor ground-truth cross-view correspondences are used in this matching. The resulting mapping associates each source pseudo-cluster with a reference cluster and initializes the block transport within the matched clusters. Subsequent CLST updates refine the transport against the fixed reference guide, allowing the initial correspondence estimates to change. We will add this matching criterion, the initialization procedure, and the search settings to Appendix B.1. **Q2: Hyperparameter selection** **A:** We thank the reviewer for pointing out this omission. Appendix A.1.5 reports the search grids but does not specify the selection criterion . For each dataset and missing rate, we evaluate all 64 combinations of $\lambda_1,\lambda_2\in\\{0.01,0.03,0.1,0.3\\}$ and $\lambda_3\in\\{0.001,0.003,0.01,0.03\\}$. We select the combination achieving the highest ACC, with NMI used to break ties. This selection uses ground-truth evaluation metrics; the labels are not used to optimize the representations. Parameter selection is performed separately for each dataset and missing rate, rather than imposing a shared triple across datasets or fixing the triple selected at 10% missingness for the other rates. Consequently, the selected values may differ across these settings. **Q3. What is the clipped column-balancing update?** **A:** Let $n_v$ be the number of observed rows and $n_{\mathrm{valid}}$ the number of canonical columns with valid cluster assignments. After normalizing each observed row to one and setting missing rows to zero, we define $\tau=n_v/n_{\mathrm{valid}}$. Each balancing step computes $$c_j^{(t)}=\sum_{i\in O_v}P_{ij}^{(t)},\qquad s_j^{(t)}= \begin{cases} \operatorname{clip}\\!\left(\tau/c_j^{(t)},0.25,4\right), & c_j^{(t)}>10^{-14},\\\\ 1, & \text{otherwise}, \end{cases}$$ and updates $$P^{(t+1)} = \operatorname{RowNorm}\left(P^{(t)}\operatorname{diag}\left(s^{(t)}\right)\right).$$ Here, RowNorm normalizes observed rows and leaves missing rows zero. This procedure discourages column-mass concentration while preserving observed-row sums; it does not enforce exact column marginals or introduce support in uncovered columns. The main CLST update uses two balancing steps. Cluster-block initialization uses none, while structural initialization uses 10 capacity-based steps with capacity $1.05\tau$, rescaling only overloaded columns. After continuation mixing, each observed row is pruned to at most $r_c k$ entries and renormalized. We will add these equations and settings to Appendix B.1.

As the authors’ response has addressed my primary concerns, I recommend accepting the manuscript.

Reviewer_g5bGOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates multi-view clustering when observations are missing and sample correspondences across views are unknown. The proposed CAMVC framework establishes canonical coordinates by selecting a reference view according to explained scatter and cluster balance, and then estimates cross-view relationships through cluster-routed local sparse transport (CLST). It uses correspondence-dependent reconstruction residuals to assess sample reliability, while covariance-aware residual modeling and higher-order tensor regularization capture feature-dependent variations and cross-view dependencies. Comparisons and ablation studies on six benchmark datasets evaluate the method under incomplete and unaligned conditions.

Critical strengths

The paper addresses a meaningful and challenging multi-view clustering setting in which missing observations and unknown sample ordering occur simultaneously. Missingness and differences between views can make several correspondences plausible for a single observation. CAMVC attempts to learn shared representations without requiring known sample pairs or cross-view permutations, which makes the problem formulation practically relevant.

Critical weaknesses
  1. Section 3.1 selects the reference view using explained scatter and cluster balance, and constructs the fixed guide Z0Z_0 from its truncated SVD. However, the experiments do not compare different reference-view choices, so the effect of this selection strategy on clustering performance remains unclear.
  2. Section 4.5 discusses the class-level pattern in Figure 5, but does not clearly explain how the matched class-level transport is calculated from the learned sparse soft correspondences. This makes the visualization difficult to interpret.
  3. The related-work discussion does not sufficiently distinguish CAMVC from recent methods addressing uncertain cross-view correspondences, making its methodological differences and novelty less clear.
Questions for authors
  1. How does the choice of reference view affect CAMVC? Could the authors compare the automatically selected reference view with alternatives on one or two representative datasets and report ACC and NMI?
  2. How is the matched class-level transport in Figure 5 computed from the learned sparse soft correspondences? Please clarify how the correspondence weights are aggregated at the class level and what “matched” means.
  3. Could the authors briefly discuss the differences between CAMVC and [A] and [B]? If their experimental settings are comparable, could the authors also include these methods in the comparison?

[A] Zhou et al. Uncover Underlying Correspondence for Robust Multi-view Clustering.

[B] Huang et al. Robust Multi-view Clustering against Imperfect Information.

3 comments
AuthorsOpenReview ↗

Response to g5bG

We thank the reviewer for the constructive comments. Our responses to the three points are provided below.

Q1 Effect of reference view selection:

A: We agree that the manuscript did not analyze how the choice of reference view affects the experimental results.We have added a controlled comparison on four datasets: Synthetic3d, Reuters, Mfeat, and Handwritten.

DatasetReference viewACCNMI
Synthetic3dAuto (view 3)0.76330.4813
Synthetic3dView 10.73000.4060
Synthetic3dView 20.63170.2903
ReutersAuto (view 3)0.49670.4351
ReutersView 10.48330.2318
ReutersView 20.44170.2233
ReutersView 40.43420.2081
ReutersView 50.45830.2080
MfeatAuto (view 4)0.66950.6032
MfeatView 10.54050.4986
MfeatView 20.46250.3514
MfeatView 30.57750.4895
MfeatView 50.60400.5303
MfeatView 60.43050.3431
HandwrittenAuto (view 6)0.65700.5814
HandwrittenView 10.52900.5112
HandwrittenView 20.48500.3783
HandwrittenView 30.57100.4962
HandwrittenView 40.44050.3383
HandwrittenView 50.56500.4745

Q2:Matched class-level transport in Figure 5

A: The original description was insufficient, and we supplement it as follows. The term matched denotes a class-level one-to-one permutation, not sample-level hard assignment. Let P(v)P^{(v)} be the learned transport of the displayed non-reference view vv. For a local class aa and a canonical class bb, define

Mab=1∣Ia∣∑i∈Ia∑j∈CbPij(v),M_{ab}=\frac{1}{|I_a|}\sum_{i\in I_a}\sum_{j\in C_b}P^{(v)}_{ij},

where IaI_a is the set of observed local rows labelled aa, and CbC_b is the set of canonical columns labelled bb. Thus MabM_{ab} is the average transport mass that local-class-aa samples assign to canonical class bb. Since every observed row satisfies ∑jPij(v)=1\sum_j P^{(v)}_{ij}=1, each row of MM sums to one.

Since an unsupervised alignment is identifiable only up to a class permutation, the columns of MM are matched using the Hungarian algorithm:

π=arg⁡max⁡π∑aMa,π(a).\pi=\arg\max_{\pi}\sum_a M_{a,\pi(a)}.

The displayed matrix is [Ma,π(b)][M_{a,\pi(b)}]. The diagonal mass reported in the text is ∑awaMa,π(a)\sum_a w_a M_{a,\pi(a)}, where wa=∣Ia∣/∑a′∣Ia′∣w_a=|I_a|/\sum_{a'}|I_{a'}|; the chance level is ∑awaπb\sum_a w_a \pi_b, where πb\pi_b is the prior proportion of canonical class bb; and the class gain is their normalized difference,

Gain=∑awaMa,π(a)−∑awaπb1−∑awaπb.\mathrm{Gain}=\frac{\sum_a w_a M_{a,\pi(a)}-\sum_a w_a \pi_b}{1-\sum_a w_a \pi_b}.

This definition has been added to Section 4.5, the caption of Figure 5 has been revised, and Appendix B.7, Class-level transport readout, now provides the formula, the matching procedure, and the quality gate.

AuthorsOpenReview ↗

Response to g5bG

Q3:Comparison with CorreGen and PLCI

A: The main difference lies in the input assumption. CorreGen [A] assumes that cross-view pairs exist but may be noisy, and does not model missing views. PLCI [B] handles missing views and noisy pairs jointly, but still requires a set of observed pairs. In contrast, CAMVC assumes neither known pairs nor a shared sample order, in addition to missing observations.

This difference leads to different correspondence mechanisms. CorreGen[A] estimates a dense soft posterior using GMM-guided marginals and a virtual sample; PLCI[B] performs prototype-level transport to infer a latent counterpart. CAMVC builds a canonical coordinate from a label-free reference view and applies cluster-routed sparse transport. The reliability signals also differ: GMM marginals or memory-effect reliability in the two baselines, versus reconstruction-residual weights with a covariance model in CAMVC.

CorreGen[A] and PLCI[B] develop their core correspondence mechanisms around two views and report experimental results exclusively under two-view configurations. Their released implementations likewise adopt a two-view setup, requiring additional code adaptation for experiments with more than two views. Although their problem definitions allow multiple views, their reported experiments do not establish effectiveness beyond two views. To compare these methods with CAMVC, we extended both released implementations to handle multiple views and evaluated them on the same datasets under the missingness and unalignment protocols specified in our paper.

We have added both methods to the comparison under the same missing masks and permutations as the other baselines. Table R2 reports ACC and NMI at 10% and 30% missingness. Implementation and adaptation details are reported in Appendix A.1.3.

Table R2: ACC and NMI (%) at 10% and 30% missingness.

DatasetRateACC-CAMVCACC-CorreGenACC-PLCINMI-CAMVCNMI-CorreGenNMI-PLCI
Hdigit10%52.2640.3220.9345.0835.828.83
30%46.9141.0118.8339.3633.225.73
Cifar1010%76.0758.3914.2269.4352.613.88
30%63.5452.3815.0955.9547.783.91
Reuters10%49.6728.1022.5043.515.682.28
30%44.2328.2918.8339.375.960.66
Mfeat10%66.9566.9221.8560.3266.1112.82
30%55.8071.5019.8544.5966.839.39
Handwritten10%65.7057.5415.0058.1457.713.49
30%53.7578.3619.2558.0573.976.58
Synthetic3d10%76.3370.5036.8348.1345.701.21
30%71.8376.5450.6723.8749.6811.06

[A] Zhou et al. Uncover Underlying Correspondence for Robust Multi-view Clustering.

[B] Huang et al. Robust Multi-view Clustering against Imperfect Information.

The authors' response addressed my main concerns, and I decided to keep my original scores.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.