acceptodds
Under review as a conference paper at ICLR 2027

Confidence-Weighted Dual-Expert Correspondence Refinement for Incomplete Multi-View Clustering

Abstract

Incomplete multi-view clustering (IMVC) aims to partition all samples effec-tively when each sample is observed in only a subset of views. However, mostexisting IMVC methods commonly treat the observed cross-view correspon-dences as equally reliable supervision, while overlooking the varying semanticand structural consistency of these correspondences under incomplete and het-erogeneous observations. To address this limitation, we propose an anchoreddual-expert framework that progressively learns confidence-weighted cross-viewcorrespondences from complementary first- and second-order structural informa-tion. Specifically, we first exploit the observed partial overlap to initialize softcross-view correspondences and select an informative anchor view, providing aninitial basis for correspondence refinement. Furthermore, a dual-expert matchingmechanism is developed, which combines a Koopmans–Beckmann QAP expertfor modeling first-order node and local relational affinities with an anchor-basedLawler QAP expert for capturing higher-order structural consistency in a compactlearned anchor space. The latter incorporates an outlier state to accommodate un-matched structures caused by incomplete observations. Finally, we introduce amomentum-based consistency refinement mechanism to fuse the complementaryevidence from the two experts and iteratively update correspondence confidences,which in turn provide progressively refined supervision for multi-view representa-tion learning. The refined cross-view and within-view affinities are subsequentlyintegrated to construct a global affinity matrix for spectral clustering. Extensiveexperiments under varying view missing rates demonstrate the effectiveness androbustness of the proposed framework.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Incomplete Multi-View Clustering

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_UL2oOpenReview ↗
Rating
2(−2)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

An anchored dual-expert framework is developed for incomplete multi-view clustering to improve cross-view correspondence estimation by leveraging complementary first- and second-order structural information. The framework integrates two QAP-based experts and employs momentum-based refinement to iteratively update correspondence confidencas. Experiments under varying view missing rates demonstrate the method’s effectiveness.

Critical strengths
  1. Integrating graph matching into multi-view representation learning is a reasonable approach to exploiting structural information across incomplete views.

  2. Combining cross-view and intra-view affinities is beneficial for preserving complementary structural information in the final clustering graph.

Critical weaknesses
  1. The interaction between correspondence optimization and representation learning remains unclear.

  2. The integration of cross-view and intra-view affinities lacks sufficient justification.

Questions for authors
  1. How do correspondence updates affect representation learning during training?

  2. How are cross-view and intra-view affinities balanced, and how sensitive is clustering performance to their relative weights?

  3. How stable are the final clustering results across random initializations and perturbations of the affinity matrix?

7 comments
~Louis_Dubois1OpenReview ↗

A Perfect-Matching Counterexample to Eq. (8): Major Soundness Concerns

The review's 4/4 Clear Acceptance cites dual-expert matching and cross-/within-view affinity fusion as strengths. Both require stronger evidence. In particular, the fusion rule in Eq. (8) can worsen clustering even when the decoded instance matches are perfect.

1. A perfect-matching counterexample to Eq. (8).

Section 3.1 assumes known global entity IDs. If an expert predicts perfect instance matches, scattering those matches into global coordinates produces only diagonal entries. Thus, for distinct entities p≠qp\ne q, the expert contributes zero to the off-diagonal affinity. Nevertheless, Eq. (8) adds a cross-view co-observation count to the denominator. Writing WpqW_{pq} for the summed within-view affinity, cpqc_{pq} for its observation count, and dpqd_{pq} for this extra count, Eq. (8) yields

Apq=Wpqcpq+dpqrather thanWpqcpq.A_{pq}=\frac{W_{pq}}{c_{pq}+d_{pq}}\quad\text{rather than}\quad\frac{W_{pq}}{c_{pq}}.

For two entities jointly observed in the anchor and one target view, their within-view edge is reduced from Wpq/2W_{pq}/2 to Wpq/3W_{pq}/3, even though neither expert adds an off-diagonal edge. This changes relative edge weights according to missingness patterns.

I checked a reproducible 16-entity, three-view example using identical observed embeddings, local-scale KNN graphs, a fixed common graph, and normalized spectral clustering. Two surrogate matching branches (KB-style/Hungarian and Gaussian-anchor/Sinkhorn/Hungarian) both decode the true instance matches with 100% accuracy. Fixing the expert outputs, within-view graphs, common graph, and clustering procedure, removing only the cross-view denominator count yields 100% clustering ACC; Eq. (8) yields 50%. The same failure occurs in a 64-entity version, and persists with self-loops removed.

This is a counterexample to the written fusion operator, not a reproduction of the authors' trained network. If the fused output is instead a dense semantic affinity, the authors must distinguish it mathematically from instance correspondence and state precisely which output and support were used to generate Table 2.

2. Pseudo-target supervision can oppose true-ID supervision.

In Eq. (3), YijY_{ij} is a BCE target, not merely a weight on a known true match. For p=σ(z)p=\sigma(z), the soft-target derivative is p−yp-y, whereas weighting a true match (T=1T=1) by yy gives y(p−1)y(p-1). At p=0.8p=0.8, y=0.3y=0.3, these equal +0.50 and -0.06: gradient descent acts in opposite directions.

Even retaining true positives in InfoNCE does not remove this possibility. For a two-instance reduced objective,

J(s)=log⁡(1+e−2s/0.8)+log⁡(1+e5s),J(s)=\log(1+e^{-2s/0.8})+\log(1+e^{5s}),

the unique optimum is s≃−0.167847s\simeq-0.167847: the false match has higher similarity. This establishes a failure mode of the stated loss components, not guaranteed failure of the full model. Additional regularizers would need to counteract it. Moreover, Eq. (2) changes directly with positive-set membership, not with continuous confidence values when that set is fixed.

3. Current ablations do not identify the claimed source of improvement.

On HW at missing rate 0.3 (Table 3), removing L-QAP costs 0.32 ACC points and removing KB-QAP costs 1.20, whereas removing commonZ costs 16.15. At rate 0.1, removing L-QAP improves NMI from 87.56% to 88.81%. These results cannot establish that dual-expert confidence refinement drives the gains.

A decisive test would fix embeddings, missing masks, and spectral clustering, then compare (i) within-view fusion, (ii) Eq. (8) with true-ID matches, and (iii) Eq. (8) with learned matches, each with and without the extra denominator count. Training controls should compare true-ID supervision, frozen initial YY, and momentum-refined YY, with correspondence error and semantic-edge quality reported directly.

4. Novelty also needs isolation.

The cited 2026 TPAMI predecessor already uses KB-QAP, Lawler QAP, and align-fuse-refine cooperation; earlier IMVC work includes confidence-guided consensus graphs and anchor alignment. This paper must demonstrate what its adaptation adds beyond these components.

Conclusion. Strong average ACC is not evidence that correspondence refinement is correct or necessary. The perfect-matching fusion counterexample, the pseudo-target optimization conflict, and missing causal ablations materially weaken the review's stated strengths. I ask the reviewer to reconsider whether Clear Acceptance is justified until the authors specify the actual Table 2 fusion operator and demonstrate an independent benefit from learned correspondences.

AuthorsOpenReview ↗

Response to the Reviewer UL2o: Correspondence Updates, Affinity Balance and Stability

Response to the Reviewer UL2o

We thank the reviewer for these constructive questions and will incorporate the clarifications below into the revision.

Q1: How do correspondence updates affect representation learning during training?

Correspondence updates act as an online supervision signal for representation learning.

  • Current correspondences guide the encoder. At each epoch, the current soft matrix Y(v)Y^{(v)} determines the positive anchor-target pairs used in the overlap-aware contrastive loss. Pairs selected by the current pseudo-correspondence are pulled together in feature space, while other observed candidates contribute negative evidence through the InfoNCE denominator.
  • Representations guide matching. The resulting features F(v)F^{(v)} are passed to the KB-QAP and L-QAP experts, which produce first-order and second-order match scores.
  • Correspondences are refined. The expert outputs are compared with the historical correspondence matrix. Pairs supported by both experts and the history are reinforced; partially supported pairs receive intermediate confidence; pairs for which both experts agree but the history disagrees are updated using the averaged expert scores; and unsupported entries decay.
  • The refined matrix controls the next epoch. A momentum update produces Ynew(v)Y^{(v)}_{\mathrm{new}}, which is used in the next epoch as the soft target for both matching losses and the positive-pair target for contrastive representation learning.

The correspondence update itself does not directly update encoder parameters; instead, it changes the supervision used by the next training iteration. During that iteration, the contrastive and expert losses jointly update the encoder. This creates the feedback loop

Yt→Ft→QAP matching→Yt+1→Ft+1Y_t \rightarrow F_t \rightarrow \text{QAP matching} \rightarrow Y_{t+1} \rightarrow F_{t+1}

The intended effect is to progressively refine cross-view alignment: improved representations provide more informative matching evidence, while refined correspondences provide more selective supervision for representation learning. Momentum, decay, and confidence-based refinement limit abrupt changes and prevent every individual expert prediction from immediately redefining the training targets.

Q2: How are cross-view and intra-view affinities balanced, and how sensitive is clustering performance to their relative weights?

Within-view and cross-view affinities are combined through an availability-normalized aggregation rather than an explicitly tunable global weight. For each pair (p,q)(p,q), the within-view KNN affinities and cross-view fused similarities are first scattered into global matrices. Their sum is then divided by the number of available contributions for that pair (Eq. 8). Thus, each observed contribution receives equal weight.

So, the current paper does not report a sensitivity study for the relative weight between within-view and cross-view evidence. The existing parameter-sensitivity experiment varies wKBw_{\mathrm{KB}}, MM, kskelk_{\mathrm{skel}}, and δ\delta, but not these affinity-level weights. Therefore, we cannot claim from the current experiments that clustering performance is insensitive to the intra-/cross-view balance. To address this point, we will add an ablation that explicitly defines

Aλ=Aintra+(1+λ)Across,λ∈{0.25,0.5,0.75,1}A_{\lambda}=A_{\mathrm{intra}}+(1+\lambda)A_{\mathrm{cross}},\qquad \lambda\in\lbrace 0.25, 0.5, 0.75, 1\rbrace

while fixing the other components, and report the mean ACC over multiple random seeds.

Settingorigλ = 0.25λ = 0.50λ = 0.75λ = 1.00
100leaves, r = 0.10.94230.95130.91880.86380.7544
100leaves, r = 0.30.83310.82250.79060.69500.4244
100leaves, r = 0.50.62540.62250.63060.61250.2794
100leaves, r = 0.70.43810.42130.41750.40630.3088
HW, r = 0.1 (0.50)0.95150.94900.94550.93950.8945
HW, r = 0.3 (0.32)0.93400.93320.94250.93650.8455
HW, r = 0.5 (0.23)0.90200.90450.90150.89900.7812
HW, r = 0.7 (0.20)0.86400.84780.84030.74350.4220

The experimental results show that clustering performance generally declines as the cross-view weight increases. In particular, when its weight reaches twice that of the intra-view affinity, the performance degradation becomes pronounced. Nevertheless, performance remains relatively insensitive to the relative weights over most of the intermediate range and deteriorates substantially only when cross-view evidence becomes dominant.

Q3: How stable are the final clustering results across random initializations and perturbations of the affinity matrix?

A single response field is capped at 5,000 characters, so Q3 is answered in the follow-up comment directly below this response, covering both random initialization and perturbation of the affinity matrix. We apologize for the inconvenience.

AuthorsOpenReview ↗

Q3: Follow-up Comment for Stability Results

Q3: How stable are the final clustering results across random initializations and perturbations of the affinity matrix?

We address this question with two sets of experiments: random initialization, and affinity-matrix perturbation of two kinds, random edge deletion and bounded multiplicative Gaussian noise.

Random initialization

We fix the random seeds before the final affinity matrix is built. After entering the clustering stage, we set the K-means seed to 0,…,190,\dots,19, run the clustering 20 times, and report the mean and standard deviation of ACC.

DatasetrACC (mean ± std)
100leaves0.10.9303 ± 0.0098
100leaves0.30.8176 ± 0.0142
100leaves0.50.6183 ± 0.0071
100leaves0.70.4272 ± 0.0060
HW0.10.9208 ± 0.0295
HW0.30.9314 ± 0.0109
HW0.50.8988 ± 0.0115
HW0.70.8458 ± 0.0003
Scene150.10.4132 ± 0.0109
Scene150.30.4038 ± 0.0150
Scene150.50.3533 ± 0.0124
Scene150.70.3158 ± 0.0066

Perturbations of the affinity matrix

We test the robustness of the affinity matrix for the three datasets at a missing rate of 0.3.

A. Random edge deletion. Let NN be the fraction of deleted affinity edges. Stability is measured by the difference between the perturbed ACC and the baseline ACC.

DatasetN = 0.1N = 0.2N = 0.3
100leaves−1.21−4.15−9.37
HW−1.08−2.21−3.08
Scene15−0.47+0.14+0.24

B. Bounded multiplicative Gaussian noise. We fix the final matrix A0A_0 and the spectral-clustering initialization and perturb only A0A_0 with a bounded, symmetric multiplicative perturbation. Each strength uses 20 independent perturbations; non-negativity, symmetry, the diagonal and the zero edges are preserved. We report mean and standard deviation.

A(σ)=max⁡(0,A0⊙(1+σϵ)),ϵ=ϵ⊤,ϵij∼U(−1,1) for i<j,ϵii=0A^{(\sigma)}=\max\left(0, A_0 \odot (1+\sigma\epsilon)\right),\quad \epsilon=\epsilon^{\top},\quad \epsilon_{ij}\sim\mathcal{U}(-1,1)\text{ for }i<j,\quad \epsilon_{ii}=0

100leaves, r = 0.3. Baseline: ACC 0.8331 / NMI 0.8958 / ARI 0.7291. Each entry is mean ± std over 20 independent perturbations.

σACCNMIARI
0.050.8091 ± 0.01300.8834 ± 0.00370.6995 ± 0.0100
0.100.8037 ± 0.01390.8840 ± 0.00370.6917 ± 0.0119
0.200.8033 ± 0.01310.8844 ± 0.00380.6916 ± 0.0118
0.300.7966 ± 0.01140.8822 ± 0.00440.6853 ± 0.0121
0.500.7917 ± 0.01270.8718 ± 0.00370.6870 ± 0.0111

HW, r = 0.3. Baseline: ACC 0.9340 / NMI 0.8694 / ARI 0.8621. Each entry is mean ± std over 20 independent perturbations.

σACCNMIARI
0.050.9337 ± 0.00040.8691 ± 0.00080.8682 ± 0.0009
0.100.9270 ± 0.02450.8664 ± 0.01020.8725 ± 0.0231
0.200.9321 ± 0.00060.8581 ± 0.00100.8669 ± 0.0012
0.300.9307 ± 0.03340.8532 ± 0.01160.8659 ± 0.0299
0.500.9199 ± 0.03570.8533 ± 0.01190.8580 ± 0.0305

Scene15, r = 0.3. Baseline: ACC 0.4198 / NMI 0.3875 / ARI 0.2357. Each entry is mean ± std over 20 independent perturbations.

σACCNMIARI
0.050.4105 ± 0.01430.3662 ± 0.00330.2213 ± 0.0050
0.100.4062 ± 0.01500.3658 ± 0.00270.2205 ± 0.0034
0.200.4057 ± 0.01330.3643 ± 0.00400.2195 ± 0.0055
0.300.4012 ± 0.01440.3605 ± 0.00380.2174 ± 0.0062
0.500.3918 ± 0.01320.3454 ± 0.00340.2119 ± 0.0068

We test stability in three ways. With 20 K-means restarts on a fixed affinity matrix, the standard deviation of ACC is at most 0.0295. Under random edge deletion, the loss is at most 1.21 points at N = 0.1, reaches 9.37 points on 100leaves at N = 0.3 but only 3.08 points on HW, and Scene15 is essentially unaffected (+0.24). Under bounded multiplicative noise at r = 0.3, ACC decreases gradually, by at most 4.14 points relative to baseline at the largest strength. Results are therefore stable to initialization and to noise. Sensitivity to edge deletion is dataset-dependent but bounded at mild levels, and a pronounced drop appears only under heavier deletion. We do not claim invariance.

Reviewer_MwZwOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses correspondence reliability in incomplete multi-view clustering by jointly modeling local relationships and high-order structural consistency, with outlier handling for unmatched samples. A momentum-based dual-expert mechanism iteratively refines correspondence confidence to guide representation learning, demonstrating effectiveness and robustness across different view missing rates.

Critical strengths
  1. The paper highlights the varying reliability of cross-view correspondences and their implications for incomplete multi-view clustering.
  2. The two experts combine local affinities with second-order structural consistency to refine cross-view correspondences.
Critical weaknesses
  1. The method distinguishes several correspondence cases based on agreement between the two experts and historical pseudo-labels. However, it remains unclear how reliably these cases reflect the actual quality of correspondences.
  2. The update rules for consistent, partially consistent, and conflicting correspondences are largely heuristic, leaving unclear under what conditions the mechanism can correct initial errors.
  3. Sections 4.3, 4.4, and 4.5 report results on only a subset of the datasets, making the experimental evaluation incomplete and limiting the assessment of the method's overall effectiveness.
Questions for authors
  1. Do the authors have empirical evidence that correspondence pairs receiving higher confidence are more likely to be correct?

  2. Why are the experimental results in Sections 4.3, 4.4, and 4.5 reported only on a subset of the datasets? Can the results on the remaining datasets also be provided?

  3. Why are some methods in Table 2 not shown in results for some datasets?

  4. The rationale for selecting the anchor view is not clear.

4 comments
AuthorsOpenReview ↗

Response to Reviewer MwZw: Confidence–Correctness Evidence, Ablation Coverage, Missing Baselines, and Anchor Selection

Response to the Reviewer MwZw

We thank the reviewer for these constructive questions and will incorporate the clarifications below into the revision.

Q1: Do the authors have empirical evidence that correspondence pairs receiving higher confidence are more likely to be correct?

To answer this directly we ran post-review experiments. The table reports precision per confidence interval at epoch 110 over 3 datasets × 4 missing rates (12 settings): 6,899,116 candidate pairs, 12,685 true (prior 0.18%). YY is dumped for each target view after the final epoch; candidate pairs are those retained by the Stage-1 mutual top-kk selection, and a pair is true when its target column holds the global sample index of the anchor sample (identity level, no class labels). Pairs are binned by confidence value and aggregated over 36 target views.

Confidence interval#Candidate pairs#True pairsPrecision
[0, 0.001)585,69800.00%
[0.001, 0.01)2,328,78600.00%
[0.01, 0.05)3,700,95700.00%
[0.05, 0.1)270,99000.00%
[0.1, 0.2)463463100.00%
[0.2, 0.3)4,2024,202100.00%
[0.3, 0.5)6,7426,742100.00%
[0.5, 1.0]1,2781,278100.00%
Total6,899,11612,6850.18%

Precision is 0.00% in all four intervals below 0.1, covering 6,886,431 pairs (99.82% of the block) with no true pair, and 100.00% at and above 0.1, where true pairs equal all pairs in each interval. The relation is therefore monotonically increasing in confidence. At epoch 0 all candidate scores saturate in [0.5,1][0.5,1] by construction, so the same binning yields one interval with precision 0.18%. The discriminative behaviour is thus acquired during training, not present at initialization.

Q2: Why are the experimental results in Sections 4.3, 4.4, and 4.5 reported only on a subset of the datasets?

Representative subsets kept these diagnostic sections concise and computationally manageable: 4.3 used 100leaves and HW for two contrasting regimes, 4.4 used HW as an optimization-stability diagnostic, and 4.5 used three datasets with different view structures because the anchor-selection ablation is expensive. We have now completed the remaining datasets under the same protocol and give the full results in the appendix. The additional ablations show dataset-dependent effect sizes rather than a uniform improvement, training losses decrease and stabilize without divergence on all evaluated datasets, and the score-selected anchor ranks best everywhere, though its margin is narrow on some datasets. To respect the length limit we keep only these aggregate conclusions here and move all tables and curves to the supplementary material.

A single response field is capped at 5,000 characters, so the complete Q2 tables do not fit in this reply. We therefore post them as a follow-up comment directly under this response, and the same tables appear in the supplementary appendix.

Q3: Why are some methods in Table 2 not shown in results for some datasets?

We thank the reviewer for identifying the missing results on Hdigit and aloi-100; the two methods failed for different reasons. DGIMVC runs terminated before convergence because of a CUDA out-of-memory error in the graph contrastive loss, which constructs a large pairwise similarity matrix, so no valid final results were obtained. DSIMVC initially failed through an error in the data-preprocessing procedure; we corrected it, reran under the same protocol, and its complete results are now in Table 2. We will revise the table with the corrected DSIMVC results and mark the DGIMVC limitation clearly.

Q4: The rationale for selecting the anchor view is not clear.

We agree that this motivation should be stated explicitly. The anchor view is the common reference for initializing all anchor–target soft correspondences, and we select it for three complementary reasons. Completeness: more observed samples give more co-observed pairs and thus more reliable bootstrap supervision. Feature informativeness: the reference view must carry discriminative information for meaningful cross-view similarities. Spectral gap of the observed-sample kk-NN graph: a larger gap indicates better separated neighbourhoods and hence more stable pseudo-correspondences. These criteria are min–max normalized and combined as

sv=αcompv+βvarv+ζgapv,α+β+ζ=1,s_v=\alpha\mathrm{comp}_v+\beta\mathrm{var}_v+\zeta\mathrm{gap}_v,\qquad \alpha+\beta+\zeta=1,

with (α,β,ζ)=(0.6,0.2,0.2)(\alpha,\beta,\zeta)=(0.6,0.2,0.2). Completeness carries the largest weight because missingness is the primary source of uncertainty when initializing cross-view correspondences. A single anchor also gives one consistent reference for all target views and avoids independent alignment per view pair.

AuthorsOpenReview ↗

Follow-up to Q2: Full Ablation, Anchor-Selection and Convergence Results

Complete Q2 ablation results . COIL20

Methodr = 0.1r = 0.3r = 0.5r = 0.7
Full (Ours)80.21 / 87.04 / 75.2176.63 / 83.38 / 69.1576.76 / 82.77 / 70.8670.11 / 80.29 / 63.52
w/o KB-QAP80.10 / 86.48 / 75.0376.45 / 83.48 / 70.8676.43 / 81.32 / 70.4567.57 / 76.17 / 59.06
w/o L-QAP70.90 / 75.30 / 56.9875.76 / 82.38 / 67.2874.17 / 78.75 / 65.3866.53 / 75.16 / 57.36
w/o commonZ67.92 / 79.72 / 59.1178.96 / 85.09 / 72.9674.65 / 81.35 / 67.1472.15 / 79.97 / 64.95

scene15

Methodr = 0.1r = 0.3r = 0.5r = 0.7
Full (Ours)41.76 / 39.98 / 24.2041.98 / 38.75 / 23.5736.34 / 31.48 / 18.5432.57 / 27.45 / 14.59
w/o KB-QAP40.13 / 39.14 / 23.0938.93 / 35.61 / 21.1335.03 / 30.35 / 16.5432.13 / 27.61 / 13.79
w/o L-QAP41.74 / 39.29 / 23.9040.22 / 36.97 / 22.3736.10 / 30.97 / 17.0430.86 / 28.06 / 13.96
w/o commonZ38.19 / 38.90 / 23.5441.69 / 37.20 / 22.9935.96 / 31.07 / 17.2231.75 / 28.13 / 14.14

Hdigit

Methodr = 0.1r = 0.3r = 0.5r = 0.7
Full (Ours)95.44 / 90.05 / 91.2893.35 / 85.28 / 85.8088.97 / 77.35 / 77.3775.92 / 70.04 / 64.31
w/o KB-QAP91.70 / 83.55 / 82.9280.95 / 77.67 / 72.8376.56 / 71.53 / 65.7564.47 / 61.89 / 53.22
w/o L-QAP78.01 / 77.37 / 70.7492.89 / 83.75 / 84.9489.25 / 77.73 / 77.8668.03 / 63.61 / 56.41
w/o commonZ94.58 / 89.36 / 90.4793.06 / 85.25 / 86.1387.16 / 77.06 / 77.5875.69 / 66.88 / 62.07

ALOI-100

Methodr = 0.1r = 0.3r = 0.5r = 0.7
Full (Ours)92.24 / 92.16 / 85.4087.19 / 87.76 / 77.5682.27 / 82.33 / 67.5961.92 / 70.61 / 44.97
w/o KB-QAP90.06 / 91.77 / 84.1785.10 / 86.46 / 73.8481.05 / 82.44 / 67.3459.25 / 69.90 / 44.23
w/o L-QAP88.99 / 90.56 / 80.7186.64 / 87.20 / 75.3677.70 / 81.10 / 62.8960.44 / 70.51 / 44.34
w/o commonZ89.74 / 89.62 / 80.4584.18 / 85.54 / 72.9180.72 / 82.47 / 67.3459.94 / 70.38 / 43.99

Wiki_fea

Methodr = 0.1r = 0.3r = 0.5r = 0.7
Full (Ours)52.20 / 45.38 / 35.7547.35 / 38.42 / 25.4439.67 / 33.17 / 14.8437.51 / 29.74 / 11.72
w/o KB-QAP44.70 / 37.32 / 21.9343.68 / 35.61 / 20.9439.85 / 31.26 / 15.1635.66 / 30.00 / 9.53
w/o L-QAP51.05 / 45.29 / 36.1943.82 / 34.18 / 23.4839.53 / 30.06 / 15.3735.07 / 29.22 / 9.34
w/o commonZ48.67 / 38.39 / 28.9640.96 / 32.16 / 21.3739.57 / 28.74 / 17.5336.04 / 28.84 / 10.24

Together with the 100leaves and HW results in Section 4.3, these five tables cover all seven benchmark datasets under the same protocol. Averaged over 4 missing rates × 3 metrics, the full model is best on every dataset. The gain varies by component and dataset, and a few individual cells favour a variant.

Anchor-selection ablation (ACC at r = 0.1). Each candidate view is fixed as the anchor while the rest of the pipeline is unchanged.

Dataset#ViewsView 0View 1View 2View 3Selected
100leaves393.8794.3294.08—View 1
Wiki_fea252.2051.05——View 0
Hdigit293.6995.44——View 1
ALOI-100490.4890.6792.2492.19View 2

In all four datasets the view chosen by the learned score is also the best fixed anchor. The margin over the second-best view is narrow in places, 0.24 points on 100leaves and 0.05 on ALOI-100, so we do not claim that the score recovers a unique optimum; what the table shows is that it never selects a clearly inferior view, whereas the spread across views reaches 1.75 points on Hdigit and 1.76 on ALOI-100.

Training-loss convergence.

Datasetrep 1ep 20ep 40ep 60ep 80ep 100final
ALOI-1000.131.45614.13743.80563.56473.40793.37333.3506
ALOI-1000.343.37246.15924.00813.72443.64503.35253.5458
ALOI-1000.532.59974.48264.21564.05103.96333.93503.8890
ALOI-1000.735.00255.74605.11534.88944.75724.71924.6959
Hdigit0.153.02553.60573.07392.99862.97262.97032.9692
Hdigit0.333.56454.10113.61713.52663.49823.49283.4925
Hdigit0.528.56674.92004.62524.44654.43304.33474.3316
Hdigit0.732.13865.72055.33485.03144.85884.82734.8191

Across the two datasets and four missing rates, the loss falls 5.6–15× by epoch 20 and stays within 6% from epoch 60 onward, with no divergence. Owing to the character limit, only these two datasets are shown here; full results are in the appendix.

Thank you for the response. My concerns have been adequately addressed from my perspective, and I therefore maintain my original rating.

Thanks for the response. I have read the comments from the other reviewers and the newly added experimental results. The authors have addressed all of my concerns, and I will maintain my positive score.

Reviewer_LrsFOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a confidence-weighted dual-expert framework for incomplete multi-view clustering. The method first initializes the soft correspondence across views by using the anchor view with higher information content and the bidirectional Top-K matching; then, it optimizes these correspondences through two complementary expert modules: KB-QAP is used to model first-order structural information, and the Lawler QAP based on the anchor points is used to capture high-order structural consistency. Subsequently, the prediction results of the two experts are fused using a momentum-based optimization mechanism, and the confidence of the correspondences is updated. Finally, these confidence levels are used to characterize learning and the construction of the global affinity matrix.

Critical strengths

The method explicitly accounts for the varying reliability of observed cross-view correspondences. This is a meaningful improvement over approaches that treat all observed pairs equally, especially when the views are highly incomplete or heterogeneous.

Critical weaknesses

The ablation study evaluates the removal of individual experts, but a more explicit comparison of the two experts under different missing rates would help explain when first-order or higher-order structural information is most beneficial.

Questions for authors
  1. As the missing rate increases, how will the relative contributions of KB-QAP and Lawler QAP change? Can the author briefly discuss this?
  2. What is the core mechanism of extending graph matching to multi-view clustering in this paper?
  3. Can the author explain formula (8) more clearly? What are the specific meanings of each term, their mathematical functions, and their roles in the entire method?
2 comments
AuthorsOpenReview ↗

Response to Reviewer LrsF: Dual-Expert Contributions, QAP-to-MVC Mechanism, and the Eq. (8) Aggregation

Response to the Reviewer LrsF

We thank the reviewer for these constructive questions and will incorporate the clarifications below into the revision.

Q1: How do the relative contributions of KB-QAP and Lawler QAP change with missingness?

Let MM denote the mean of ACC, NMI, and ARI. We measure the relative expert contribution by D(r)=M(KB-only,r)−M(L-only,r)D(r)=M(\text{KB-only},r)-M(\text{L-only},r); the no-expert baseline cancels in this difference. Positive values favor KB-QAP, and negative values favor L-QAP.

Datasetr=0.1r=0.3r=0.5r=0.7
HW+0.93+1.56+0.68+0.06
100leaves−6.60+10.89+6.17−3.10
ALOI-100−1.91+1.27−3.05+0.64
Hdigit−10.68+10.04+10.27+2.82

Three observations follow. First, at r=0.1r=0.1, L-QAP is stronger on 100leaves and ALOI-100, while the two experts are nearly tied on HW. Second, at r=0.3r=0.3, KB-QAP is stronger on all four datasets. Third, at r=0.5r=0.5 and r=0.7r=0.7 the ordering becomes dataset-dependent: at r=0.5r=0.5 KB-QAP leads on HW, 100leaves, and Hdigit, whereas L-QAP leads on ALOI-100; at r=0.7r=0.7 100leaves favors L-QAP, HW remains tied, and ALOI-100 and Hdigit show a slight edge for KB-QAP.

The relative contribution is therefore not monotonic in the missing rate. Instead, it exhibits a dataset-by-missing-rate interaction: missingness controls how much co-observed evidence remains, while view number and heterogeneity, anchor-target overlap, clustering granularity, and dataset-specific hyperparameters determine whether first-order (KB-QAP) or second-order (L-QAP) matching is more useful.

Q2: What is the core mechanism for extending graph matching to multi-view clustering?

We embed graph matching in an anchored dual-expert align-fuse-refine framework rather than applying QAP independently to every view pair. An informative anchor view is selected using completeness, feature informativeness, and spectral information. For each target view, a soft correspondence matrix is initialized from co-observed samples by mutual top-rr matching in a shared PCA space.

Two QAP-based experts share this soft target: KB-QAP models first-order node similarity and local KNN-edge consistency, while L-QAP models second-order consistency in a compact anchor space using a Kronecker KNN skeleton prior, feature-induced dynamic affinity, TransformerConv, and Sinkhorn normalization; its anchor-level result is projected back to sample space.

Expert outputs are compared with previous pseudo-labels and classified as consistent, partially consistent, or conflicting. A momentum update produces the refined confidence matrix, which in the next iteration serves as the target for both matching losses and as positive-pair supervision for overlap-aware contrastive learning. This closes the representation-matching feedback loop. Finally, within-view and cross-view affinities are aggregated by Equation (8), combined with common-representation affinity, symmetrized, and passed to spectral clustering.

Q3: What does Equation (8) mean?

Equation (8) is an inference-time global-affinity aggregation rule, not a training loss. Let E(v)\mathcal{E}^{(v)} be the entities observed in view vv and g(v)g^{(v)} the local-to-global map.

Apq=∑v=1VA^pq(v)+∑v≠v⋆VS^pq(v)max⁡(∑v=1V1[p,q∈E(v)]+∑v≠v⋆V1[p,q∈E(v⋆)∩E(v)],1).A_{pq}=\frac{\sum_{v=1}^{V}\hat{\mathbf{A}}^{(v)}_{pq}+\sum_{v\neq v^{\star}}^{V}\hat{\mathbf{S}}^{(v)}_{pq}}{\max\left(\sum_{v=1}^{V}\mathbf{1}\left[p,q\in\mathcal{E}^{(v)}\right]+\sum_{v\neq v^{\star}}^{V}\mathbf{1}\left[p,q\in\mathcal{E}^{(v^{\star})}\cap\mathcal{E}^{(v)}\right],1\right)}.

ApqA_{pq} is the global affinity between pp and qq. A^(v)\hat A^{(v)} is the within-view KNN affinity mapped to global indices; S^(v)\hat S^{(v)} is the overlap-masked fused anchor-to-target similarity from KB-QAP and L-QAP. E(v)\mathcal{E}^{(v)} restricts contributions to observed entities, while g(v)g^{(v)} maps local matrices into one global N×NN\times N matrix.

The first numerator sum aggregates local structural evidence; the second aggregates cross-view correspondence evidence. The first denominator term counts views observing both pp and qq. The second counts target views whose overlap with the anchor contains both pp and qq. The maximum with one prevents division by zero.

Equation (8) therefore consolidates and normalizes multi-source evidence so that pairs with different observation coverage are comparable. It produces the intermediate global affinity, which is then blended with common-representation affinity, symmetrized, and used for spectral clustering. It has no learned parameters and receives no gradient; its role is to turn refined correspondence evidence into a clustering-ready graph.

We apologize that the protocol used in our earlier reply was inconsistent with that of Table 3, which caused inconvenience to your review. In this revision, we have unified the experimental protocol and present the new results below.

Comments on the paper

3 comments
~Louis_Dubois1OpenReview ↗

On the Paper's Incremental Contribution and Its Supporting Evidence

My concern is not whether the complete pipeline can produce competitive clustering scores. It is whether the mechanisms presented as contributions are new, necessary, and shown to account for those scores. The paper and rebuttal leave that question unresolved.

Prior work and the actual increment. The cited TPAMI 2026 COMMON+ already combines KB-QAP and Lawler-QAP experts, Kronecker-factorized association graphs, graph-based matching, outlier rejection, equal-weight fusion, and collaborative refinement with momentum. The overlap extends to the refinement rules themselves. What this submission adds is primarily IMVC adaptation: anchor selection, mutual-nearest initialization, compressed and annealed anchor-space matching, pseudo-label EMA, and the Eq. (8) spectral graph readout. These are legitimate candidates for a contribution, but their effects must be distinguished from the inherited architecture. COMMON+ also implements sparse association graphs; the quoted dense O(N4)O(N^4)-to-O(M4)O(M^4) reduction does not establish superiority over that implementation. CGMAA and SIMVC-SA are further IMVC precedents.

What is being learned? Section 3.1 gives local-to-global entity mappings g(v)g^{(v)}. For co-observed entities, the exact identity correspondence is Tij=1[g(a)(i)=g(v)(j)]T_{ij}=\mathbf{1}[g^{(a)}(i)=g^{(v)}(j)], obtainable by an ID join without class labels. Identity, reliability, and off-identity semantics are distinct tasks requiring distinct evidence. Replacing TT with a pseudo-target YY is not simply confidence weighting: BCE logit gradients w(p−Y)w(p-Y) and α(p−T)\alpha(p-T) agree over a nontrivial score interval only when Y=TY=T and w=αw=\alpha, for positive weights. Removing one expert does not establish a gain over retaining known identities and learning quality weights on them.

A denominator-only counterexample to Eq. (8). Fix embeddings, observation masks, perfect identity matches, affinity numerators, common graph, and the spectral-clustering procedure. Change only the divisor from max⁡(cij+dij,1)\max(c_{ij}+d_{ij},1) to max⁡(cij,1)\max(c_{ij},1), where dijd_{ij} is the extra anchor-target co-observation count. Even with perfect matches, dijd_{ij} changes intra-view edges. Two independently checked Euclidean k-NN constructions show that this can reverse the final clustering:

  • A connected four-dimensional torus with fixed k=4k=4 and nonvanishing edge weights switches its leading nontrivial spectral plane. At N=540,2160,8640N=540,2160,8640, identical indexed Lloyd starts yield 100% ACC without dijd_{ij} versus 33.33% with Eq. (8).
  • A connected fixed-k=22k=22 line-graph family has uniform normalized spectral gaps (λ2>0.025\lambda_2>0.025, λ3−λ2>0.0045\lambda_3-\lambda_2>0.0045) yet switches its unique optimal binary partition from 100% to 50% ACC. The finite construction is stable across independent K-means starts.

These structured-mask, ideal-expert examples test the inference operator, not Table 2. Crucially, the rebuttal varies numerator weights, whereas the failure is caused by the denominator. That intervention remains untested on the reported data.

The new ablation figures do not reconcile under a common protocol. The rebuttal defines CK=MK−M0C_K=M_K-M_0 and CL=ML−M0C_L=M_L-M_0, where MM averages ACC, NMI, and ARI. Thus CK−CL=MK−MLC_K-C_L=M_K-M_L is an identity for corresponding single-expert runs. Comparing Table 3, the supplementary ablations, and the rebuttal gains with exact integer arithmetic and rounding intervals yields 9 of 12 configurations with no feasible common M0M_0. At ALOI-100, r=0.3r=0.3, the two implied no-expert means are 66.4667% and 73.7400%; their admissible rounding intervals are separated by at least 7.2533 percentage points. Three settings also reverse expert ordering. If protocols differ, they must be identified; such figures cannot validate the original ablations as a matched comparison.

The missing evidence is specific: a matched COMMON+ adaptation or appropriate anchor baseline; true-ID and true-ID-plus-reliability controls; a frozen-affinity comparison changing only Eq. (8)'s denominator; and reconciled four-arm results (M0,MK,ML,MKL)(M_0,M_K,M_L,M_{KL}) on identified masks and seeds. An anonymized release of code, checkpoints, masks, and per-run outputs before decisions would permit independent verification.

As written, the paper reports performance of an assembled system but has not isolated the contribution, or established the reliability, of its distinctive mechanisms.

Sources: COMMON+ · official code · submission · rebuttal

We sincerely thank the reviewer for their constructive comments. Below, we provide point-by-point responses.

Response to the Comment on Prior Work and the Actual Increment

We thank the reviewer for this precise discussion of COMMON+ (Lin et al., TPAMI 2026). We agree that KB-QAP, Lawler-QAP and the align–fuse–refine skeleton originate in that work, and we do not claim these components as our own. Our contribution is the adaptation of this machinery to unsupervised, incomplete, multi-view clustering, where the annotations and pairwise output assumed by COMMON+ are unavailable; the adaptation is substantive in four respects.

(1) The refinement rule needs a different supervision. In COMMON+, co-divide and co-refine compare predictions against an annotated permutation. No such permutation exists here: Y^(v) is initialized from observed overlap and refined from expert agreement and historical confidence, so the rule cannot be transferred unchanged.

(2) Second-order matching moves into a learnable anchor space. COMMON+ builds its association graph over complete keypoint sets; our L-QAP expert matches with M << N, reducing the dominant cost from O(N^4) to O(M^4). We do not claim this alone establishes superiority over its sparse implementation; it makes second-order matching feasible on 10k-sample benchmarks such as ALOI-100 and Hdigit.

(3) The output and inference problem differ. COMMON+ returns a permutation for an image pair; we align one anchor against multiple targets and fuse within-view affinity, cross-view correspondence and the common graph before spectral clustering — a global readout with no counterpart in a pairwise matching pipeline.

(4) Correspondence quality is instrumental: the learned matrix supervises representation learning while the reported endpoint is clustering quality.

Two further precedents need the same distinction. CGMAA aligns anchors by cross-view graph matching with bipartite graph tensor learning; SIMVC-SA learns a soft alignment matrix under an orthogonal constraint, jointly with its anchor graph. In both, alignment is a variable of a shallow factorization whose output builds the clustering graph; here the correspondence is a per-pair confidence target for the contrastive positives and the two expert losses, and after momentum smoothing it supervises the next epoch. The difference is not soft versus hard assignment but the role the correspondence plays.

Conclusion. The shared elements are the solvers and the anchor-space factorization; what differs is the object supervised, the origin of that supervision, the arbitration between two structural orders, and the inference problem (V masked views fused into one affinity rather than a pairwise permutation). An increment adds a module inside an unchanged objective; here the supervision itself is reconstructed, so ablating the refinement changes what the encoder is trained on.

What is being learned?

We thank the reviewer for this precise clarification. Section 3.1 defines the known local-to-global maps

g(v):[Nv]→[N]g^{(v)}:[N_v]\rightarrow[N],

from which the exact identity correspondence

Tij=1[g(a)(i)=g(v)(j)]T_{ij}=\mathbf{1}[g^{(a)}(i)=g^{(v)}(j)]

can be constructed for co-observed entities without class labels. Although the paper does not explicitly introduce TT, the reviewer is correct that this identity target is available.

We also agree with the reviewer’s gradient argument: because Eq. (3) uses YijY_{ij} as the BCE target, replacing the exact identity target TijT_{ij} with a pseudo-target YijY_{ij} changes the optimization target and is not equivalent to merely multiplying the identity target by a confidence weight.

Our formulation distinguishes three quantities. The maps g(v)g^{(v)}, and therefore the implicit identity correspondence TT, encode exact entity identity. The term wijw_{ij} is an overlap-aware sample weight that gives larger weight to co-observed pairs, while YijY_{ij} is a similarity-derived soft pseudo-target. Specifically, Eq. (1) constructs YY from mutual top-KK matching and thresholded similarity, and the paper explicitly describes the result as “a per-pair soft confidence matrix rather than a verified instance correspondence.” Eqs. (6)–(7) subsequently update YY using expert agreement and historical confidence, after which YY is used as the BCE target in Eq. (3). Thus, YY is not obtained as TT multiplied by a reliability score; it is a separate pseudo-target that may assign nonzero values to off-identity pairs.

This distinction also clarifies the scope of the expert ablation. Removing one expert measures its marginal contribution within the YY-based supervision loop; it does not establish that YY is superior to retaining the exact identity matrix TT and learning a separate quality weight on it. We therefore do not make that stronger claim.

A denominator-only counterexample to Eq. (8)

We thank the reviewer for this careful counterexample. Under the stated assumptions—fixed embeddings, observation masks, affinity numerators, common graph, and spectral-clustering procedure—the construction is valid: changing only the divisor from max⁡(cij+dij,1)\max(c_{ij}+d_{ij},1) to max⁡(cij,1)\max(c_{ij},1) can change intra-view edge weights and alter the final clustering. Thus, we agree that Eq. (8) is not mathematically invariant to the denominator.

This result should nevertheless be distinguished from the behavior on our reported data. In our implementation, the denominator is defined over the fixed co-observation domain and counts every pair in that domain, including pairs whose fused similarity is zero; it is therefore an observation-support count rather than a count of nonzero matrix entries. We compared the original denominator with the support-count denominator on 100leaves and HW while holding embeddings, missingness masks, the common graph, and normalized spectral clustering fixed. Across the eight settings in Table R1, ACC differs by 0.15–1.62 points, with a mean absolute difference of 0.83 points and no systematic direction. The two normalizations therefore agree within 1.62 ACC points in all eight settings.

The counterexample consequently establishes an important mathematical limitation of the written operator, but it does not by itself establish a performance degradation on the reported datasets.

The new ablation figures do not reconcile under a common protocol.

We apologize that the protocol used in our earlier reply was inconsistent with that of Table 3, which caused inconvenience to your review. In this revision, we have unified the experimental protocol and present the new results below. We define the standalone contribution of expert e at missing rate r as �0��0�, where M is the average of ACC, NMI, and ARI. To compare the two experts without relying on an additional baseline run, we report the difference

�1��1� The no-expert baseline cancels in this subtraction, so �2��2�, where both single-expert variants are taken from Table 3 under the identical protocol. Positive values indicate that KB-QAP contributes more; negative values indicate that L-QAP contributes more.

Datasetr=0.1r=0.3r=0.5r=0.7
HW+0.93+1.56+0.68+0.06
100leaves−6.60+10.89+6.17−3.10
ALOI-100−1.91+1.27−3.05+0.64
Hdigit−10.68+10.04+10.27+2.82

Under the unified protocol, the relative contribution of the two experts is not monotonic in the missing rate but shows a dataset × missing-rate interaction.

As written, the paper reports performance of an assembled system but has not isolated the contribution, or established the reliability, of its distinctive mechanisms.

We thank the reviewer for this precise diagnosis. The main text and appendix already report ablations on all datasets; in original response we provide the complete set of controls the reviewer requested.

Operator level. With the encoders, missing-view masks, anchor selection, the common graph, k-NN construction and normalized spectral clustering held fixed, we compare (i) within-view fusion, (ii) Eq. (8) with true-ID matches, and (iii) Eq. (8) with learned matches, each evaluated both with and without the additional support-count denominator. Replacing the co-observation denominator by the support count changes ACC by at most 1.62 points (mean change -0.26; four settings improve, four worsen), and learned matches differ from true-ID matches by -1.45 points on average, again with no systematic direction.

Training level. We compare true-ID supervision, the frozen initial YY, and the momentum-refined YY at four missing rates, reporting correspondence error and semantic-edge quality directly. Correspondence error is 0.0000 for true-ID supervision, 0.9983 for the frozen target and 0.9709 for the momentum-refined target at r=0.1; momentum refinement lowers it from 0.9992 to 0.9756 at r=0.1 and from 1.0000 to 0.8598 at r=0.7.

These results show that our components do contribute.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.