- Rating
- 2(−2)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
An anchored dual-expert framework is developed for incomplete multi-view clustering to improve cross-view correspondence estimation by leveraging complementary first- and second-order structural information. The framework integrates two QAP-based experts and employs momentum-based refinement to iteratively update correspondence confidencas. Experiments under varying view missing rates demonstrate the method’s effectiveness.
Critical strengths
-
Integrating graph matching into multi-view representation learning is a reasonable approach to exploiting structural information across incomplete views.
-
Combining cross-view and intra-view affinities is beneficial for preserving complementary structural information in the final clustering graph.
Critical weaknesses
-
The interaction between correspondence optimization and representation learning remains unclear.
-
The integration of cross-view and intra-view affinities lacks sufficient justification.
Questions for authors
-
How do correspondence updates affect representation learning during training?
-
How are cross-view and intra-view affinities balanced, and how sensitive is clustering performance to their relative weights?
-
How stable are the final clustering results across random initializations and perturbations of the affinity matrix?
7 comments
Response to the Reviewer UL2o: Correspondence Updates, Affinity Balance and Stability
Response to the Reviewer UL2o
We thank the reviewer for these constructive questions and will incorporate the clarifications below into the revision.
Q1: How do correspondence updates affect representation learning during training?
Correspondence updates act as an online supervision signal for representation learning.
- Current correspondences guide the encoder. At each epoch, the current soft matrix determines the positive anchor-target pairs used in the overlap-aware contrastive loss. Pairs selected by the current pseudo-correspondence are pulled together in feature space, while other observed candidates contribute negative evidence through the InfoNCE denominator.
- Representations guide matching. The resulting features are passed to the KB-QAP and L-QAP experts, which produce first-order and second-order match scores.
- Correspondences are refined. The expert outputs are compared with the historical correspondence matrix. Pairs supported by both experts and the history are reinforced; partially supported pairs receive intermediate confidence; pairs for which both experts agree but the history disagrees are updated using the averaged expert scores; and unsupported entries decay.
- The refined matrix controls the next epoch. A momentum update produces , which is used in the next epoch as the soft target for both matching losses and the positive-pair target for contrastive representation learning.
The correspondence update itself does not directly update encoder parameters; instead, it changes the supervision used by the next training iteration. During that iteration, the contrastive and expert losses jointly update the encoder. This creates the feedback loop
The intended effect is to progressively refine cross-view alignment: improved representations provide more informative matching evidence, while refined correspondences provide more selective supervision for representation learning. Momentum, decay, and confidence-based refinement limit abrupt changes and prevent every individual expert prediction from immediately redefining the training targets.
Q2: How are cross-view and intra-view affinities balanced, and how sensitive is clustering performance to their relative weights?
Within-view and cross-view affinities are combined through an availability-normalized aggregation rather than an explicitly tunable global weight. For each pair , the within-view KNN affinities and cross-view fused similarities are first scattered into global matrices. Their sum is then divided by the number of available contributions for that pair (Eq. 8). Thus, each observed contribution receives equal weight.
So, the current paper does not report a sensitivity study for the relative weight between within-view and cross-view evidence. The existing parameter-sensitivity experiment varies , , , and , but not these affinity-level weights. Therefore, we cannot claim from the current experiments that clustering performance is insensitive to the intra-/cross-view balance. To address this point, we will add an ablation that explicitly defines
while fixing the other components, and report the mean ACC over multiple random seeds.
| Setting | orig | λ = 0.25 | λ = 0.50 | λ = 0.75 | λ = 1.00 |
|---|---|---|---|---|---|
| 100leaves, r = 0.1 | 0.9423 | 0.9513 | 0.9188 | 0.8638 | 0.7544 |
| 100leaves, r = 0.3 | 0.8331 | 0.8225 | 0.7906 | 0.6950 | 0.4244 |
| 100leaves, r = 0.5 | 0.6254 | 0.6225 | 0.6306 | 0.6125 | 0.2794 |
| 100leaves, r = 0.7 | 0.4381 | 0.4213 | 0.4175 | 0.4063 | 0.3088 |
| HW, r = 0.1 (0.50) | 0.9515 | 0.9490 | 0.9455 | 0.9395 | 0.8945 |
| HW, r = 0.3 (0.32) | 0.9340 | 0.9332 | 0.9425 | 0.9365 | 0.8455 |
| HW, r = 0.5 (0.23) | 0.9020 | 0.9045 | 0.9015 | 0.8990 | 0.7812 |
| HW, r = 0.7 (0.20) | 0.8640 | 0.8478 | 0.8403 | 0.7435 | 0.4220 |
The experimental results show that clustering performance generally declines as the cross-view weight increases. In particular, when its weight reaches twice that of the intra-view affinity, the performance degradation becomes pronounced. Nevertheless, performance remains relatively insensitive to the relative weights over most of the intermediate range and deteriorates substantially only when cross-view evidence becomes dominant.
Q3: How stable are the final clustering results across random initializations and perturbations of the affinity matrix?
A single response field is capped at 5,000 characters, so Q3 is answered in the follow-up comment directly below this response, covering both random initialization and perturbation of the affinity matrix. We apologize for the inconvenience.
Q3: Follow-up Comment for Stability Results
Q3: How stable are the final clustering results across random initializations and perturbations of the affinity matrix?
We address this question with two sets of experiments: random initialization, and affinity-matrix perturbation of two kinds, random edge deletion and bounded multiplicative Gaussian noise.
Random initialization
We fix the random seeds before the final affinity matrix is built. After entering the clustering stage, we set the K-means seed to , run the clustering 20 times, and report the mean and standard deviation of ACC.
| Dataset | r | ACC (mean ± std) |
|---|---|---|
| 100leaves | 0.1 | 0.9303 ± 0.0098 |
| 100leaves | 0.3 | 0.8176 ± 0.0142 |
| 100leaves | 0.5 | 0.6183 ± 0.0071 |
| 100leaves | 0.7 | 0.4272 ± 0.0060 |
| HW | 0.1 | 0.9208 ± 0.0295 |
| HW | 0.3 | 0.9314 ± 0.0109 |
| HW | 0.5 | 0.8988 ± 0.0115 |
| HW | 0.7 | 0.8458 ± 0.0003 |
| Scene15 | 0.1 | 0.4132 ± 0.0109 |
| Scene15 | 0.3 | 0.4038 ± 0.0150 |
| Scene15 | 0.5 | 0.3533 ± 0.0124 |
| Scene15 | 0.7 | 0.3158 ± 0.0066 |
Perturbations of the affinity matrix
We test the robustness of the affinity matrix for the three datasets at a missing rate of 0.3.
A. Random edge deletion. Let be the fraction of deleted affinity edges. Stability is measured by the difference between the perturbed ACC and the baseline ACC.
| Dataset | N = 0.1 | N = 0.2 | N = 0.3 |
|---|---|---|---|
| 100leaves | −1.21 | −4.15 | −9.37 |
| HW | −1.08 | −2.21 | −3.08 |
| Scene15 | −0.47 | +0.14 | +0.24 |
B. Bounded multiplicative Gaussian noise. We fix the final matrix and the spectral-clustering initialization and perturb only with a bounded, symmetric multiplicative perturbation. Each strength uses 20 independent perturbations; non-negativity, symmetry, the diagonal and the zero edges are preserved. We report mean and standard deviation.
100leaves, r = 0.3. Baseline: ACC 0.8331 / NMI 0.8958 / ARI 0.7291. Each entry is mean ± std over 20 independent perturbations.
| σ | ACC | NMI | ARI |
|---|---|---|---|
| 0.05 | 0.8091 ± 0.0130 | 0.8834 ± 0.0037 | 0.6995 ± 0.0100 |
| 0.10 | 0.8037 ± 0.0139 | 0.8840 ± 0.0037 | 0.6917 ± 0.0119 |
| 0.20 | 0.8033 ± 0.0131 | 0.8844 ± 0.0038 | 0.6916 ± 0.0118 |
| 0.30 | 0.7966 ± 0.0114 | 0.8822 ± 0.0044 | 0.6853 ± 0.0121 |
| 0.50 | 0.7917 ± 0.0127 | 0.8718 ± 0.0037 | 0.6870 ± 0.0111 |
HW, r = 0.3. Baseline: ACC 0.9340 / NMI 0.8694 / ARI 0.8621. Each entry is mean ± std over 20 independent perturbations.
| σ | ACC | NMI | ARI |
|---|---|---|---|
| 0.05 | 0.9337 ± 0.0004 | 0.8691 ± 0.0008 | 0.8682 ± 0.0009 |
| 0.10 | 0.9270 ± 0.0245 | 0.8664 ± 0.0102 | 0.8725 ± 0.0231 |
| 0.20 | 0.9321 ± 0.0006 | 0.8581 ± 0.0010 | 0.8669 ± 0.0012 |
| 0.30 | 0.9307 ± 0.0334 | 0.8532 ± 0.0116 | 0.8659 ± 0.0299 |
| 0.50 | 0.9199 ± 0.0357 | 0.8533 ± 0.0119 | 0.8580 ± 0.0305 |
Scene15, r = 0.3. Baseline: ACC 0.4198 / NMI 0.3875 / ARI 0.2357. Each entry is mean ± std over 20 independent perturbations.
| σ | ACC | NMI | ARI |
|---|---|---|---|
| 0.05 | 0.4105 ± 0.0143 | 0.3662 ± 0.0033 | 0.2213 ± 0.0050 |
| 0.10 | 0.4062 ± 0.0150 | 0.3658 ± 0.0027 | 0.2205 ± 0.0034 |
| 0.20 | 0.4057 ± 0.0133 | 0.3643 ± 0.0040 | 0.2195 ± 0.0055 |
| 0.30 | 0.4012 ± 0.0144 | 0.3605 ± 0.0038 | 0.2174 ± 0.0062 |
| 0.50 | 0.3918 ± 0.0132 | 0.3454 ± 0.0034 | 0.2119 ± 0.0068 |
We test stability in three ways. With 20 K-means restarts on a fixed affinity matrix, the standard deviation of ACC is at most 0.0295. Under random edge deletion, the loss is at most 1.21 points at N = 0.1, reaches 9.37 points on 100leaves at N = 0.3 but only 3.08 points on HW, and Scene15 is essentially unaffected (+0.24). Under bounded multiplicative noise at r = 0.3, ACC decreases gradually, by at most 4.14 points relative to baseline at the largest strength. Results are therefore stable to initialization and to noise. Sensitivity to edge deletion is dataset-dependent but bounded at mild levels, and a pronounced drop appears only under heavier deletion. We do not claim invariance.
A Perfect-Matching Counterexample to Eq. (8): Major Soundness Concerns
The review's 4/4 Clear Acceptance cites dual-expert matching and cross-/within-view affinity fusion as strengths. Both require stronger evidence. In particular, the fusion rule in Eq. (8) can worsen clustering even when the decoded instance matches are perfect.
1. A perfect-matching counterexample to Eq. (8).
Section 3.1 assumes known global entity IDs. If an expert predicts perfect instance matches, scattering those matches into global coordinates produces only diagonal entries. Thus, for distinct entities p=q, the expert contributes zero to the off-diagonal affinity. Nevertheless, Eq. (8) adds a cross-view co-observation count to the denominator. Writing Wpq for the summed within-view affinity, cpq for its observation count, and dpq for this extra count, Eq. (8) yields
Apq=cpq+dpqWpqrather thancpqWpq.
For two entities jointly observed in the anchor and one target view, their within-view edge is reduced from Wpq/2 to Wpq/3, even though neither expert adds an off-diagonal edge. This changes relative edge weights according to missingness patterns.
I checked a reproducible 16-entity, three-view example using identical observed embeddings, local-scale KNN graphs, a fixed common graph, and normalized spectral clustering. Two surrogate matching branches (KB-style/Hungarian and Gaussian-anchor/Sinkhorn/Hungarian) both decode the true instance matches with 100% accuracy. Fixing the expert outputs, within-view graphs, common graph, and clustering procedure, removing only the cross-view denominator count yields 100% clustering ACC; Eq. (8) yields 50%. The same failure occurs in a 64-entity version, and persists with self-loops removed.
This is a counterexample to the written fusion operator, not a reproduction of the authors' trained network. If the fused output is instead a dense semantic affinity, the authors must distinguish it mathematically from instance correspondence and state precisely which output and support were used to generate Table 2.
2. Pseudo-target supervision can oppose true-ID supervision.
In Eq. (3), Yij is a BCE target, not merely a weight on a known true match. For p=σ(z), the soft-target derivative is p−y, whereas weighting a true match (T=1) by y gives y(p−1). At p=0.8, y=0.3, these equal +0.50 and -0.06: gradient descent acts in opposite directions.
Even retaining true positives in InfoNCE does not remove this possibility. For a two-instance reduced objective,
J(s)=log(1+e−2s/0.8)+log(1+e5s),
the unique optimum is s≃−0.167847: the false match has higher similarity. This establishes a failure mode of the stated loss components, not guaranteed failure of the full model. Additional regularizers would need to counteract it. Moreover, Eq. (2) changes directly with positive-set membership, not with continuous confidence values when that set is fixed.
3. Current ablations do not identify the claimed source of improvement.
On HW at missing rate 0.3 (Table 3), removing L-QAP costs 0.32 ACC points and removing KB-QAP costs 1.20, whereas removing commonZ costs 16.15. At rate 0.1, removing L-QAP improves NMI from 87.56% to 88.81%. These results cannot establish that dual-expert confidence refinement drives the gains.
A decisive test would fix embeddings, missing masks, and spectral clustering, then compare (i) within-view fusion, (ii) Eq. (8) with true-ID matches, and (iii) Eq. (8) with learned matches, each with and without the extra denominator count. Training controls should compare true-ID supervision, frozen initial Y, and momentum-refined Y, with correspondence error and semantic-edge quality reported directly.
4. Novelty also needs isolation.
The cited 2026 TPAMI predecessor already uses KB-QAP, Lawler QAP, and align-fuse-refine cooperation; earlier IMVC work includes confidence-guided consensus graphs and anchor alignment. This paper must demonstrate what its adaptation adds beyond these components.
Conclusion. Strong average ACC is not evidence that correspondence refinement is correct or necessary. The perfect-matching fusion counterexample, the pseudo-target optimization conflict, and missing causal ablations materially weaken the review's stated strengths. I ask the reviewer to reconsider whether Clear Acceptance is justified until the authors specify the actual Table 2 fusion operator and demonstrate an independent benefit from learned correspondences.