acceptodds
Under review as a conference paper at ICLR 2027

Cross-View Prototype Calibration via Reciprocal Cyclic Transitions for Incomplete and Noisy Multi-View Clustering

Abstract

Multi-view clustering aims to uncover a consensus cluster structure from heterogeneous views of the same objects. Prototype-based methods summarize the clusters of each view with semantic prototypes and align them across views. However, missing views and feature noise distort local neighborhood structures and shift view-specific prototypes, rendering cross-view correspondences ambiguous and prone to many-to-one matches. Establishing reliable and reciprocal prototype correspondences under such degradation therefore remains challenging. We propose ReCAP, an imputation-free framework for Reciprocal Cyclic cAlibration of cross-view Prototypes under incomplete and noisy settings. ReCAP first applies mutual k-nearest-neighbor graph contrastive learning to preserve reliable local neighborhoods and stabilize prototype estimation. It then constructs sample-mediated cyclic transitions that propagate probability mass from each source prototype through the samples and prototypes of another view and back to the source view. Maximizing the probability of returning to the source prototype couples forward and backward transitions; we prove that, in the zero-loss limit, the two directional transitions reduce to a permutation and its inverse, i.e., a reciprocal one-to-one correspondence. A visitation regularizer further prevents the returned mass from collapsing onto a few prototypes. Experiments on six benchmark datasets demonstrate competitive or superior clustering performance under incomplete, noisy, and compound degradation settings, while further analyses support the effectiveness of cyclic prototype calibration. Our code is available at https://anonymous.4open.science/r/ReCAP-0301.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Incomplete Multi-view Clustering, Prototype Alignment, Cyclic Calibration, Representation Learning.

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_aJiGOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ReCAP for incomplete and noisy multi-view clustering. Mutual-nearest-neighbor contrastive learning stabilizes view-specific embeddings and prototypes. Sample-mediated transitions then carry probability between prototype sets and back, with a cyclic-return loss and a marginal visitation regularizer. The paper establishes a finite-loss bound on cyclic returnability and characterizes reciprocal permutations in the zero-loss limit. Experiments cover six datasets, nine baselines, separate and compound degradation, and several ablations.

Critical strengths

S1: The paper first aggregates representations through mutual nearest-neighbor contrastive learning and then gradually introduces prototype calibration. This design provides more stable representations before enforcing cross-view correspondences, reducing the risk of reinforcing unreliable prototype matches during early training. S2: The paper derives a finite-loss bound on cyclic returnability and shows that zero-loss limiting transitions form mutually inverse permutations. This analysis explains how the return objective encourages reciprocal one-to-one correspondence and gives the proposed calibration mechanism a precise structural interpretation. S3: The paper compares sample-mediated calibration with direct prototype cycles and replaces the alignment objective while retaining the other components. These controlled experiments help isolate the contribution of the proposed transition path and calibration objective, providing more direct support than overall clustering comparisons alone.

Critical weaknesses

W1. The methodological increment over prototypical random walks needs sharper articulation. ReCAP introduces distinct view-specific prototype sets and reciprocal effective transitions, which may be valuable, but their specific benefit should be separated from the inherited objective. W2. Some important evaluation and implementation details remain underspecified. CPAS combines raw similarity entropy with transitions reconstructed using ReCAP's definition; these quantities may differ from a baseline's actual matching rule. W3.The paper also does not clearly explain how the learned view-specific assignments produce one final partition when observed views differ across samples.

Questions for authors

Q1. What is the precise increment over Prototypical Random Walk Networks (Ayyad et al., 2021), beyond adapting the walk to distinct view-specific prototype sets?
Q2. How are view-specific prototypes or assignments converted into a single consensus label for each sample, particularly under fully incomplete two-view data?

4 comments
AuthorsOpenReview ↗

Response to Reviewer aJiG(1/2)

We thank the reviewer for the positive assessment and constructive feedback. We appreciate your recognition of our progressive learning strategy, theoretical foundations of prototype calibration, and experimental contributions. Below, we address the methodological novelty of cross-view prototype calibration, the evaluation details of CPAS, and the procedure for obtaining unified clustering assignments. Changes in the revised manuscript are highlighted in blue.

Concern 1. From Prototype Return Walks to Reciprocal Cross-View Calibration (W1 & Q1)

We clarify ReCAP’s contribution relative to PRWN as follows. ReCAP inherits the principles of prototype return and visitation regularization. Its main contribution is cross-view prototype correspondence calibration through the composition of two sample-mediated effective transitions.

PRWN uses prototype–sample–prototype return walks to promote compact, well-separated representations. ReCAP instead constructs two effective maps between view-specific prototype sets, A:Pv→PuA:P_v\rightarrow P_u and B:Pu→PvB:P_u\rightarrow P_v and optimizes the return probabilities of ABAB and BABA. Each complete cycle therefore traverses both prototype sets. The return objective couples the two cross-view maps and constrains their composed correspondence structure, without requiring predefined prototype-index correspondences across views. This is the main structural contribution beyond introducing separate prototype sets. Two accompanying designs adapt this mechanism to cross-view calibration. First, we optimize returns from both views to treat the starting directions symmetrically during finite-loss training. Second, we regularize the returned-prototype marginal, whereas PRWN regularizes sample visitation. The regularization target thus shifts from sample coverage to prototype-level return distributions. Appendices B and C provide a finite-loss bound on cyclic returnability and characterize reciprocal permutation structure in the zero-loss limit. Appendix D further connects structural and semantic consistency: when the return probabilities of semantically incorrect correspondences satisfy the stated upper bound, the lower bound on semantically consistent probability mass is non-decreasing as the cyclic loss decreases.

We assess these designs through controlled ablations within the same ReCAP framework: ①PRW_CrossView vs CrossView_SampleVisit: compare the walk structures while retaining sample-visitation regularization. ②CrossView_SampleVisit vs Full_PCC: compare visitation targets while retaining the same cross-view cycle.

The experiment is conducted in an incomplete scenario with a missingness rate of 0.5. All other training components and experimental settings are held fixed.( For the full experiment, see Table 13 of Appendix L. )

DatasetVariantACC (%)ARI (%)NMI (%)SRSSCRCPAS
Caltech5VPRW_CrossView84.14 ± 0.6370.54 ± 1.6775.01 ± 1.310.80350.83890.8190
Caltech5VCrossView_SampleVisit87.93 ± 1.5876.86 ± 3.0579.07 ± 2.450.80350.74930.7752
Caltech5VOurs88.89 ± 0.4078.66 ± 0.6480.72 ± 0.640.80100.89450.8452
CUBPRW_CrossView80.17 ± 0.4363.08 ± 0.6871.15 ± 1.080.81050.90060.8532
CUBCrossView_SampleVisit81.07 ± 2.4964.92 ± 3.4272.47 ± 2.660.81380.81340.8136
CUBOurs81.17 ± 2.2965.06 ± 2.6672.62 ± 2.010.79570.99110.8827

With sample-level visitation, CrossView_SampleVisit improves mean ACC over PRW_CrossView by 3.79 points on Caltech5V and 0.90 points on CUB, while SCR and CPAS decrease. With the cycle fixed, prototype-level visitation improves both SCR and CPAS on both datasets.

AuthorsOpenReview ↗

Response to Reviewer aJiG(2/2)

Concern 2. Clarification of Evaluation Metrics and Implementation Details (W2)

Thank you for the suggestion. CPAS uses a unified post-hoc protocol to assess the structure and semantic properties of prototypes learned by different methods under the same evaluation operator. Reconstructing transitions is part of this diagnostic protocol; each method retains its own training and matching mechanisms. Similarity-based entropy measures the concentration of prototype affinities, while transition-based metrics assess reciprocity and semantic consistency.

To complement CPAS, we additionally report semantic matching accuracy (SMA), which measures the semantic correctness of cross-view prototype correspondences, and class coverage rate (CCR), which measures the classes represented by the prototype set. Their definitions have been added to Appendix I. The correspondence rules used for each method and the supplementary results are reported below.

DatasetMethodSMA (%)CCR (%)
CUBCPMN10.00 ± 10.9592.00 ± 4.00
CUBProImp86.00 ± 23.3297.00 ± 4.00
CUBCPSPAN10.00 ± 3.1680.00 ± 3.16
CUBOurs97.00 ± 4.0099.00 ± 2.00
Caltech5VCPSPAN14.10 ± 3.8486.29 ± 5.24
Caltech5VOurs85.00 ± 4.0098.00 ± 1.00

Note. For CPMN and ProImp, SMA is computed using each method’s native prototype matching rule, and CCR uses its corresponding sample-to-prototype assignments. For CPSPAN, SMA and CCR are computed using the shared post-hoc evaluation protocol.

Among the methods shown, ReCAP achieves the highest mean SMA and CCR on both datasets. On CUB, ReCAP achieves 97% SMA and 99% CCR, compared with 86% and 97% for ProImp. On Caltech5V, ReCAP achieves 85% SMA and 98% CCR, compared with 14.10% and 86.29% for CPSPAN.


Concern 3. Unified Cluster Assignment under Missing Views (W3 & Q2)

We thank the reviewer for the constructive suggestion. We clarify that view-specific prototypes and their soft assignments are used for representation learning during training. The final partition is obtained by global clustering of the learned representations, without merging view-specific prototype indices or soft assignments.

After training, we compute the mask-normalized mean of the observed view representations for each sample:

zˉi=∑v=1VWi(v)zi(v)∑v=1VWi(v).\bar{\mathbf{z}}_i = \frac{\sum_{v=1}^{V} W_i^{(v)} \mathbf{z}_i^{(v)}}{\sum_{v=1}^{V} W_i^{(v)}}.

Here, Wi(v)∈{0,1}W_i^{(v)} \in \lbrace 0,1\rbrace indicates whether view vv is observed for sample ii. We apply a single KK-means model to all fused embeddings, using the benchmark class count for KK.

In the fully unpaired two-view setting, each sample has only one observed view, so its fused representation equals the corresponding view embedding. All embeddings from both views are clustered jointly to obtain a unified partition, without requiring paired observations or post-hoc alignment of view-specific prototype indices.

The author's response has resolved my issue, but I will keep my rating as it is.

We are pleased that our responses have addressed your concerns. Once again, we appreciate your positive evaluation!

Reviewer_Q27aOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ReCAP, an imputation-free framework for incomplete and noisy multi-view clustering. The method combines mutual kNN-based graph contrastive learning with sample-mediated cyclic transitions to improve cross-view prototype alignment. A cyclic return objective and a visitation regularizer are introduced to encourage reciprocal prototype correspondences. Experiments on six benchmark datasets evaluate the method under incomplete, noisy, and combined degradation settings.

Critical strengths
  1. The paper addresses a relevant problem in multi-view clustering by considering both missing views and feature noise within a unified framework.
  2. The proposed framework is clearly structured, and the cyclic prototype calibration mechanism is supported by mathematical analysis.
  3. The experimental evaluation covers multiple datasets and degradation settings, with comparisons against several existing methods. Ablation studies and sensitivity analyses are also provided to examine the proposed components.
Critical weaknesses
  1. The theoretical results in Proposition 1 and Corollary 1 establish that minimizing the cyclic return loss encourages reciprocal one-to-one prototype correspondences. However, structural reciprocity does not necessarily imply semantic correctness. For example, an incorrect mapping between prototypes from different semantic categories could still satisfy the cyclic consistency constraint, provided that the reverse mapping is consistent.
  2. PCC imposes bidirectional cyclic constraints on each pair of views independently. However, for three or more views, pairwise reciprocity does not guarantee global consistency. For instance, the correspondence from view 1 to view 3 may differ from the correspondence obtained through view 2. The current framework does not explicitly address this potential inconsistency.
  3. The paper reports competitive performance even under fully incomplete settings. However, when paired observations are entirely absent, the independently trained view-specific encoders lack explicit shared semantic anchors. Both cross-view MkNN construction and the sample-mediated transition mechanism rely on cosine similarities between representations from different views. It is therefore unclear how reliable cross-view semantic relationships can be established in the absence of paired observations, or how the method avoids reinforcing incorrect correspondences caused by initially misaligned representation spaces.
  4. The ablation results in Appendix I (Table 8) reveal substantial variation in the contribution of PCC. For example, introducing PCC improves ACC from 80.33% to 81.17% on CUB, but from 77.87% to 88.89% on Caltech5V. The paper provides limited analysis of the factors underlying this discrepancy.
Questions for authors
  1. Could the authors clarify how PCC prevents semantically incorrect but cycle-consistent prototype correspondences? Are there any mechanisms or empirical evidence demonstrating semantic correctness beyond structural reciprocity?
  2. How does the proposed method ensure globally consistent prototype correspondences across three or more views? Could the authors analyze potential inconsistencies between direct and indirect cross-view matching paths?
  3. How are reliable cross-view semantic relationships established when no paired observations are available? Could the authors further explain the alignment mechanism and provide evidence of its robustness to initially misaligned latent spaces?
  4. Could the authors analyze why PCC provides substantially different performance gains across datasets, particularly CUB and Caltech5V?
3 comments
AuthorsOpenReview ↗

Response to Reviewer Q27a(1/2)

We sincerely thank the reviewer for recognizing the relevance of our research problem, the clarity and theoretical support of our framework, and the comprehensiveness of our experimental evaluation. We address four issues below: semantic consistency of cross-view prototype correspondences, agreement between direct and indirect matches across views, alignment under fully unpaired conditions with initially misaligned representations, and why PCC’s gains vary across datasets. Changes in the revised manuscript are highlighted in blue.


Concern 1: Semantic Consistency of Prototype Correspondences (W1 & Q1)

Our cycle-consistency analysis establishes structural reciprocity between view-specific prototypes. ReCAP explicitly calibrates correspondences between view-specific prototypes, complementing FreeCSL’s focus on cross-view semantic consensus [1] and HiWA’s recovery of cross-domain correspondences under geometric assumptions [2]. During warm-up, PAL encourages within-view clustering. PCC then calibrates cross-view prototype correspondences through sample-mediated cycles.

To connect structural reciprocity with semantic consistency, we show that when the return probabilities of semantically incorrect matches are uniformly bounded above, reducing the cyclic return loss can only raise or preserve the lower bound on the transition probability mass assigned to semantically consistent matches. Under this condition, PCC’s cycle objective helps constrain semantic alignment. The derivation is given in Appendix D.

We evaluate semantic agreement directly with SMA, the fraction of prototype matches whose assigned semantic labels agree. CCR measures the fraction of ground-truth classes represented by at least one prototype’s majority label, and Purity measures how concentrated the labels of samples assigned to each prototype are. Definitions of SMA and CCR are provided in Appendix I.

DatasetVariantSMA (%)PurityCCR (%)
CUBw_oPCC91.00 ± 11.000.69 ± 0.0297.00 ± 4.00
CUBDirect_Prototype_Cycle91.00 ± 7.000.70 ± 0.0297.00 ± 2.00
CUBPCC97.00 ± 4.000.73 ± 0.0299.00 ± 2.00
Caltech5Vw_oPCC79.00 ± 4.000.58 ± 0.0194.00 ± 2.00
Caltech5VDirect_Prototype_Cycle79.00 ± 3.000.58 ± 0.0195.00 ± 2.00
Caltech5VPCC85.00 ± 4.000.71 ± 0.0298.00 ± 1.00

All experiments used a missing rate of 0.5. ReCAP improved SMA by 6 percentage points over w/o PCC on Caltech5V and by 6 points over both baselines on CUB. On Caltech5V, PCC also improved Purity by 0.13 over direct prototype cycling, suggesting that sample-mediated calibration yields more semantically coherent prototypes. ReCAP achieved the highest CCR on both datasets. Experimental results are updated in Table 10 of Appendix K.

[1] Dai, Yuzhuo, et al. "Imputation-free and alignment-free: Incomplete multi-view clustering driven by consensus semantic learning." 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025.

[2] Lee, John, et al. "Hierarchical optimal transport for multimodal distribution alignment." Advances in neural information processing systems 32 (2019).


Concern 2. Global Consistency of Cross-View Prototype Correspondences (W2 & Q2)

Although ReCAP does not impose a strict explicit constraint that guarantees global prototype consistency across three or more views, all view pairs are jointly optimized within the same framework and implicitly coupled through shared parameters.

To further examine potential differences between direct and indirect paths, we introduce a path consistency (PC) metric and report the mean PC scores below. The experiments were conducted in the incomplete multi-view setting with a missing rate of 0.5. (The results have been updated in Table 11 of Appendix K.)

Datasetw/o PCC PC (%)PCC PC (%)ΔPC (pp)
Caltech5V78.38 ± 0.9993.57 ± 3.06+15.19
Reuters89.22 ± 6.00100.00 ± 0.00+10.78
NUS-WIDE36.83 ± 3.0051.98 ± 3.73+15.15

With PCC, mean PC increases by 10.78–15.19 percentage points across the three datasets. Direct and indirect paths agree more often with PCC, although substantial disagreement remains on NUS-WIDE.

AuthorsOpenReview ↗

Response to Reviewer Q27a(2/2)

Concern3. Reliability and Robustness of Cross-View Semantic Alignment in Fully Unpaired Settings (W3 & Q3)

Thank you for raising this point. In the fully unpaired setting, PCC uses the same transition rule as in Eq. (10), with each view contributing its own observed sample pool. The transitions can therefore be computed from disjoint observations, and their semantic reliability still depends on the learned cross-view similarities. To assess sensitivity to latent misalignment, we apply an orthogonal perturbation and report the resulting clustering performance on DHA in Table 15 of Appendix L.

MethodConditionACC (%)ARI (%)NMI (%)
ReCAPNormal46.67 ± 0.5531.34 ± 1.2665.59 ± 0.92
Stress45.47 ± 0.5829.57 ± 1.1362.86 ± 0.70
CPMNNormal36.69 ± 2.5416.38 ± 1.9549.01 ± 1.65
Stress37.76 ± 2.0419.31 ± 1.8350.15 ± 2.23
SMILENormal36.65 ± 2.1520.87 ± 1.5852.36 ± 1.26
Stress38.72 ± 2.0522.11 ± 2.3853.72 ± 2.04
GLGCNormal38.47 ± 1.8022.40 ± 1.9554.62 ± 2.32
Stress36.48 ± 2.2819.63 ± 1.4451.21 ± 1.44
CAMERANormal36.77 ± 0.9321.51 ± 0.6654.52 ± 0.75
Stress38.63 ± 1.6924.31 ± 0.9657.61 ± 0.80

For ReCAP, the Normal and Stress branches start from the same checkpoint at the end of epoch 19 of joint training. Before the first PCC update, the Stress branch applies one fixed, label-independent orthogonal transformation to both the second-view embeddings and prototypes:

\widetilde{P}^{(2)} = P^{(2)} Q^\top,\qquad Q^\top Q = I.$$ We sample $Q$ by sign-corrected QR decomposition of a Gaussian random matrix [1]. At the point of intervention, this transformation preserves within-view distances and cosine similarities while perturbing cross-view similarities. On fully unpaired DHA, the perturbation reduces ReCAP’s mean ACC, ARI, and NMI by 1.20, 1.77, and 2.73 percentage points, respectively. Under the stress condition, ReCAP retains the highest mean on all three clustering metrics among the evaluated methods. [1] Mezzadri, Francesco. "How to generate random matrices from the classical compact groups." _arXiv preprint math-ph/0609050_ (2006). --- >### **Concern 4. Dataset-Dependent Performance Gains of PCC (W4 & Q4)** We thank the reviewer for pointing out that the gains from PCC vary across datasets. In Table 8, PCC improves ACC by 0.84 percentage points on CUB and 11.02 points on Caltech5V. To examine this difference, we evaluate prototype semantic quality and path consistency in the incomplete multi-view setting with a missing rate of 0.5. As shown below, without PCC, CUB already achieves an SMA of 91.00%, compared with 79.00% on Caltech5V, where prototype purity is also lower. With PCC, SMA increases by about 6 percentage points on both datasets, and CCR improves on both. Purity increases more on Caltech5V (from 0.58 to 0.71) than on CUB (from 0.69 to 0.73). On Caltech5V, PC also rises from 78.38% to 93.57%, indicating closer agreement between direct and indirect prototype matches. **These results suggest that the larger clustering gain on Caltech5V may be related to improvements in both prototype purity and agreement across matching paths.** | Dataset | Method | PC (%) | SMA(%)| Purity| CCR(%)| |:---:|:---:|:---:|:---:| :---:| :---:| | Caltech5V | PCC | **93.57 ± 3.06** | **85.00± 4.00** | **0.71±0.02** | **98.00 ± 1.00** | | Caltech5V | w/o PCC | 78.38 ± 0.99 | 79.00 ± 4.00 | 0.58±0.01 | 94.00± 2.00 | | CUB | PCC | - | **97.00± 4.00** | **0.73±0.02** | **99.00± 2.00** | | CUB | w/o PCC | - | 91.00 ± 11.00 | 0.69±0.02 | 97.00± 4.00|

The authors' response has addressed my concerns, and I will raise my score accordingly.

Reviewer_zk2TOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents ReCAP for multi-view clustering with missing observations and feature corruption. The framework uses view-specific autoencoders to encode and reconstruct observed inputs, contrastive objectives based on mutual nearest neighbors to organize representations, and periodically refreshed prototypes to represent clusters. Its calibration module constructs cross-view prototype transitions through sample representations and optimizes bidirectional return probabilities together with a returned-marginal regularizer. A DEC-style self-training objective sharpens sample assignments. The paper evaluates the framework on six datasets under synthetic missing-view, Gaussian-noise, and combined degradation settings, and reports component ablations, prototype-alignment measurements, and computational analyses.

Critical strengths
  1. The framework handles missing observations and feature corruption within an imputation-free formulation, with observation masks excluding unavailable inputs from reconstruction.

  2. At a missing rate of 0.5, the reported ACC exceeds the strongest listed competitors by 7.20, 8.94, and 8.43 percentage points on CUB, Reuters, and Caltech5V, respectively.

  3. The evaluation includes recent baselines, five-run statistics, and multiple degradation settings. Table 10 provides a relevant direct-cycle control and reports improvements from sample-mediated transitions on all three evaluated datasets.

Critical weaknesses
  1. ReCAP combines CPMN-style graph contrastive learning, PRWN-style prototype walks, and DEC-style self-training. The existing ablations do not separately establish the contributions of cross-view composition, bidirectional optimization, and the changed visitation target.

  2. Permutation assignments already admit mutually inverse mappings. The theoretical results characterize generic stochastic cycles without establishing semantic correspondence accuracy, while Figure 1 displays softmax affinities rather than the baselines’ actual hard assignments.

  3. The manuscript does not specify unified-label prediction, prototype reindexing, assignment fusion under missing views, or coordination of pairwise mappings across multiple views. The procedures for refreshing prototypes and updating self-training targets also lack operational details.

Questions for authors

please refer to critical weaknesses

3 comments
AuthorsOpenReview ↗

Response to Reviewer zk2T(1/3)

We thank the reviewer for the constructive feedback and for recognizing the strengths of our imputation-free framework. We provide component ablations, analyze how prototype transitions relate to semantic correspondence, and present additional semantic consistency results. We also clarify the metrics in Fig. 1, cluster label generation, and how mappings are coordinated across views. All additional experiments use the partially incomplete setting. Changes in the revised manuscript are highlighted in blue.


Concern 1: Additional Ablations of Cross-View Composition, Bidirectional Optimization, and Visitation Regularization (W1)

In the Ablation Studies and Hyperparameter Analysis sections of the appendix, we have examined the effects of within-view transition steps, sample-mediated transitions, and visitation regularization, demonstrating their contributions to ReCAP's performance. To further isolate the contributions of individual mechanisms, we conduct the following additional ablation studies:

① PRW_CrossView: Initiates transitions from prototypes in each view and returns to the source prototypes via samples from another view, using sample-level visitation regularization.

② CrossView_SampleVisit: Retains the complete cross-view prototype cycle and bidirectional return constraints, but applies visitation regularization to the sample visitation distribution.

③ Return_v_only and ④ Return_u_only: Retain the return loss in only one direction, respectively, while preserving bidirectional prototype visitation regularization.

The experimental results are presented below, and the corresponding analysis will be incorporated into Table 14 of Appendix L.

DatasetVariantACC (%)ARI (%)NMI (%)SRSSCRCPAS
Caltech5VPRW_CrossView84.14 ± 0.6370.54 ± 1.6775.01 ± 1.310.80350.83890.8190
Caltech5VCrossView_SampleVisit87.93 ± 1.5876.86 ± 3.0579.07 ± 2.450.80350.74930.7752
Caltech5VReturn_v_only88.66 ± 0.8278.24 ± 1.6480.18 ± 1.370.80060.74920.7738
Caltech5VReturn_u_only88.44 ± 0.9577.82 ± 1.7879.77 ± 1.350.79870.74710.7717
Caltech5VOurs88.89 ± 0.4078.66 ± 0.6480.72 ± 0.640.80100.89450.8452
CUBPRW_CrossView80.17 ± 0.4363.08 ± 0.6871.15 ± 1.080.81050.90060.8532
CUBCrossView_SampleVisit81.07 ± 2.4964.92 ± 3.4272.47 ± 2.660.81380.81340.8136
CUBReturn_v_only80.63 ± 2.5564.49 ± 3.1772.64 ± 2.020.81020.81140.8108
CUBReturn_u_only79.53 ± 2.9363.79 ± 2.8472.09 ± 1.820.80600.80520.8056
CUBOurs81.17 ± 2.2965.06 ± 2.6672.62 ± 2.010.79570.99110.8827
  1. Cross-View Transition Composition. With sample-level visitation regularization, CrossView_SampleVisit raises mean ACC over PRW_CrossView by 3.79 percentage points on Caltech5V and 0.90 on CUB, while SCR and CPAS decrease. Full ReCAP improves all three metrics over PRW_CrossView on both datasets.

  2. Visitation Target. Compared with CrossView_SampleVisit, ReCAP achieves comparable or better clustering performance while improving SCR and CPAS, supporting the role of prototype-level visitation regularization in structural calibration.

  3. Bidirectional Return Constraints. Return_v_only achieves clustering performance close to that of full ReCAP but yields lower SCR and CPAS, suggesting that constraining returns from both views improves cross-view structural calibration.

AuthorsOpenReview ↗

Response to Reviewer zk2T(2/3)

Concern 2: Semantic Correspondence Accuracy and Clarification of Figure 1 (W2)

Response to Semantic Correspondence:

Thank you for your constructive comments. Our theoretical analysis of cycle consistency establishes a reciprocal correspondence structure, rather than unconditional semantic identifiability. ReCAP focuses on explicitly calibrating reciprocal correspondences between view-specific prototypes, complementing existing studies such as FreeCSL [1], which emphasizes semantic consensus, and HiWA [2], which investigates correspondence recovery under geometric assumptions. Specifically, PAL learns preliminary semantic structures during warm-up, after which PCC calibrates cross-view prototype correspondences through sample-mediated cyclic transitions.

To further connect structural and semantic consistency, we derive a conditional bound showing that, when the return probabilities of semantically incorrect correspondences are sufficiently bounded, the lower bound on semantically consistent probability mass is non-decreasing as the cyclic loss decreases. The detailed derivation has been added to Appendix D.

We further evaluate semantic correspondence using three metrics: Semantic Matching Accuracy (SMA), which measures the proportion of semantically matched prototypes across views; Class Coverage Rate (CCR), which measures the proportion of ground-truth classes represented by prototypes; and Purity, which measures the proportion of samples belonging to the dominant class within each prototype. Definitions of SMA and CCR have been added to Appendix I.

We consider two comparison variants: (1) Direct_Prototype_Cycle, which performs cyclic transitions directly between prototypes, and (2) w/o PCC, which removes the PCC module. The results are presented below.

DatasetVariantSMA (%)PurityCCR (%)
CUBw_oPCC91.00 ± 11.000.69 ± 0.0297.00 ± 4.00
CUBDirect_Prototype_Cycle91.00 ± 7.000.70 ± 0.0297.00 ± 2.00
CUBPCC97.00 ± 4.000.73 ± 0.0299.00 ± 2.00
Caltech5Vw_oPCC79.00 ± 4.000.58 ± 0.0194.00 ± 2.00
Caltech5VDirect_Prototype_Cycle79.00 ± 3.000.58 ± 0.0195.00 ± 2.00
Caltech5VPCC85.00 ± 4.000.71 ± 0.0298.00 ± 1.00

Compared with w/o PCC, ReCAP improves SMA by 6 percentage points on Caltech5V and by 6 points on CUB under both settings. Compared with direct prototype cycles, PCC further improves Purity by 0.13 on Caltech5V, demonstrating the empirical semantic benefits of sample-mediated calibration. Moreover, ReCAP achieves the highest CCR across all evaluated settings. Experimental results are updated in Table 10 of Appendix K.

Clarification Regarding Figure 1: A hard permutation assignment is one-to-one and admits an inverse. Figure 1 applies the same row-wise cosine–softmax transformation and temperature across methods to visualize the concentration of cross-view prototype affinities. The baselines' hard assignments are determined by their original matching rules.

[1] Dai, Yuzhuo, et al. "Imputation-free and alignment-free: Incomplete multi-view clustering driven by consensus semantic learning." 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025.

[2] Lee, John, et al. "Hierarchical optimal transport for multimodal distribution alignment." Advances in neural information processing systems 32 (2019).

AuthorsOpenReview ↗

Response to Reviewer zk2T(3/3)

Concern 3: Implementation Details of Label Prediction and Prototype Coordination

Thank you for the feedback. We explain how ReCAP combines embeddings from available views to assign one cluster label per sample, maintains prototype indices within each view, coordinates pairwise mappings across views, and updates prototypes and self-training targets.

① Unified Cluster Label Prediction and Assignment Strategy under Missing Views:

View-specific prototypes and their soft assignments are used for representation learning during training. Final cluster labels are not obtained by directly merging prototype indices or soft assignments across views. Instead, after representation learning, we compute the mask-normalized average of the available view embeddings for each sample:

zˉi=∑v=1VWi(v)zi(v)∑v=1VWi(v).\bar{\mathbf{z}}_i = \frac{\sum_{v=1}^{V} W_i^{(v)} \mathbf{z}_i^{(v)}}{\sum_{v=1}^{V} W_i^{(v)}}.

Here, Wi(v)∈{0,1}W_i^{(v)}\in \lbrace 0,1 \rbrace indicates whether view vv of sample ii is observed, assuming at least one available view per sample. We then apply global K-means to the fused embeddings of all samples to obtain the final clustering assignments. Thus, all predicted labels are determined by a single clustering process, without requiring post-hoc alignment of view-specific prototype indices. Ground-truth labels are not used in this process.

For samples with only one observed view, the fused embedding equals that view's embedding. When all views are available, the formulation reduces to their equally weighted average.

② Prototype Index Reordering:

View-specific prototypes remain learnable throughout training.

  • Early in training, we periodically run K-means on each view’s observed embeddings to estimate candidate centers.
  • Within each view, Hungarian matching aligns the candidate centers with the current prototype indices before the momentum update in Eq. (8), keeping the indices consistent across refreshes.
  • After structural warm-up, we gradually introduce the PCC loss to jointly optimize the encoders and prototypes through cross-view cycles.
  • Once momentum updates stop, the joint loss continues to update the prototypes through backpropagation.

③ Coordination of Pairwise Mappings Across Multiple Views

ReCAP jointly optimizes PCC across view pairs. Each view uses the same encoder and prototypes in all pairwise objectives involving that view. An update from one pair can therefore affect that view’s mappings to other views.

We measure path consistency (PC) as the fraction of source prototypes for which the direct mapping and the mapping through an intermediate view reach the same target prototype. Table 11 in Appendix K reports the results under the incomplete-view setting.

Datasetw/o PCC PC (%)PCC PC (%)ΔPC (pp)
Caltech5V78.38 ± 0.9993.57 ± 3.06+15.19
Reuters89.22 ± 6.00100.00 ± 0.00+10.78
NUS-WIDE36.83 ± 3.0051.98 ± 3.73+15.15

With PCC, mean PC increases by 10.78–15.19 percentage points across the three datasets. Direct and indirect paths agree more often with PCC, although substantial disagreement remains on NUS-WIDE.

④ Prototype Updates and Self-Training Target Updates

Prototype Updates: View-specific prototypes are maintained as learnable parameters throughout training.

(i) During the first 100 epochs of joint training, we apply K-means to the observed sample embeddings of each view within each batch at epochs 10, 20, ..., 100 to estimate candidate centers.

(ii) If the number of observed samples in a view is smaller than the number of prototypes KK, the refresh is skipped for that view. Otherwise, Hungarian matching aligns the candidate centers with the current prototype indices within the same view, followed by momentum updates according to Eq. (8). This matching does not directly align prototype indices across views.

(iii) After structural warm-up, the PCC loss is gradually introduced to jointly optimize encoders and prototypes through backpropagation, calibrating cross-view correspondences. Momentum-based refreshment stops after epoch 100, while prototypes continue to be updated through the joint loss.

Self-Training Target Updates:

  • At each iteration when the clustering loss is active, we compute Student-t soft assignments $Q^{(v)} $ from the current embeddings and prototypes for the observed samples in each view.
  • We compute the targets P(v)P^{(v)} using the DEC formulation [1] in Eq. (20), with soft cluster frequencies computed from each view’s observed samples in the current batch.
  • We stop gradients through $P^{(v)} $ and backpropagate through $Q^{(v)} $ to update the encoders and prototypes. We downweight per-sample KL losses for low-confidence assignments.

[1] Xie, Junyuan, Ross Girshick, and Ali Farhadi. "Unsupervised deep embedding for clustering analysis." International conference on machine learning. PMLR, 2016.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revision

Dear Reviewers and AC/SAC,

We thank all reviewers for their constructive feedback. We are delighted that reviewers find our paper has a well-structured framework (Q27a), clear presentation ( zk2T, aJiG), sufficient mathematical analysis (Q27a, aJiG) and comprehensive experimental evaluation (zk2T, Q27a, aJiG).

We address the reviewers’ questions in our point-by-point responses and summarize the main updates below. We have incorporated these clarifications and new experimental results into the revised manuscript and appendix. Changes in the revised manuscript are highlighted in blue.

  • ReCAP’s increment over PRWN and the role of each design choice (zk2T, aJiG). We compare alternative cross-view transition constructions and ablate bidirectional optimization and the target visitation distribution.
  • Structural reciprocity and semantic correctness (zk2T, Q27a). We derive a conditional lower bound on semantically consistent transition mass in terms of cycle return probability and use ablations to assess PCC’s effect on semantic agreement.
  • Prototype correspondence across multiple views (zk2T, Q27a). We explain how ReCAP handles prototype correspondence across three or more views and measure agreement between direct and indirect matches.
  • Semantic alignment without paired observations (Q27a). We explain how ReCAP constructs cross-view transitions without paired samples and report a comparative stress test of its sensitivity to initial latent-space misalignment.
  • Cluster label assignment, experimental settings, and metrics (zk2T, aJiG). We clarify how ReCAP produces cluster labels and specify the configurations and evaluation protocols used in our comparisons.
  • Variation in PCC gains across datasets (Q27a). We compare ReCAP with and without PCC and examine how the quality of cross-view prototype matches varies across datasets.

Please see our point-by-point responses for details.

Best regards,

Authors of Paper 2621

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.