acceptodds
Under review as a conference paper at ICLR 2027

Soft-Label Tensor Learning for Continual Multi-View Clustering

Abstract

Multi-view clustering (MVC) partitions data with multiple views in an unsupervised manner. Commonly, it assumes that all data views are available in advance; however, in many real-world scenarios, new data views are accumulated over time, bringing the topic of continual multi-view clustering (CMVC). In this paper, we suggest two key constraints that the task of CMVC should follow: cross-session information utilization, given the limitations on historical data, newly arrived view information and retained historical clustering information should be effectively utilized for cross-session knowledge transfer; structural stability, in an incremental scenario, the clustering structure must be updated in a stable manner. Specifically, to achieve these constraints, we introduce the probability membership module to learn the relationship between current soft labels and historical soft labels, thereby identifying useful information introduced by the current view while leveraging historical clustering information. We further introduce the tensor correction learning module, which jointly models the current soft labels, historical soft labels, and the learned consistency structure as a third-order tensor. It employs the Low-Bias Refined norm to suppress redundant structural components while retaining the dominant ones, thereby enhancing the stability of the clustering structure. Upon these above ideas, we propose a novel CMVC method, namely Soft-Label Tensor Learning for Continual Multi-View Clustering (STL-CMVC). We conduct extensive experiments to evaluate STL-CMVC on multiple benchmark datasets. Empirical results demonstrate that STL-CMVC can significantly outperform existing baseline methods.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Multi-view clustering, Tensor Learning, Continual Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_oCy5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies continual multi-view clustering, where views of the same samples arrive one after another and only the soft labels from the previous session are kept. In each session, SLT-CMVC runs entropy-regularized fuzzy C-means on the new view and uses a probability membership module to relate the current soft labels to the historical ones. It then stacks the current soft labels, the historical soft labels, and the learned consistency matrix into a third-order tensor, and regularizes it with a new Low-Bias Refined (LBR) norm. Optimization is done with ADMM. Experiments on seven datasets compare SLT-CMVC with ten single-view and incremental multi-view clustering methods.

Critical strengths

This paper considers a practical setting for continual multi-view clustering: raw features of earlier views are deleted, and only compact clustering information is kept.

By combining cross-session consistency learning with tensor-based correction, the framework gives a fairly complete way to use new views while keeping useful historical information.

Critical weaknesses
  1. Several update rules are derived incorrectly. In Eqs. (19)–(20), quadratic terms that depend on UtU^t itself (the cross-session term and the ADMM penalty) are treated as constants, so the softmax update does not solve the UtU^t subproblem. In Eq. (29), a μ\mu-related term appears in the numerator but not in the denominator, so ∑kqik=1\sum_k q_{ik}=1 is not guaranteed. Eq. (26) updates the multiplier as A+μ(U−M)A+\mu(U-M), which does not match the penalty in Eq. (5), and the proof in Appendix 8.3 uses A+μ(M−U)A+\mu(M-U) instead. I would like to see correct derivations for each subproblem and confirmation that the code follows them.
  2. Many equations and notations are inconsistent, making the method hard to follow and reproduce. For example, Eq. (24) leaves out μ\mu, and Eq. (25) takes the derivative at ηiter+1\eta^{\mathrm{iter+1}}, which does not agree with the MM derivation that follows. Indices i,j,ki,j,k are mixed up in several equations, the size and normalization constraint of QQ change between sections, and the numerator and denominator of Eq. (29) use different terms. In the main text, Eqs. (1), (2), and (4) only require uik∈(0,1)u_{ik}\in(0,1) and leave out ∑kuik=1\sum_k u_{ik}=1.
  3. Convergence is not properly proved. According to the paper, the objective is semi-algebraic because it consists of polynomials, logarithms, and exponentials, and therefore satisfies the KL property. Logarithms and exponentials are not semi-algebraic in general, so the KL property has to be shown in another way. Boundedness is also argued in a circle: MM is updated using AA, and AA is updated using U−MU-M, yet each is used to bound the other. Sufficient descent and the relative-error condition are mostly stated rather than proved, and they do not cover every alternating update. As it stands, the proof does not show convergence to a KKT stationary point.
  4. It is not clear that LBR offers a real advantage. Log-Sum, Schatten-pp, and other nonconvex surrogates are all designed to keep large singular values while shrinking small ones. In Table 6, LBR improves on them only slightly and is even worse on one dataset. The paper also does not say whether the other surrogates use their best parameter settings, so it is hard to tell whether these differences are meaningful.
Questions for authors
  1. Could the authors give a complete and rigorous proof that the algorithm converges to a KKT stationary point?
  2. View order is tested only from high to low quality and the reverse. Is the method still stable under random view orders?
3 comments

Q1: Thank you for requesting a rigorous convergence analysis. We have corrected the UU and QQ updates using KL majorization and aligned the tensor shrinkage step with the LBR penalty.

For a fixed session with finite data, positive row-normalized initial memberships, and μk=μ0ρk\mu_k=\mu_0\rho^k, where μ0>0\mu_0>0 and ρ>1\rho>1, the multiplier sequence is bounded and the primal iterates converge to a feasible limit in exact arithmetic. Let Uk=stack⁡(H,Uk,Qk)\mathcal U^k=\operatorname{stack}(H,U^k,Q^k), with HH fixed, and Zk=Uk+1−Ak/μkZ^k=\mathcal U^{k+1}-A^k/\mu_k.

Since 0≤fε′(σ)≤1/ε0\le f_\varepsilon'(\sigma)\le 1/\varepsilon, each singular value in the tensor update shrinks by at most 1/(εμk)1/(\varepsilon\mu_k). Parseval's identity and the multiplier update give ∥Ak+1∥F=μk∥Mk+1−Zk∥F≤min⁡(n,c)ε\|A^{k+1}\|_F =\mu_k\|\mathcal M^{k+1}-Z^k\|_F \le \frac{\sqrt{\min(n,c)}}{\varepsilon}. Thus, for a uniform bound CC on ∥Ak∥F\|A^k\|_F, ∥Mk+1−Uk+1∥F=∥Ak+1−Ak∥Fμk≤2Cμk→0\|\mathcal M^{k+1}-\mathcal U^{k+1}\|_F=\frac{\|A^{k+1}-A^k\|_F}{\mu_k} \le \frac{2C}{\mu_k}\to0.

The membership rows lie on the simplex and the updated centers lie in the convex hull of the data. Together with μk{\mu_k} ∥Mk−Uk∥F≤2Cρ\|\mathcal M^k-\mathcal U^k\|_F\le 2C\rho,

for k≥1k\ge1, this bounds the non-entropy gradients in the UU and QQ updates. For either block, a row pkp^k is updated by normalizing the exponentials of zak=τklog⁡pak−gakη+τk,τk≥μkz_a^k=\frac{\tau_k\log p_a^k-g_a^k}{\eta+\tau_k}, \qquad \tau_k\ge\mu_k, where η≥0\eta\ge0 is its entropy coefficient and ∥gk∥∞≤G\|g^k\|_\infty\le G.

Consequently, ∣log⁡pak+1pbk+1∣≤∣log⁡pakpbk∣+2Gμk\left|\log\frac{p_a^{k+1}}{p_b^{k+1}}\right|\le \left|\log\frac{p_a^k}{p_b^k}\right| +\frac{2G}{\mu_k}. This also holds with the implementation's floor on centered logits, since clipping cannot enlarge a pairwise logit difference. As ∑kμk−1<∞\sum_k\mu_k^{-1}<\infty, the log ratios and log⁡pk\log p^k are uniformly bounded. Hence ∥zk−log⁡pk∥∞=O(μk−1)\|z^k-\log p^k\|_\infty=O(\mu_k^{-1}).

The normalized exponential map is Lipschitz and maps log⁡pk\log p^k back to pkp^k for each previously updated row, including with the logit floor. It follows that ∥pk+1−pk∥∞=O(μk−1)\|p^{k+1}-p^k\|_\infty=O(\mu_k^{-1}). The row differences are summable, so UkU^k and QkQ^k converge to strictly positive probability matrices. The weighted center update then gives convergence of VkV^k, and the primal residual bound gives M⋆=stack⁡(H,U⋆,Q⋆)\mathcal M^\star=\operatorname{stack}(H,U^\star,Q^\star).

The result establishes primal convergence and feasibility. (Gaussian Regression-Driven Tensorized Incomplete Multi-View Clustering with Dual Manifold Regularization NeurIPS-2025)

Q2: Thank you for raising the issue of random view order. We evaluated all six permutations of Synthetic3d. For each permutation, we ran 50 independent runs using the same set of 50 seeds. The table below shows the mean final ACC.

View order1→2→31→3→22→1→32→3→13→1→23→2→1
Final ACC (%)90.4592.1894.3191.5290.1289.97

The final average ACC value was 91.43%. This indicates that even with a random view order, the proposed method is still able to maintain a stable clustering structure.

Dear Reviewer oCy5 ,

Thank you for your time and effort in reviewing our paper. Your comments have been valuable in helping us improve its quality.

As the author–reviewer discussion period is scheduled to end in approximately two days, we kindly encourage you to review our rebuttal and let us know whether your concerns have been adequately addressed.

Thank you again for your time and consideration.

Best regards, The Authors

Thank you for the detailed response and additional clarifications. The rebuttal is helpful, but my overall assessment of the paper remains unchanged. I will maintain my original score.

Reviewer_pT9dOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies continual multi-view clustering under a restricted-memory setting, where historical features are discarded and only soft clustering assignments are retained. The proposed STL-CMVC combines entropy-regularized fuzzy clustering, a probability membership module for cross-session consistency, and a tensor correction module using a bounded spectral penalty termed the Low-Bias Refined norm. Experiments compare the method with ten baselines on seven datasets, with additional ablation, efficiency, regularizer, and view-order studies.

Critical strengths
  1. The restricted historical-access setting is practically motivated. Maintaining cluster-level information rather than historical feature representations provides a compact mechanism for transferring information between sessions.

  2. The experimental coverage is reasonably broad. In addition to benchmark comparisons, the appendix includes runtime and memory measurements, alternative spectral regularizers, and two view-order settings. These analyses help characterize the proposed method beyond its final clustering accuracy.

Critical weaknesses

1. The optimization procedure is not convincingly derived from the stated objective, and the convergence proof does not resolve the discrepancies.

For example, define sij=qi⊤qjs_{ij}=q_i^\top q_j. The derivative of the cross-session term

∑jsij∥uit−ujt−1∥22\sum_j s_{ij}\left\|u_i^t-u_j^{t-1}\right\|_2^2

with respect to uiktu_{ik}^t is

-2\sum_j s_{ij}u_{jk}^{t-1}.$$ This does not match the expression involving $h_{ij}$ in Eq. (19). Consequently, the claimed update in Eq. (20) is not established as a solution of the stated subproblem. There is also an explicit inconsistency in the multiplier update. Eq. (26) uses $$A\leftarrow A+\mu\left(\mathcal U-\mathcal M\right),$$ whereas Appendix 8.3 uses the opposite residual: $$A\leftarrow A+\mu\left(\mathcal M-\mathcal U\right).$$ The theoretical analysis therefore appears to concern a different update from the algorithm as written. Moreover, descent of the M-step alone does not establish the claimed sufficient descent over an entire iteration including the multiplier update. These issues require corrected derivations and clarification of the implementation used for the experiments. **2. The empirical evidence contains metric inconsistencies that undermine confidence in the reported improvements.** Under standard definitions, clustering Purity must be at least permutation-matched ACC when computed from the same predictions: $$\operatorname{Purity}\geq\operatorname{ACC}.$$ This inequality also holds after averaging over the same runs. However, Table 2 reports ACC/Purity values of 35.21/26.51 for the PMM-only variant on Wiki and 42.18/29.51 for the TCLM-only variant. Table 1 contains similar discrepancies, such as 68.12/37.65 for UCFCM on Synthetic3d. These differences are too large to be explained by rounding. The authors should provide the metric definitions, evaluation code, and corrected results. Since the inconsistencies affect both baseline comparisons and ablations, they directly weaken the evidence for the proposed components. Furthermore, although Table 7 examines two view orders, the evaluation mainly reports final-session performance. It does not directly establish the central structural-stability claim through session-wise performance or controlled experiments with uninformative or corrupted incoming views. The absence of variability estimates, despite reporting averages over 50 runs, also makes the smaller improvements difficult to assess.
Questions for authors
  1. Can the authors provide corrected derivations for the membership updates and reconcile the opposite multiplier-update signs in Eq. (26) and Appendix 8.3? Which equations were actually implemented, and do the corrections change the reported results?

  2. How are ACC and Purity computed? Can the authors explain the cases where Purity is substantially below ACC and provide evaluation code and corrected tables where necessary?

  3. Can the authors provide session-wise clustering results, including variability across runs, to show whether historical clustering quality is preserved when a weak or noisy view arrives? This would materially affect my assessment of the claimed structural stability.

3 comments

Q1: Thank you for identifying these inconsistencies. We have corrected the membership updates and reconciled the multiplier convention.

Corrected update of UtU^t in Eqs. (19)–(20). Let dik=∣xit−vkt∣22d_{ik}=\\|x_i^t-v_k^t\\|_2^2 and sij=qiTqjs_{ij}=q_i^Tq_j. In the penalty term of Eq. (18), we correct the definition to B=M+A/μ\mathcal B=\mathcal M+\mathcal A/\mu, with bikb_{ik} denoting entry (i,k)(i,k) of the slice corresponding to UtU^t. Fixing the other variables and differentiating Eq. (18) gives the corrected first-order condition for row ii: $ d_{ik}+\lambda(\ln u_{ik}^t+1) +2\lambda_1\sum_{j=1}^{n}s_{ij}(u_{ik}^t-u_{jk}^{t-1}) +\mu(u_{ik}^t-b_{ik})+\alpha=0, \sum_{k=1}^{c}u_{ik}^t=1. $ Here α\alpha is the multiplier for the row-sum constraint of row ii. Both quadratic terms contribute terms that depend on the unknown uiktu_{ik}^t. To retain the exponential normalization in Eq. (20), we linearize the quadratic terms at uikt,iteru_{ik}^{t,\mathrm{iter}} and upper-bound their quadratic remainder using $ \frac12\sum_{m=1}^{c}(u_{im}^{t}-u_{im}^{t,\mathrm{iter}})^2 \leq \sum_{m=1}^{c}u_{im}^{t} \ln\frac{u_{im}^{t}}{u_{im}^{t,\mathrm{iter}}}.$ The remainder has coefficient 2λ1∑j=1nsij+μ2\lambda_1\sum_{j=1}^{n}s_{ij}+\mu. Minimizing the resulting upper bound under the row-sum constraint yields. $ u_{ik}^{t,\mathrm{iter}+1} = \frac{ \exp\!\left[ \frac{ (2\lambda_1\sum_{j=1}^{n}s_{ij}+\mu) (\ln u_{ik}^{t,\mathrm{iter}}-u_{ik}^{t,\mathrm{iter}}) -d_{ik}+2\lambda_1\sum_{j=1}^{n}s_{ij}u_{jk}^{t-1}+\mu b_{ik} }{ \lambda+2\lambda_1\sum_{j=1}^{n}s_{ij}+\mu } \right] }{ \displaystyle\sum_{m=1}^{c} \exp\!\left[ \frac{ (2\lambda_1\sum_{j=1}^{n}s_{ij}+\mu) (\ln u_{im}^{t,\mathrm{iter}}-u_{im}^{t,\mathrm{iter}}) -d_{im}+2\lambda_1\sum_{j=1}^{n}s_{ij}u_{jm}^{t-1}+\mu b_{im} }{ \lambda+2\lambda_1\sum_{j=1}^{n}s_{ij}+\mu } \right] }. $ The upper bound agrees with the original row objective at the current iterate. Its minimization therefore gives one majorization–minimization update with a descent guarantee for the UtU^t subproblem with the other variables fixed. Multiplier convention and implementation. The residual in Eq. (5) is R=M−UR=\mathcal M-\mathcal U. The corresponding dual update is $ A\leftarrow A+\mu(\mathcal M-\mathcal U). $ Both the original and revised implementations use this sign, as does Appendix 8.3. We will correct the opposite sign printed in Eq. (26). The original UU update used a fixed affinity matrix and summed the penalty center across each row. The resulting row constant cancels during softmax normalization. The original QQ update normalized linear scores from the first cc columns of the sample-distance matrix DD. The revised implementation uses the gradients and KL updates derived above.

Q2: Thank you for pointing out the inconsistent entries. The cases where the tables show Purity below ACC arose from transcription errors during table preparation. The original evaluation outputs for the affected runs were computed correctly.

Q3: Thank you for proposing a direct test of the stability of historical clustering. We evaluated the revised solver using the Synthetic3d dataset, following the view sequence 1→2→3. The first and third views remained in their original state. In the second stage, we compared the clean view with three perturbation scenarios: randomly shuffling 50% of the features in the selected samples, and shuffling the features of all samples. For all conditions, we used the same stage-specific parameters selected for the clean sequence.

InputSession 1 ACC (%)Session 2 ACC (%)Session 3 ACC (%)
Clean87.5088.4790.45
50% of samples shuffled87.5070.8388.13
All samples shuffled87.5056.1679.59

We will report these results and qualify the structural-stability claim to reflect both the sensitivity to corrupted views and the partial recovery observed on Synthetic3d. As can be seen from the table, even when there is severe noise interference, the proposed algorithm still maintains a relatively stable clustering structure and can effectively mitigate errors from previous sessions when clean views are subsequently input.

Although the theoretical corrections (MM updates) and the newly added noise-robustness experiment are appreciated, the new experiment actually reveals a significant performance degradation under corrupted views, which contradicts the structural stability claimed in the paper. Therefore, I maintain my original score.

We appreciate the reviewers’ feedback, but we would like to point out that when the sample order is completely reversed, Session 2 can be considered to introduce an unrelated noise view. In such a scenario, it is difficult for any algorithm to extract information that supports the clustering structure, and a large amount of noise can severely disrupt the existing clustering structure. The proposed algorithm is able to maintain relative stability in the clustering structure, which is not inconsistent with the experimental results.

Reviewer_ysGPOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates CMVC under a cluster-level memory setting, where historical features are unavailable and only soft clustering assignments are retained. To address cross-session information transfer and structural stability, the authors propose SLT-CMVC, which introduces a PMM to learn consistency between current and historical clustering states. The learned representations are further modeled through a third-order tensor with a LBR spectral regularizer to preserve dominant clustering structures while reducing redundant components. The framework demonstrates superior performance across multiple datasets.

Critical strengths

In my opinion, the strengths of this paper are as follows: 1.This paper introduces a practical cluster-level memory setting, where historical features are discarded and only soft clustering states are retained. This setting is more challenging than conventional CMVC and requires effective knowledge transfer with limited historical information. 2.The authors conduct extensive experiments on seven benchmark datasets with various analyses, including ablation studies, parameter sensitivity, runtime comparisons, and view-order evaluations. The results show competitive performance, and the ablation studies demonstrate the contributions of PMM and tensor-based structural correction.

Critical weaknesses
  1. The theoretical discussion of the LBR regularization could be improved, especially regarding its mathematical properties and optimization behavior. Clarifying these aspects would make the contribution more rigorous.
  2. The main formula imposes the constraint 0<uik<10<u_{ik}<1, while the optimization derivation imposes the constraint ∑kuik=1\sum_k u_{ik}=1. Please clarify and standardize the constraints on soft assignment variables throughout the text; this will improve the readability of the paper.
Questions for authors

1.Could the authors provide more explanation on the motivation and regularization behavior of LBR, especially how it preserves dominant singular components while suppressing redundant structures? 2.Since the paper states that each experiment is independently repeated 50 times, could the authors provide standard deviations (at least for the main results) and comment on whether the improvements over the strongest baselines remain consistent across runs? This would help determine how robust the reported gains are.

2 comments

Q1: Thank you for asking us to clarify the motivation and behavior of LBR. We designed the LBR spectral penalty to reduce the shrinkage of dominant singular components in the soft-membership tensor while allowing weaker components to be suppressed. A constant shrinkage threshold applies the same reduction to every retained singular value. LBR makes this reduction depend on the magnitude of each component.

For a singular value xx, the scalar penalty is fε(x)=ln⁡(2−e−x/ε)f_\varepsilon(x)=\ln(2-e^{-x/\varepsilon}). Its derivative is

fε′(x)=1ε(2ex/ε−1).f_\varepsilon'(x)=\frac{1}{\varepsilon(2e^{x/\varepsilon}-1)}.

The derivative starts at 1/ε1/\varepsilon at zero and decreases to zero as xx grows. In the MM update, we evaluate this derivative at the previous inner iterate ηiter\eta^{\mathrm{iter}}. For an input singular value σ\sigma, the update is

\sigma-\frac{f_\varepsilon'(\eta^{\mathrm{iter}})}{\mu} \right\\}.$$ A weak component with a small current estimate receives a larger threshold and is set to zero when its input singular value falls below that threshold. A dominant component with a large estimate receives a smaller threshold, so its magnitude is better preserved. We will add this explanation to the manuscript and describe LBR as a nonconvex spectral penalty. Q2: Thank you for your suggestion. We have provided the experimental results.
DatasetMetricDECMLKMUFCERUCFCMCEHMACMVCINMKCCDIMVCLAIMVCASIAMVCOurs
3SourcesACC64.29 ± 4.6750.41 ± 2.7862.72 ± 0.9759.18 ± 0.0035.53 ± 1.7858.79 ± 6.5455.03 ± 2.4953.25 ± 0.0047.34 ± 3.4366.27 ± 1.3273.96 ± 0.32
3SourcesNMI62.70 ± 3.2327.82 ± 2.1449.46 ± 1.2635.94 ± 0.0020.15 ± 0.7956.36 ± 7.5948.80 ± 1.7641.07 ± 0.0027.90 ± 4.1252.02 ± 0.7164.85 ± 0.17
3SourcesPurity70.61 ± 2.5752.13 ± 3.2762.72 ± 0.7860.79 ± 0.0041.12 ± 1.2762.54 ± 6.7368.91 ± 1.2260.95 ± 0.0056.80 ± 2.7472.78 ± 1.5475.15 ± 1.21
MSRCACC69.75 ± 5.6162.34 ± 0.7972.86 ± 2.5373.33 ± 0.0070.21 ± 3.7677.89 ± 5.8740.90 ± 3.2768.10 ± 0.0040.95 ± 2.1970.17 ± 1.2684.76 ± 1.25
MSRCNMI52.68 ± 3.3957.71 ± 1.1268.73 ± 3.0455.11 ± 0.0050.98 ± 2.8775.16 ± 6.2931.06 ± 2.1450.18 ± 0.0024.20 ± 1.3660.39 ± 0.4176.18 ± 0.73
MSRCPurity70.51 ± 5.2663.81 ± 0.5774.13 ± 2.9776.29 ± 0.0075.29 ± 3.4580.00 ± 6.3443.33 ± 3.4670.18 ± 0.0040.95 ± 2.1970.17 ± 1.2685.76 ± 1.31
Synthetic3dACC79.68 ± 3.5188.17 ± 6.1780.17 ± 2.7468.12 ± 0.0084.33 ± 0.3788.98 ± 2.7590.12 ± 0.3492.00 ± 0.0087.33 ± 4.2780.72 ± 3.1294.67 ± 0.54
Synthetic3dNMI65.92 ± 1.7466.98 ± 4.5966.13 ± 1.5033.29 ± 0.0061.69 ± 1.2367.78 ± 1.9670.59 ± 1.1979.91 ± 0.0063.36 ± 3.5266.36 ± 1.3780.31 ± 0.32
Synthetic3dPurity79.68 ± 3.5188.17 ± 6.1781.12 ± 2.1171.34 ± 0.0085.42 ± 3.5788.98 ± 2.7590.15 ± 0.9392.10 ± 0.0087.33 ± 4.2780.94 ± 3.1094.67 ± 0.54
WikiACC43.73 ± 1.2754.01 ± 1.0350.91 ± 2.0229.18 ± 0.0053.21 ± 0.7255.42 ± 1.4441.17 ± 0.0744.28 ± 0.0057.15 ± 3.2456.02 ± 0.2761.51 ± 0.21
WikiNMI25.37 ± 0.8453.45 ± 0.9149.13 ± 1.9718.55 ± 0.0039.90 ± 0.2151.29 ± 0.9926.22 ± 0.0127.81 ± 0.0054.71 ± 2.1050.12 ± 0.1856.26 ± 0.14
WikiPurity45.18 ± 0.9461.74 ± 1.5459.18 ± 1.2430.18 ± 0.0055.21 ± 0.6459.83 ± 1.0145.01 ± 0.1448.19 ± 0.0060.46 ± 3.9158.67 ± 0.3462.53 ± 0.19
Scene15ACC20.51 ± 0.1228.18 ± 0.2730.15 ± 0.3724.01 ± 0.0032.74 ± 0.1529.65 ± 0.5632.54 ± 0.0733.73 ± 0.0026.27 ± 0.3730.14 ± 0.0034.14 ± 0.11
Scene15NMI13.99 ± 0.0927.52 ± 0.3629.52 ± 0.1120.32 ± 0.0019.33 ± 0.1018.92 ± 0.3433.25 ± 0.0431.59 ± 0.0018.37 ± 0.1232.01 ± 0.0135.62 ± 0.32
Scene15Purity24.32 ± 0.1131.79 ± 0.1831.55 ± 0.2929.81 ± 0.0036.10 ± 0.2129.87 ± 0.4732.78 ± 0.1236.97 ± 0.0029.36 ± 0.2932.59 ± 0.0038.93 ± 0.21
FashionACC48.38 ± 1.0051.74 ± 2.1158.15 ± 0.1162.31 ± 0.0058.21 ± 0.01--75.59 ± 4.5472.31 ± 0.0060.00 ± 0.1059.31 ± 2.1974.51 ± 0.00
FashionNMI46.52 ± 0.6751.24 ± 0.9060.32 ± 1.5455.18 ± 0.0041.04 ± 0.00--69.87 ± 2.8370.35 ± 0.0057.11 ± 0.0760.28 ± 0.6773.25 ± 0.00
FashionPurity52.11 ± 1.7960.18 ± 1.7862.81 ± 0.8962.31 ± 0.0062.12 ± 0.02--75.59 ± 4.5472.31 ± 0.0064.08 ± 0.0964.22 ± 1.5475.13 ± 0.00
ReutersACC30.98 ± 0.1335.72 ± 0.0050.79 ± 2.3134.78 ± 0.00----34.37 ± 0.27--40.96 ± 1.59--55.96 ± 0.00
ReutersNMI18.97 ± 0.3829.93 ± 0.0034.47 ± 1.7426.32 ± 0.00----15.65 ± 0.11--29.98 ± 1.07--37.81 ± 0.00
ReutersPurity32.38 ± 0.1239.14 ± 0.0059.84 ± 2.9139.98 ± 0.00----41.40 ± 0.68--50.65 ± 2.34--60.74 ± 0.00

Comments on the paper

2 comments
~Louis_Dubois1OpenReview ↗

Additional Fundamental Concerns: Permutation Invariance, LBR Validity, and View-Order Consistency

I raise three technical concerns that are not resolved by the existing reviews. They concern the semantics of cross-session consistency, the mathematical status of the proposed LBR functional, and the comparability of the newly reported view-order experiment.

1. Cross-session consistency is not invariant to cluster-label permutations (Eq. 2).

A clustering partition is unchanged by relabeling its clusters. Yet the proposed term

J(Ut,Ut−1,Q)=∑i,j(qi⊤qj)∥uit−ujt−1∥22J(U^t,U^{t-1},Q)=\sum_{i,j}(q_i^\top q_j)\|u_i^t-u_j^{t-1}\|_2^2

is not invariant to a permutation of the current cluster indices relative to the historical ones.

This can be demonstrated entirely within the stated open probability simplex. Let 0<δ<1/20<\delta<1/2 and set

U=\begin{bmatrix}\delta&1-\delta\\1-\delta&\delta\end{bmatrix}.$$ $H$ and $U$ encode the same soft assignment structure up to exchanging two cluster labels. Set $U^{t-1}=H$ and $U^t=U$. For each $i$, $\|u_i^t-u_i^{t-1}\|_2^2=2(1-2\delta)^2$; since $q_i$ is a two-class probability vector, $\|q_i\|_2^2\geq 1/2$. All summands are nonnegative. Therefore, for every admissible $Q$, $$J(U,H,Q)\geq 2(1-2\delta)^2>0.$$ By contrast, for the unpermuted assignment $U^t=H$, choosing $q_1$ and $q_2$ arbitrarily close to distinct one-hot probability vectors makes $J(H,H,Q)$ arbitrarily close to zero. Thus, equivalent partitions can incur fundamentally different consistency costs solely because of cluster numbering, even after optimizing the auxiliary memberships. The issue is not that the model must be permutation-invariant in all parameters, but that the claimed *structural consistency* requires a demonstrated mechanism for maintaining cluster-index correspondence between sessions. Where is this correspondence enforced or proved? In particular, what happens when a new view's soft cluster assignments are column-permuted while representing exactly the same partition? An explicit alignment rule, an invariant reformulation, or a justified initialization guarantee is needed to support the interpretation of Eq. (2). **2. LBR is not a norm as defined (Definition 1, Eq. 3).** The singular-value penalty is $f_\varepsilon(x)=\ln(2-e^{-x/\varepsilon})$, and the paper itself notes that $f_\varepsilon(x)\to\ln 2$ as $x\to\infty$. Consequently, the resulting spectral functional remains bounded along the ray $\alpha\mathcal U$ for any fixed nonzero finite tensor $\mathcal U$, as $\alpha\to\infty$. This contradicts the positive homogeneity required of every norm, $\|\alpha\mathcal U\|=|\alpha|\|\mathcal U\|$. Numerically, at the paper's $\varepsilon=2$, $f_2(2)\approx0.489880\neq2f_2(1)\approx0.663593$. LBR can still be a valid **nonconvex spectral regularizer**. However, naming it a new norm is mathematically incorrect. The paper should correct Definition 1 and clearly identify what properties are proved for this particular penalty, rather than conflating bounded spectral regularization with a norm. **3. The new view-order experiment does not yet establish reproducibility of the main result.** In the November 12 response, the six permutations of Synthetic3d give ACCs of 90.45, 92.18, 94.31, 91.52, 90.12, and 89.97%, respectively (mean 91.425%; maximum 94.31%). Table 1, however, reports 94.67%, which Table 7 also attributes to the high-to-low order. None of the six newly reported permutation means matches this figure. This is **not** by itself evidence of incorrect results: the original and additional experiments may have used different seeds or other settings. It does mean that their relationship needs to be specified. Which permutation yielded 94.67%, and were the initialization, hyperparameters, update equations, stopping rule, and random-seed protocol identical in the two experiments? How was the high-to-low ordering determined without ground-truth cluster labels? Providing the precise protocol would make the view-order robustness claim interpretable and reproducible. **Conclusion.** These are additional questions about the definition of cross-session structure, the validity of the proposed mathematical object, and the consistency of the experimental protocol. The first two follow from explicit counterexamples; the third requests a reconciliation, not a presumption of misconduct. They should be addressed directly because they concern the paper's stated contributions rather than ancillary implementation details.

Response Regarding Cross-Session Cluster Correspondences

Thank you for providing the counterexample.

We resolve this ambiguity through initialization. For each session t>1t > 1, our code sets Ut,0=Ut−1U^{t,0} = U^{t-1} and initializes the cluster centers using the features from the current view: vkt,0=∑i=1nuikt−1xit∑i=1nuikt−1.v_k^{t,0} = \frac{\sum_{i=1}^{n}u_{ik}^{t-1}x_i^t} {\sum_{i=1}^{n}u_{ik}^{t-1}}.

Thus, the kkth column of the current membership matrix and its corresponding cluster center are inherited from cluster index kk of the previous session. This avoids introducing independent cluster numbers upon receiving a new view. Subsequent optimizations can update membership weights and centers based on the inherited column order, utilizing the new view.

We will explicitly describe this initialization process in Algorithm 1 and clarify that Equation (2) measures the consistency within these shared coordinate systems. This initialization establishes the initial column mappings.

Response on the mathematical status of LBR

Thank you for pointing this out. We agree that the functional in Definition 1 is a nonconvex spectral penalty. For any fixed nonzero tensor U\mathcal U, the singular values of its Fourier slices scale linearly under multiplication by α>0\alpha>0. Since fε(x)=ln⁡(2−e−x/ε)f_\varepsilon(x)=\ln(2-e^{-x/\varepsilon}) is bounded above by ln⁡2\ln 2, the spectral penalty remains bounded as α→∞\alpha\to\infty. Positive homogeneity would instead require RLBR(αU)=αRLBR(U)\mathcal R_{\mathrm{LBR}}(\alpha\mathcal U)=\alpha\mathcal R_{\mathrm{LBR}}(\mathcal U), which grows without bound. This confirms the reviewer's counterexample.

We will rename the functional the LBR spectral penalty, use the notation RLBR(U)\mathcal R_{\mathrm{LBR}}(\mathcal U) in Eq. (3), and clarify the properties used in the optimization. The scalar penalty vanishes at zero, is strictly increasing and concave on the nonnegative axis, and saturates at ln⁡2\ln 2. Its derivative is

fε′(x)=1ε(2ex/ε−1).f_\varepsilon'(x)=\frac{1}{\varepsilon(2e^{x/\varepsilon}-1)}.

The singular-value update uses the tangent upper bound supplied by concavity. Since the derivative decreases to zero as the singular value grows, the update assigns smaller shrinkage weights to larger singular values. We will state these properties explicitly in the revised definition and derivation.

Response on the reproducibility of the view-order comparison

Thank you for pointing out the difference. The 94.67% ACC in Tables 1 and 7 was obtained for the original high-to-low view order. The six permutation means reported in our November 12 response were obtained using a different set of random seeds for clustering initialization. The two sets of results therefore average over different initializations, which can yield different ACC values. We will provide the complete form at a later date.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.