acceptodds
Under review as a conference paper at ICLR 2027

Low-Rank Smooth Unified Anchor Tensor Nonnegative Orthogonal Factorization for Multi-view Clustering

Abstract

Anchor-based tensor learning provides an efficient framework for large-scale multi-view clustering by combining compact anchor representations with high-order cross-view modeling. However, existing methods often rely on predefined anchors, separating anchor construction from subsequent tensor learning and preventing their joint optimization. Moreover, independently learned anchors in heterogeneous feature spaces may lack explicit cross-view correspondence, while conventional low-rank tensor regularization tends to overlook local smoothness among samples. To address these limitations, we propose a ow-Rank mooth nified nchor ensor onnegative rthogonal actorization () model for multi-view clustering. Specifically, heterogeneous views are projected into a common feature space, where a unified anchor matrix is adaptively learned and shared across all views. Unlike two-stage schemes, the unified anchors and tensor representations are jointly optimized, allowing the high-order structure revealed by tensor factorization to feed back into anchor learning. Tensor total variation Schatten- regularization is further incorporated to capture global low-rankness and sample-wise local smoothness within a single term. An efficient ADMM-based optimization algorithm is developed. Extensive experiments on eight benchmark datasets demonstrate that achieves competitive or superior clustering performance with favorable convergence and computational efficiency.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Multi-view clustering, Unified anchor learning, Nonnegative orthogonal tensor factorization, Tensor total variation

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_QvU3OpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a Low-Rank Smooth Unified Anchor Tensor Factorization method (LS-UATNOF) to address the reliance on predefined anchors and the lack of cross-view anchor consistency in existing methods. The proposed method projects heterogeneous multi-view data into a common feature space to learn a unified anchor matrix, and incorporates low-rank smoothness regularization to capture structural priors. Extensive comparative and ablation experiments demonstrate the effectiveness and competitiveness of LS-UATNOF.

Critical strengths

1 The method is well motivated. It jointly optimizes unified anchors and tensor factorization to enhance cross-view consistency, while integrating low-rank and smoothness priors through tensor total variation Schatten-p regularization, reducing the need for additional hyperparameter tuning. 2 The experimental results are convincing. LS-UATNOF outperforms state-of-the-art methods on multiple benchmark datasets, demonstrating strong clustering performance.

Critical weaknesses

1 The rationale behind the model design is insufficiently discussed. The authors should clarify why Schatten-p regularization is preferred over the standard tensor nuclear norm and explain the advantages of shared anchors over independently learned anchors. 2 The rationale for setting the common latent dimension to (d=min{c,min d_v}), as well as its impact on clustering performance, requires further clarification.

Questions for authors

1 Why do the authors apply regularization to the anchor representation tensor rather than the cluster-indicator tensor? 2 What are the respective roles of the nonnegative and orthogonal tensor factors, and what motivates imposing different constraints on them? 3 Is there a consistent pattern in the optimal values of p across different datasets? Could the authors offer recommendations on how to choose p in practice?

4 comments
AuthorsOpenReview ↗

Response to Reviewer QvU3 (Part 1/2)

We are grateful to the reviewer for recognizing the motivation of our approach and its clustering performance. We also welcome the questions about the regularization design, latent dimension, and tensor-factor constraints, as well as the request for practical guidance on choosing pp. Our point-by-point responses below explain these design choices and discuss the relevant experimental findings.

Weakness 1: Rationale for the model design.

We thank the reviewer for encouraging us to clarify our motivation. For 0<p<10 < p < 1, Schatten-pp regularization applies the nonconvex mapping σp\sigma^p to each singular value σ\sigma. Its slope pσp−1p\sigma^{p-1} decreases as σ\sigma increases, placing stronger shrinkage on small singular values while reducing the bias on large ones compared with the nuclear norm. This helps suppress weak components while preserving dominant structures; p=1p=1 recovers the nuclear-norm case.

Our motivation for unified anchors goes beyond cross-view consistency: they provide aligned anchor indices for subsequent tensor factorization. When anchors are learned independently, the same row index in different views' anchor representations may refer to different anchors. Stacking these representations can therefore mix anchor mismatch with meaningful cross-view variation. Sharing A\mathbf A establishes a common anchor reference, allowing tensor factorization to capture cross-view relationships while retaining view-specific representation coefficients.

Weakness 2: Choice of the common latent dimension.

We appreciate the reviewer's question about this setting. The number of clusters cc provides a task-related reference for the latent dimension, yielding a compact representation whose size scales with the clustering task. Using it as the default also avoids introducing an additional dimension-search procedure. Since row orthogonality of P(v)\mathbf P^{(v)} requires d≤dvd\leq d_v for every view, we set d=min⁡c,min⁡vdvd=\min\\{c,\min_v d_v\\}. This provides a practical, task-informed choice while ensuring feasibility across views, rather than assuming a universally optimal dimension.

We further examine the effect of dd on BDGP and CCV, fixing m=10m=10 and m=20m=20, respectively, and retaining all other settings except the two regularization parameters. For each dimension, we independently search the same 49 combinations of λ1\lambda_1 and λ2\lambda_2 and report the results of the combination with the highest ACC.

dd345 (default)6810
ACC (%)99.7299.5699.5299.6099.5699.72

ACC on BDGP; the best results are shown in bold.

dd101520 (default)
ACC (%)86.2786.9587.18

ACC on CCV; the best result is shown in bold.

On BDGP, ACC ranges from 99.5299.52\\% to 99.7299.72\\%, a difference of only 0.20 percentage points; the default d=5d=5 thus remains competitive across the tested dimensions. On CCV, ACC increases from 86.2786.27\\% at d=10d=10 to 87.1887.18\\% at the default d=20d=20, which achieves the best result among the tested values. Together, these results support the proposed dimension rule as a practical default, providing strong performance on these two datasets without an additional dataset-specific dimension search.

AuthorsOpenReview ↗

Response to Reviewer QvU3 (Part 2/2)

Question 1: Regularizing the anchor representation tensor.

Thank you for this question. We regularize H\mathcal H because it directly represents sample-anchor relationships learned from the observed data. Reconstruction alone may retain noise and variations unrelated to clustering; TTV--Schatten-pp regularization encourages low-rank, smooth variations in these representations and guides the subsequent tensor factorization. By contrast, Z\mathcal Z is the compact clustering representation, whose orthogonality constraint reduces redundancy among its components. These mechanisms serve complementary purposes: improving the structure of the anchor representations and separating components in the final clustering representation.

Question 2: Different constraints on the tensor factors.

We appreciate this question. Nonnegativity on F\mathcal F provides a nonnegative basis for expressing anchor-space patterns, whereas orthogonality on Z\mathcal Z reduces redundancy among the components used for clustering. The constraints therefore reflect the different roles of the two factors. Applying them separately also simplifies optimization: F\mathcal F is updated through nonnegative projection, while Z\mathcal Z is updated through slice-wise SVD in the Fourier domain, without requiring both constraints to be satisfied by the same factor.

Question 3: Selection of the Schatten parameter pp.

Thank you for asking about practical parameter selection. The table below summarizes the ACC results underlying Figure 5 in Appendix D, with all other parameters fixed and only pp varied. The preferred value depends on the dataset, and performance does not improve monotonically with pp. Nevertheless, p=1p=1 matches the best ACC on HW and BDGP and is within 0.05 percentage points of the best on CCV and Animal. We therefore recommend p=1p=1 as a practical starting point rather than a universally optimal choice. It recovers the tensor nuclear-norm penalty on the directional gradient tensor while retaining the smoothness prior; smaller values can be explored when additional tuning is affordable.

ppHWBDGPCCVAnimal
0.199.7599.5239.7095.76
0.299.8599.5273.7997.20
0.399.8099.5287.0890.93
0.499.7099.5286.5194.79
0.599.7099.5286.3194.78
0.699.5599.5279.4688.85
0.799.6599.5285.0497.16
0.899.9099.5287.1797.15
0.9100.0099.5287.2397.15
1.0100.0099.5287.1897.16

ACC (%); the best result in each dataset is shown in bold.

Reviewer_QvU3OpenReview ↗

15579

The authors have addressed my concerns properly and I prefer to increase the rating score.

AuthorsOpenReview ↗

Reply to Reviewer QvU3

Thank you for your positive feedback and for increasing your rating. We are pleased that our revisions and responses have addressed your concerns. We will incorporate the relevant revisions and clarifications into the final version of the paper.

Reviewer_P3XtOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

To address the lack of consistent anchors across views in multi-view clustering, this paper proposes LS-UATNOF by integrating unified anchor graph learning with tensor factorization. The model jointly learns shared anchors to promote cross-view consistency and incorporates a tensor total variation regularizer to capture both low-rankness and smoothness within a joint optimization framework. Experimental results demonstrate the effectiveness of the proposed method.

Critical strengths
  1. The model is coherently designed, jointly optimizing unified anchor learning and tensor factorization to enable effective exchange of structural information across views.

  2. The experiments are reasonably comprehensive and demonstrate competitive clustering performance. The ablation results also highlight the contributions of the individual modules.

Critical weaknesses

1.The theoretical analysis could be strengthened by justifying the choice of the sample dimension for total variation regularization and establishing convergence guarantees for the nonconvex multi-block optimization algorithm.

  1. The paper lacks a comparison between fixed anchors and jointly learned anchors to validate the benefits of adaptive anchor learning.
Questions for authors
  1. How does the proposed method balance cross-view consistency and view-specific complementary information when learning shared anchors?
  2. What advantages does the proposed method offer in computational efficiency? These are not apparent from the reported running times.
  3. The reconstruction error is penalized using the 2,1 norm, whereas the tensor factorization error uses the squared Frobenius norm. What noise assumptions underlie these two choices?
3 comments
AuthorsOpenReview ↗

Response to Reviewer P3Xt

We appreciate the reviewer's careful evaluation and recognition of the coherent model design and comprehensive experiments. The comments on theoretical justification, adaptive anchor learning, computational efficiency, and error modeling help us identify where further explanation and evidence are needed. Below, we respond to the individual concerns and present an additional comparison with fixed anchors.

Weakness 1: Theoretical justification and convergence.

Compared with existing tensor methods that perform the FFT along the sample dimension [1,2], tensor total-variation regularization first computes finite differences along this dimension and then applies an FFT-based tensor nuclear-norm penalty to the resulting gradient tensor. This formulation promotes local smoothness among samples, thereby preserving cross-view consistency and within-view smoothness.

Regarding convergence, we agree that a formal guarantee would strengthen the analysis. Our current evidence is empirical: the constraint residuals decrease and the clustering performance stabilizes in the reported experiments. These observations do not establish convergence to a stationary point. Since our algorithm involves multiple variable blocks and approximate updates, standard ADMM convergence results cannot be directly applied. We acknowledge this limitation and leave a rigorous analysis of the specific update scheme for future work.

[1] Y. Xie et al., “On unifying multi-view self-representations for clustering by tensor multi-rank minimization,” IJCV, 2018.

[2] Q. Gao et al., “Tensor-SVD based graph learning for multi-view subspace clustering,” in Proc. AAAI, 2020.

Weakness 2: Fixed versus jointly learned anchors.

We thank the reviewer for suggesting this comparison. We added a fixed-anchor variant, where k-means centers in a common latent space are projected to satisfy AAT=Id\mathbf A\mathbf A^{\mathrm T}=\mathbf I_d. The resulting A\mathbf A remains fixed while the other variables are updated. We tune λ1\lambda_1 and λ2\lambda_2 and report the best results below.

DatasetFixed anchorsLS-UATNOF
MSRC100.00100.00
NGs98.8099.80
HW99.75100.00
BDGP97.1299.52
CCV80.0787.18
Animal90.7797.16

ACC (%); the best result in each row is shown in bold.

Fixed anchors match our method on MSRC, whereas jointly learned anchors yield higher ACC on the other five datasets. The larger gains on CCV and Animal support allowing anchors to adapt during representation learning rather than keeping them fixed throughout optimization.

Question 1: Balancing consistency and complementarity.

We appreciate the reviewer’s interest in this balance. The shared dictionary A\mathbf A aligns anchor indices across views, providing a consistent basis for subsequent tensor factorization. Meanwhile, each view retains its own projection P(v)\mathbf P^{(v)} and anchor representation H(v)\mathbf H^{(v)}, allowing heterogeneous features and complementary relationships to be represented differently. Thus, sharing anchors establishes a common reference without forcing identical representations across views.

Question 2: Computational efficiency.

We thank the reviewer for this question. Our efficiency advantage is particularly evident relative to self-representation methods such as t-SVD-MSC and WTNNM. By using m×nm\times n anchor representations instead of n×nn\times n self-representation matrices, our method reduces computational and memory costs. For example, on CCV, LS-UATNOF takes 14.96 seconds, compared with 2547.91 and 3068.47 seconds for t-SVD-MSC and WTNNM, respectively. It also handles NUSWIDEOBJ and NoisyMNIST, where both methods encounter out-of-memory issues under our experimental setting. Although not the fastest among anchor-based methods, LS-UATNOF offers improved scalability over these self-representation baselines.

Question 3: Different error penalties.

Thank you for asking about these choices. Compared with the ℓ1\ell_1 norm, which penalizes individual entries, the ℓ2,1\ell_{2,1} norm treats each sample's error vector as a group. This makes it better suited to handling corrupted samples whose errors affect multiple features. The squared Frobenius norm on E\mathcal E instead measures the overall tensor-factorization error without requiring sparsity, since approximation errors may occur throughout the learned representation. It also makes the corresponding update straightforward to compute.

Thank you for your response and the additional experiments. I have read the response, and it addresses my main concerns. I will keep my score.

AuthorsOpenReview ↗

Reply to Reviewer P3Xt

Thank you for your positive feedback and for taking the time to review our response and additional experiments. We sincerely appreciate your recognition that our revisions have addressed your main concerns, as well as your constructive comments throughout the review process.

Reviewer_DQb2OpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper employs nonnegative orthogonal tensor factorization to learn clustering representations and introduces TTV-Schatten-p regularization to jointly model global low-rank structure and local smoothness. Experiments on eight multi-view datasets demonstrate strong clustering performance. Ablation studies, parameter sensitivity analyses, and convergence experiments further support the effectiveness of the model.

Critical strengths
  1. The joint optimization of unified anchor learning and nonnegative orthogonal tensor factorization offers a degree of novelty, with shared anchors enhancing cross-view consistency.
  2. Experiments on eight datasets demonstrate good clustering performance, while ablation studies and efficiency and convergence analyses further support the method’s effectiveness.
Critical weaknesses
  1. The constraint AA’=I enforces row orthogonality, whereas anchors correspond to columns. Further explanation is needed regarding how this constraint promotes anchor diversity.
  2. The influence of the number of anchors m is insufficiently analyzed. The authors should further discuss its effects on clustering performance and computational cost and provide practical guidelines for selecting m.
Questions for authors
  1. The final clustering is performed by applying k-means to the concatenated view-specific representations. Why is this strategy chosen over averaging the representations?
  2. Could using unified anchors across views, rather than view-specific anchors, hinder the exploitation of complementary information?
  3. Cross-view consistency suggests that anchor representation matrices may also exhibit local smoothness along the view dimension. What are the underlying reasons for the poor performance when (\Gamma={3})?
  4. Why are different constraints imposed on tensors (F) and (G)?
3 comments
AuthorsOpenReview ↗

Response to Reviewer DQb2

We thank the reviewer for the positive assessment of our work and the constructive comments on the orthogonality constraints, the number of anchors, and cross-view representation learning. To address these concerns, we clarify the rationale behind our model design and discuss the effects of the number of anchors and the final clustering strategy. Detailed point-by-point responses to all concerns and questions are provided below.

Weakness 1: Orthogonality constraint.

We thank the reviewer for this insightful question. For A∈Rd×m\mathbf A\in\mathbb R^{d\times m}, the constraint AAT=Id\mathbf A\mathbf A^{\mathrm T}=\mathbf I_d ensures a full-row-rank shared dictionary with decorrelated latent coordinates. It therefore maintains coverage of the dd-dimensional latent space while allowing m>dm > d anchors. The distinct roles of individual anchors are encouraged through joint reconstruction across views and the tensor-structured learning of their representation coefficients.

Weakness 2: Impact of the number of anchors mm.

We thank the reviewer for highlighting this important detail. Figure 4 in Appendix D reports the clustering performance with m∈d,2d,3d,4d,5dm\in\\{d,2d,3d,4d,5d\\}, where dd is the common projected dimension and is held fixed for each dataset. The results indicate that increasing mm does not consistently improve performance, and the preferred anchor number varies across datasets. A larger mm provides greater representation capacity but also increases computational and storage costs. Therefore, a moderate anchor number offers a practical balance between clustering performance and efficiency.

Question 1: Concatenation versus averaging for clustering.

We thank the reviewer for this question. Given the learned tensor Z∈Rn×c×V\mathcal Z\in\mathbb R^{n\times c\times V}, we concatenate its frontal slices to form

R=[Z(1),…,Z(V)]∈Rn×cV.\mathbf R=[\mathcal Z^{(1)},\ldots,\mathcal Z^{(V)}] \in\mathbb R^{n\times cV}.

Each row is then ℓ2\ell_2-normalized before applying kk-means with cc clusters. Concatenation retains the information encoded in each slice, whereas averaging compresses these representations into a single n×cn\times c matrix and may obscure complementary information. This strategy therefore uses all slices directly for clustering without constructing an additional sample-level affinity matrix.

Question 2: Unified anchors and cross-view complementarity.

We appreciate the reviewer's concern about preserving complementary information. Sharing A\mathbf A provides consistent anchor correspondence across views, enabling tensor factorization to model cross-view relationships along aligned anchor indices. This does not require identical view-specific representations: the projections P(v)\mathbf P^{(v)} and anchor representations H(v)\mathbf H^{(v)} remain view-dependent, while the slices Z(v)\mathbf Z^{(v)} retain view-specific information for final clustering. Thus, consistency is imposed on the anchor dictionary, while complementarity is preserved through the learned representations.

Question 3: Smoothness along the view dimension.

We appreciate this insightful question. Cross-view consistency does not require identical representations. View-wise differences,

(∇3H)(v)=H(v+1)−H(v),(\nabla_3\mathcal H)^{(v)} =\mathbf H^{(v+1)}-\mathbf H^{(v)},

with periodic indexing, may encode useful complementary information. Regularizing these differences can therefore be overly restrictive, especially since heterogeneous views lack a natural ordering. The results from Table 5 below show that view-wise smoothness alone performs poorly, and adding it to the sample-wise prior also reduces ACC.

DatasetΓ=3\Gamma=\\{3\\}Γ=2,3\Gamma=\\{2,3\\}Γ=2\Gamma=\\{2\\}
HW86.199.9100.0
BDGP53.293.499.5
CCV21.783.687.2
Animal16.382.497.2
NUSWIDEOBJ13.522.327.4
NoisyMNIST39.099.799.8

ACC (%); higher is better.

Question 4: Different constraints on the tensor factors.

We thank the reviewer for pointing this out. The two factors serve different roles: nonnegativity on F\mathcal F yields an additive anchor-space basis, while tensor orthogonality on Z\mathcal Z (referred to as G\mathcal G in the question) reduces redundancy among the clustering components. These complementary constraints also simplify optimization: nonnegativity can be handled by projection, and orthogonality by slice-wise SVD in the Fourier domain, without imposing both constraints on the same factor.

My concerns have basically been addressed, and I maintain my original rating.

AuthorsOpenReview ↗

Reply to Reviewer DQb2

We are grateful for your careful reconsideration of our work and for acknowledging that our responses have largely resolved your concerns. Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.