acceptodds
Under review as a conference paper at ICLR 2027

AM-GMAE: Adaptive Manifold Graph Masked Autoencoder for Multi-view Remote Sensing Data Clustering

Abstract

Multi-view clustering has emerged as a mainstream unsupervised paradigm for analyzing remote sensing data by jointly exploiting complementary multi-sensor observations. Graph Masked Autoencoders (GMAEs), which learn informative graph representations through self-supervised reconstruction, offer a promising solution for these multi-view scenarios. Yet, effectively extending GMAEs to heterogeneous multi-view remote sensing data remains challenging for three main reasons: i) existing masking strategies ignore node importance, risking trivial cross-view reconstruction shortcuts; ii) initial similarity-based graphs suffer from noise and class ambiguity, distorting the underlying manifold during message passing; and iii) forced alignment of heterogeneous latent spaces often induces negative transfer. To address these issues, we propose the Adaptive Manifold Graph Masked Autoencoder (AM-GMAE). Specifically, AM-GMAE introduces a Clustering-Saliency Adversarial Mask module to identify and adaptively mask nodes with high clustering-uncertainty sensitivity across correlated views, thereby increasing reconstruction difficulty and reducing reliance on trivial cross-view cues. Furthermore, we develop a Dynamic Graph Purification and Topology Optimization module to prune misleading edges and refine the graph manifold using structural information-theoretic constraints. Finally, a Multi-marginal Optimal Transport module with Wasserstein barycenter estimation is employed to align heterogeneous representations through a shared Wasserstein semantic geometry without imposing rigid pairwise coupling. Extensive experiments on four widely used remote sensing benchmarks demonstrate that AM-GMAE outperforms SOTA methods.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Multi-view clustering, graph learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_2scEOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes a novel framework for trustworthy multi-modal clustering. It integrates information bottleneck theory with evidential deep learning, which represents a fresh and reasonable direction that has not been fully explored before. The experiments are sufficient and cover multiple public multi-modal datasets. The evaluation is conducted on several benchmark datasets and compared against a broad set of recent baselines, which lends credibility to the reported results.

Critical strengths
  1. The paper addresses a relevant problem in unsupervised Earth observation. Unreliable neighborhood relationships and differences across sensing modalities are practical challenges, and the proposed framework tackles both through a unified training objective.
  2. The evaluation covers four scenes and includes comparisons with the closely related GMAE baseline, component ablations, and controlled experiments examining the effects of data corruption and geometry. The paper also reports using ten random seeds and paired initialization for the alignment comparisons.
  3. The paper is well motivated, with the main challenges clearly linked to the proposed components. The experimental figures are easy to follow, and the qualitative clustering maps help illustrate differences in spatial coherence and boundary preservation across methods.
Critical weaknesses
  1. The parameter range stated in Section 3.2 is inconsistent with the experiments in Figure 10. Section 3.2 requires γ≥1\gamma \geq 1, whereas Figure 10 includes γ=0\gamma = 0, 0.10.1, and 0.50.5, without explaining why values outside this range are tested. In particular, γ=0\gamma = 0 makes the reconstruction loss constant wherever the power is defined, so it no longer provides a useful reconstruction signal for training.
  2. The role of the graph learned by DGPO in the overall training pipeline is not clearly explained, particularly its connection to the graph encoder.
  3. Appendix A.3 lists many dataset-specific parameter settings but does not explain how they were selected. It is also unclear whether the baselines used comparable input features and tuning budgets.
  4. The initial and purified graphs on Salinas in Figures 3(a) and (b) look fairly similar. Although purification reduces some off-diagonal connections, the improvement is less evident than the description in Section 4.5 suggests, particularly regarding the formation of a more cohesive block-diagonal structure and the suppression of misleading edges.
  5. Some references in the Appendix are not properly rendered. For example, “Figure ??” appears in Appendix C.2, in the subsection titled “Sensitivity to the entropy regularization coefficient ϵ\epsilon.” The citation below Eq. (102) in the proof of Theorem 8 also appears as “?”.
Questions for authors
  1. Could the authors clarify the intended range of gamma and explain why Figure 10 includes γ\gamma = 0, 0.1, and 0.5 despite the restriction γ\gamma ≥ 1 in Section 3.2? How is the reconstruction loss handled at γ\gamma = 0, and does this setting effectively remove its contribution to model training?
  2. Could the authors explain how the graph learned by DGPO is used during training, particularly its connection to the graph encoder? Is the updated graph used for message passing in later training iterations, or does it affect training through the structural and topology-consistent reconstruction losses?
  3. Please describe how the dataset-specific parameter settings in Appendix A.3 were selected? Please also specify the input features and preprocessing used for the baselines, and explain how their parameter-tuning budgets compare with that of AM-GMAE.
  4. In Figures 3(a) and (b), the most visible change is a reduction in some off-diagonal connections, while the overall block structure appears similar. Could the authors explain how this comparison supports the claims in Section 4.5 about a more cohesive block-diagonal structure and the suppression of misleading edges? A clearer visualization highlighting the relevant changes would be helpful.
7 comments

We sincerely thank you for the constructive feedback and insightful questions. We address your specific concerns in detail below.

W1 & Q1: Thank you for pointing out this inconsistency and giving us the opportunity to clarify the range of γ\gamma values. In Section 3.2, the condition γ≥1\gamma \ge 1 defines the standard operating range of the scaled cosine reconstruction loss during regular training. In all main experiments, we set γ=1\gamma = 1. In contrast, Figure 10 in Appendix C.2 is intended to provide a broader sensitivity analysis; therefore, we initially tested values between 00 and 11 to observe how clustering performance changes as the reconstruction penalty is weakened. To avoid misunderstanding and fully cover the case where γ≥1\gamma \ge 1, we have expanded Figure 10 in the revised manuscript to include γ∈{0.0,0.1,0.5,1.0,2.0,3.0,5.0}\gamma \in \{0.0, 0.1, 0.5, 1.0, 2.0, 3.0, 5.0\}.

When γ=0\gamma = 0, the 00-th power of the cosine reconstruction error becomes the constant 11, which results in a zero gradient, thereby rendering LReconL_{\mathrm{Recon}} completely ineffective during training. We treat γ=0\gamma = 0 as a boundary case in our ablation experiments, aiming to evaluate the model’s performance when “direct masked attribute reconstruction” is disabled. Under this setting, the model does not collapse because the remaining parts of the objective function in Equation (13) still provide gradients for updating the encoder and decoder. However, as shown in Figure 10, removing LReconL_{\mathrm{Recon}} when γ=0\gamma = 0 leads to a decrease in the model’s accuracy across all datasets; this confirms that introducing an effective reconstruction loss is essential for achieving optimal clustering performance.

W2 & Q2: Thank you for asking us to clarify how the purified graph A~(v)\tilde{\mathcal{A}}^{(v)} learned by DGPO is used during training. We have updated Appendix A.4 in the revised manuscript to present the complete training flow in detail. In each training epoch, the forward pass of the graph encoder always uses the normalized initial kk-NN graph A~(v)\tilde{\mathbf{A}}^{(v)} together with the CSAM-masked node features to compute the view-specific latent representations H(v)\mathbf{H}^{(v)}, and the decoder maps H(v)\mathbf{H}^{(v)} to the reconstructed node features Z^(v)\hat{\mathbf{Z}}^{(v)}. Next, DGPO takes the encoder outputs H(v)\mathbf{H}^{(v)} to dynamically construct the purified adjacency matrix A~(v)\tilde{\mathcal{A}}^{(v)} via differentiable thresholding in Eq. (6).

The updated graph A~(v)\tilde{\mathcal{A}}^{(v)} is not used to replace A~(v)\tilde{\mathbf{A}}^{(v)} for forward message passing in subsequent training iterations, which prevents early-stage topological errors from destabilizing GNN propagation. Instead, A~(v)\tilde{\mathcal{A}}^{(v)} is coupled with the graph encoder during backpropagation through the structural and topology-consistent reconstruction losses. First, because A~(v)\tilde{\mathcal{A}}^{(v)} is computed directly from the encoder representations H(v)\mathbf{H}^{(v)} through a differentiable thresholding operator, minimizing the structural entropy loss Lstr\mathcal{L}_{\mathrm{str}} backpropagates gradients through A~(v)\tilde{\mathcal{A}}^{(v)} into H(v)\mathbf{H}^{(v)} and the encoder parameters, encouraging a compact and cluster-consistent latent manifold where cross-class edges are suppressed. Second, the topology-consistent reconstruction loss Ltopo\mathcal{L}_{\mathrm{topo}} uses A~(v)\tilde{\mathcal{A}}^{(v)} to compute random-walk-smoothed features as a clean structural target to supervise the decoded outputs Z^(v)\hat{\mathbf{Z}}^{(v)}, guiding the encoder and decoder to recover masked node attributes from reliable, class-consistent neighbors.

W3 & Q3: Thank you for asking us to detail the hyperparameter selection strategy, baseline preprocessing, and tuning fairness. For AM-GMAE, general training configurations including the learning rate (10−310^{-3}), training epochs 100100, masking ratio rm=0.5r_m = 0.5, reconstruction exponent γ=1\gamma = 1, and initial threshold τth=0.3\tau_{th} = 0.3 are fixed across all datasets. The dataset-specific parameters in Appendix A.3, namely the loss weights (α,β,λ1,λ2)(\alpha, \beta, \lambda_1, \lambda_2), temperatures (τc,τm)(\tau_c, \tau_m), cross-view saliency weight η\eta, and Sinkhorn regularization ε\varepsilon, were selected via grid search. Specifically, we searched the loss coefficients over {0.01,0.1,1,10,100}\{0.01, 0.1, 1, 10, 100\} and the remaining parameters over the candidate grids shown in the sensitivity analyses of Appendix C.2 (Figures 8 to 13), and then reported the mean performance over 10 random seeds {0,1,…,9}\{0, 1, \dots, 9\}.

For all baseline methods, we use the same multi-view input modalities, feature dimensions, and feature normalization as AM-GMAE across all four datasets, as described in Table 3 and Appendix A.1. For superpixel-based remote sensing graph baselines such as SAMVGC and CG-MAE, we provide the same SLIC superpixel segmentation, region-averaged node features, and initial kk-NN graphs used by our method. For the remaining baselines, we follow their official preprocessing and graph or anchor construction pipelines on the same normalized multi-view features.

To ensure a fair parameter-tuning budget across all methods, we evaluated each baseline using its official open-source code under a consistent protocol. For baselines whose original papers already reported experiments on the corresponding dataset, we directly adopted the optimal hyperparameters provided by the authors. For baselines that had not been evaluated on a given dataset in their original papers, we first tested their default parameter settings and then conducted a grid search over their key hyperparameters within the candidate ranges recommended by the original authors to explore better performance. Finally, using each baseline's best-performing parameter configuration, we ran 10 independent trials with the same random seeds {0,1,…,9}\{0, 1, \dots, 9\} and reported the average results in Table 1.

W4 & Q4: Thank you for this helpful suggestion regarding the visualization in Figure 3. In the original submission, Figures 3(a) and 3(b) were plotted on the Salinas dataset, which has 16 fine-grained classes and 3,500 superpixel nodes; as a result, the individual class blocks were relatively small, and the visual changes were less obvious at full-matrix resolution. In the revised manuscript (Section 4.5, page 9), we have redrawn Figures 3(a) and 3(b) on the Trento dataset and enhanced the visual contrast of the adjacency matrices. In the updated figures, the initial graph in Figure 3(a) exhibits substantial off-diagonal inter-class noise, whereas the DGPO-purified graph in Figure 3(b) shows a much cleaner and more cohesive block-diagonal structure with off-diagonal connections clearly suppressed.

Furthermore, the claims in Section 4.5 are quantitatively supported across all four datasets by the structural metrics in Figure 3(c) and the inter-class edge weight distributions in Figure 3(d) (detailed in Appendix A.3). On Trento, DGPO improves edge purity from 0.68880.6888 to 0.84020.8402, weighted homophily from 0.69990.6999 to 0.86420.8642, modularity from 0.45290.4529 to 0.75970.7597, and increases the intra/inter-class edge weight ratio from 2.42832.4283 to 5.98565.9856. Similarly, on Salinas, DGPO increases edge purity from 0.57580.5758 to 0.71920.7192, weighted homophily from 0.57940.5794 to 0.74780.7478, modularity from 0.38250.3825 to 0.63920.6392, and more than doubles the intra/inter-class weight ratio from 1.38361.3836 to 3.19323.1932. Consistently, the KDE curves in Figure 3(d) show that DGPO shifts the distribution of remaining inter-class edge weights toward lower values across all four datasets, confirming that misleading cross-class edges are effectively suppressed.

W5: Thank you for catching these rendering errors. They were caused by typographical mistakes in the LaTeX labels and BibTeX keys during compilation, and we have corrected all of them in the revised manuscript. Specifically, the broken reference in Appendix C.2 under the subsection on the entropy regularization coefficient ε\varepsilon now properly renders as Figure 11 alongside the corresponding plot on page 36, and the missing citation below Eq. (102) in the proof of Theorem 8 now properly renders as (Benamou et al., 2015). We have also checked the entire manuscript and appendix to ensure that all cross-references and citations compile properly.

Reviewer_iHUCOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates unsupervised clustering of multi-view remote sensing data, focusing on three challenges in applying graph masked autoencoders to heterogeneous observations: masking strategies that are insufficiently aware of clustering relevance, noisy initial graph structures, and negative transfer caused by cross-view alignment. The authors propose AM-GMAE, which combines a Clustering-Saliency Adversarial Mask (CSAM) for adaptive masking of clustering-sensitive nodes, Dynamic Graph Purification and Topology Optimization (DGPO) for refining graph connectivity, and Multi-marginal Optimal Transport (MOT) with a Wasserstein barycenter for aligning heterogeneous representations. Experiments on four remote sensing datasets, including comparisons and ablations, report competitive clustering results.

Critical strengths
  1. A unified treatment of relevant challenges. The paper addresses multiple interconnected difficulties in multi-view remote sensing clustering, including the learning of graph representations, the reliability of graph structures, and alignment across heterogeneous views. These problems are clearly motivated by the application setting.

  2. Thorough empirical evaluation. The authors compare AM-GMAE with a range of existing multi-view clustering methods on four remote sensing datasets and report several clustering metrics, including ACC, Kappa, NMI, ARI, and PUR. Ablation studies, graph-topology analyses, and sensitivity experiments provide useful evidence about the proposed design.

Critical weaknesses
  1. High methodological complexity and unclear necessity of the combined components. AM-GMAE integrates clustering-aware masking, graph purification, topology optimization, and optimal transport. Although each component has a reasonable motivation, the paper does not justify why these mechanisms must be combined, nor does it clearly identify the most distinctive methodological contribution.

  2. Limited direct evidence for some central mechanisms. The paper argues that CSAM suppresses trivial cross-view reconstruction shortcuts and that MOT alleviates negative transfer during heterogeneous-view alignment. However, the experiments support these claims primarily through downstream clustering improvements. While graph-topology analyses offer more direct evidence for DGPO, there is comparatively little direct measurement of cross-view information leakage, reconstruction shortcuts, or negative transfer.

Questions for authors
  1. Compared with random masking, existing adaptive masking methods, and conventional pairwise optimal transport, what are the distinctive advantages of CSAM and MOT? Could the authors provide more controlled comparisons to separate the effects of these mechanisms from added model complexity?

  2. Could the authors directly evaluate the proposed claims by controlling cross-view correlation, introducing incorrect associations, or increasing modality heterogeneity? In particular, can they measure whether CSAM reduces reconstruction shortcuts and whether MOT mitigates negative transfer?

7 comments

We sincerely thank you for the constructive feedback and insightful questions. We address your specific concerns in detail below.

W1: Thank you for the opportunity to clarify the central contribution and its task-specific motivation. The central contribution of AM-GMAE is a task-specific feedback design for unsupervised clustering of multi-view remote-sensing data. Each node is a superpixel at a fixed geographic location, so the same node index refers to the same place across views. The sensors nevertheless record different physical quantities: HSI and MSI record optical spectra, SAR records microwave backscatter, and DSM records elevation. Consequently, a land-cover boundary may be clear in one view but weak in another, and two nearby regions may look similar in one sensor while belonging to different classes in another. This creates three coupled failure modes: a masked node can be reconstructed from its unmasked counterpart in another view, graph propagation can strengthen cross-class edges, and rigid alignment can transfer an unreliable modality's geometry to a reliable one.

AM-GMAE addresses this task-specific failure pattern with a feedback loop in which the current clustering structure controls three different objects. CSAM uses clustering uncertainty to decide which nodes should be removed for reconstruction; DGPO uses the same semantic gaps to decide which graph edges remain trustworthy; and MOT aligns view-specific prototype distributions through a shared semantic marginal instead of forcing node-to-node equality. This design reflects the fact that remote-sensing views share locations but measure different physical properties.

The following controlled results quantify each part of this feedback loop. All values are means over 10 seeds. For CSAM, only the masking rule changes:

DatasetRandom ACCRandom NMIRandom ARIFull CSAM ACCFull CSAM NMIFull CSAM ARI
Trento0.89670.88210.88940.95850.94150.9588
Salinas0.78960.79580.73490.85720.85840.8168
XuZhou0.78550.69590.68860.84350.75180.7465
MDAS0.44710.30590.30600.51010.36220.3715

For DGPO, CSAM and MOT are fixed and only the graph-update rule changes:

DatasetStatic k-NN ACCStatic k-NN NMIStatic k-NN ARIFull DGPO ACCFull DGPO NMIFull DGPO ARI
Trento0.93120.91490.93160.95840.94170.9590
Salinas0.81560.82130.74500.85740.85830.8169
XuZhou0.81100.72490.71930.84360.75180.7467
MDAS0.46420.32580.33160.51000.36240.3714

For MOT, the same CSAM, DGPO, encoder, optimizer, initialization, and seeds are used for all alignment variants. We compare three alignment designs and one capacity control. Sample-level pairwise OT transports individual node embeddings between each pair of views with an M x M node-to-node cost matrix, following the entropy-regularized OT formulation of Cuturi (NeurIPS 2013) and its multi-view use in Maiorca et al. (NeurIPS 2023). Prototype-level pairwise OT transports node-to-prototype assignment distributions separately for each view pair; it retains pairwise couplings but has no shared barycentric marginal, as in SwAV (Caron et al., NeurIPS 2020) and DHOT (Luo et al., TPAMI 2023). MOT jointly aligns all views through a shared semantic marginal obtained from a Wasserstein barycenter, following Cuturi and Doucet (ICML 2014). The capacity-control MLP is a non-transport control: it replaces the alignment module with a nonlinear projection head and matches or exceeds the transport baselines in parameter count. The table reports the three clustering metrics separately for each method:

DatasetMethodACCNMIARI
SalinasSample-level pairwise OT0.82700.82450.7670
SalinasPrototype-level pairwise OT0.83970.84050.7875
SalinasCapacity-control MLP0.82120.82140.7570
SalinasMOT0.85430.85750.8135
XuZhouSample-level pairwise OT0.82820.73480.7302
XuZhouPrototype-level pairwise OT0.83290.74000.7366
XuZhouCapacity-control MLP0.82380.73350.7284
XuZhouMOT0.84060.75380.7438
MDASSample-level pairwise OT0.45890.32140.3203
MDASPrototype-level pairwise OT0.49510.34600.3538
MDASCapacity-control MLP0.48110.33450.3422
MDASMOT0.51090.36190.3729

Q1: Thank you for asking us to clarify the controlled comparisons and the motivation for each baseline category. The experiment changes one rule at a time on Trento, Salinas, XuZhou, and MDAS while keeping data preprocessing, architecture, optimizer, training length, CSAM/DGPO/MOT settings, and the paired seed set fixed.

The seven masking schemes are classified by which signal supplies the sampling score and therefore determines each node’s masking probability. The first five rows are baseline signals: none, graph degree, reconstruction error, assignment entropy, and single-view adversarial reconstruction, whereas the last two rows are the CSAM ablation pair: clustering sensitivity without calibration and clustering sensitivity with cross-view calibration. In this ordering, Random Masking has no score, Degree-Guided Masking uses graph degree, Loss-Guided Adaptive Masking uses reconstruction error, High-Entropy Masking uses assignment entropy, and Uncorrelated Adversarial Masking uses a single-view adversarial reconstruction signal. CSAM without calibration then replaces these generic signals with first-order clustering sensitivity, while Full CSAM uses the same clustering sensitivity plus cross-view calibration. Thus, the sixth and seventh rows are an ablation pair: they isolate the gain from calibrating a clustering-derived score across views.

  1. Random Masking uses no node-dependent signal: pi(v)=1/Mp_i^{(v)}=1/M. This is the random masking setting of GraphMAE (Hou et al., KDD 2022) and GraphMAE2 (Hou et al., WWW 2023).
  2. Degree-Guided Masking uses only the initial graph degree: pi(v)=di(v)/∑jdj(v)p_i^{(v)}=d_i^{(v)}/\sum_j d_j^{(v)}. This follows StructMAE (Liu et al., IJCAI 2024) and MaskGAE (Li et al., KDD 2023).
  3. Loss-Guided Adaptive Masking uses the previous-round feature-reconstruction error ei(v)=1−cos⁡(zi(v),z^i(v))e_i^{(v)}=1-\cos(\mathbf z_i^{(v)},\hat{\mathbf z}_i^{(v)}) as the sampling score. It represents HGMAE (Tian et al., AAAI 2023), UGMAE (Tian et al., 2024), and Cur-MGAE (Li et al., ICML 2025).
  4. High-Entropy Masking uses the Shannon entropy magnitude Hi(v)\mathcal H_i^{(v)} of the current soft assignment and does not use the local first-order perturbation sensitivity. This follows CG-MAE (Guan et al., AAAI 2026).
  5. Uncorrelated Adversarial Masking uses gradient ascent on the single-view reconstruction loss. It does not use the clustering soft assignment or any cross-view calibration, following AUG-MAE (Wang et al., AAAI 2024) and ACGMAE (Shen et al., AAAI 2025).
  6. CSAM without calibration uses the current first-order clustering-uncertainty sensitivity Si(v)S_i^{(v)} independently in each view.
  7. Full CSAM uses the same clustering sensitivity.

The complete masking results are below. Values are the mean ± standard deviation over the paired seeds.

DatasetStrategyACCNMIARI
Trento10.8967 ± 0.00740.8821 ± 0.00670.8894 ± 0.0061
Trento20.9177 ± 0.00850.8893 ± 0.00980.9092 ± 0.0087
Trento30.9070 ± 0.00690.9050 ± 0.00630.9012 ± 0.0073
Trento40.9145 ± 0.00920.8974 ± 0.00690.9053 ± 0.0091
Trento50.9305 ± 0.00900.9120 ± 0.00990.9229 ± 0.0100
Trento60.9440 ± 0.00820.9311 ± 0.00770.9417 ± 0.0082
Trento70.9585 ± 0.00210.9415 ± 0.00280.9588 ± 0.0019
Salinas10.7896 ± 0.01410.7958 ± 0.00990.7349 ± 0.0150
Salinas20.7835 ± 0.01320.7985 ± 0.00970.7266 ± 0.0120
Salinas30.8087 ± 0.01250.8259 ± 0.00990.7643 ± 0.0147
Salinas40.8005 ± 0.01190.8042 ± 0.00790.7511 ± 0.0126
Salinas50.8229 ± 0.01370.8192 ± 0.01140.7792 ± 0.0132
Salinas60.8411 ± 0.01320.8423 ± 0.00950.7971 ± 0.0124
Salinas70.8572 ± 0.01310.8584 ± 0.00650.8168 ± 0.0103
XuZhou10.7855 ± 0.01250.6959 ± 0.01170.6886 ± 0.0141
XuZhou20.7910 ± 0.01210.7123 ± 0.01000.6920 ± 0.0132
XuZhou30.8021 ± 0.01280.7154 ± 0.01150.7104 ± 0.0130
XuZhou40.8059 ± 0.01270.7036 ± 0.01140.7018 ± 0.0154
XuZhou50.8236 ± 0.01300.7308 ± 0.00960.7232 ± 0.0129
XuZhou60.8311 ± 0.01200.7389 ± 0.00960.7344 ± 0.0122
XuZhou70.8435 ± 0.00850.7518 ± 0.00550.7465 ± 0.0092
MDAS10.4471 ± 0.01200.3059 ± 0.01050.3060 ± 0.0118
MDAS20.4555 ± 0.00910.3269 ± 0.00900.2989 ± 0.0089
MDAS30.4648 ± 0.00950.3395 ± 0.01040.3253 ± 0.0094
MDAS40.4646 ± 0.01130.3183 ± 0.01010.3316 ± 0.0089
MDAS50.4778 ± 0.01030.3291 ± 0.01070.3446 ± 0.0091
MDAS60.4965 ± 0.01010.3487 ± 0.00940.3554 ± 0.0077
MDAS70.5101 ± 0.00650.3622 ± 0.00540.3715 ± 0.0047

Q1: For the OT experiment, CSAM and DGPO are fixed. Each method is trained with the controlled objective Ldiagχ=Ltask+ξLalignχ\mathcal L_{diag}^\chi=\mathcal L_{task}+\xi\mathcal L_{align}^\chi, the same paired initialization, the same 10 seeds, and the same ξ\xi. The comparison is classified by the transport object and the coupling pattern. Sample-level pairwise OT constructs one M×MM \times M node-to-node cost matrix for every view pair and solves (V2)\binom{V}{2} separate Sinkhorn problems, following Cuturi (NeurIPS 2013) and Maiorca et al. (NeurIPS 2023). Prototype-level pairwise OT aligns sample-to-prototype assignment distributions separately for each pair and has no shared barycentric marginal, following SwAV (Caron et al., NeurIPS 2020) and DHOT (Luo et al., TPAMI 2023). The capacity-control MLP removes transport and adds a nonlinear projection head with 787,968 extra parameters on Salinas and XuZhou and 1,050,624 on MDAS. MOT uses the shared semantic marginal b∗b^* of the Wasserstein barycentric relaxation, following Cuturi and Doucet (ICML 2014).

DatasetMethodACCNMIARIExtra parameters (count)Alignment time (ms)Module peak increment (MB)Peak GPU memory (MB)
MDASPrototype-level pairwise OT0.4951 ± 0.00080.3460 ± 0.00080.3538 ± 0.0007019.8421.92661.10
MDASCapacity-control MLP0.4811 ± 0.00080.3345 ± 0.00090.3422 ± 0.00071,050,6242.4510.63649.81
MDASMOT0.5109 ± 0.00170.3619 ± 0.00140.3729 ± 0.000806.012.45641.63
MDASSample-level pairwise OT0.4589 ± 0.00070.3214 ± 0.00190.3203 ± 0.0011021.98104.47743.65
SalinasPrototype-level pairwise OT0.8397 ± 0.00110.8405 ± 0.00140.7875 ± 0.0019011.7018.883402.20
SalinasCapacity-control MLP0.8212 ± 0.00070.8214 ± 0.00050.7570 ± 0.0009787,9682.5210.033393.35
SalinasMOT0.8543 ± 0.00060.8575 ± 0.00100.8135 ± 0.001104.642.533385.85
SalinasSample-level pairwise OT0.8270 ± 0.00100.8245 ± 0.00040.7670 ± 0.0023016.93116.163499.48
XuZhouPrototype-level pairwise OT0.8329 ± 0.00050.7400 ± 0.00060.7366 ± 0.0008017.172.32801.41
XuZhouCapacity-control MLP0.8238 ± 0.00090.7335 ± 0.00080.7284 ± 0.0006787,9681.595.59804.68
XuZhouMOT0.8406 ± 0.00030.7538 ± 0.00050.7438 ± 0.000702.681.00800.08
XuZhouSample-level pairwise OT0.8282 ± 0.00050.7348 ± 0.00100.7302 ± 0.0005014.6622.89821.98

Our MOT formulation first notes that exact MOT requires a joint probability tensor with NP-hard O(KV)O(K^V) complexity. The implemented method uses the Wasserstein-barycentric relaxation with a shared semantic marginal. Its geometric cost is an M×KM\times K node-to-prototype matrix, which removes the O(M2)O(M^2) node-to-node comparison used by pairwise OT; entropy regularization makes the transport plans solvable by Sinkhorn iterations.

Reviewer_hDd8OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies multi-view remote sensing data clustering using Graph Masked Autoencoders (GMAEs) and proposes Adaptive Manifold Graph Masked AutoEncoder (AM-GMAE). The paper identifies three challenges when extending GMAEs to heterogeneous multi-view remote sensing scenarios: existing masking strategies may not sufficiently consider clustering relevance, graph structures constructed from similarity relationships may contain noisy connections that affect manifold learning, and heterogeneous view representations may be difficult to align effectively. To address these challenges, the authors introduce three components. First, the Clustering-Saliency Adversarial Mask (CSAM) module selects nodes for masking based on clustering uncertainty sensitivity, aiming to encourage reconstruction from more informative and clustering-related signals. Second, the Dynamic Graph Purification and Topology Optimization (DGPO) module refines graph structures by incorporating topology-aware constraints. Third, the Multi-marginal Optimal Transport (MOT) module aligns heterogeneous view representations by learning a shared semantic geometry through Wasserstein barycenter estimation. The proposed framework is evaluated on four multi-view remote sensing clustering benchmarks (Trento, Salinas, XuZhou, and MDAS) against several recent multi-view clustering baselines. The reported experiments show that AM-GMAE achieves competitive performance on the evaluated datasets, and ablation studies are conducted to investigate the contribution of different components.

Critical strengths

1.The paper addresses an important limitation of applying Graph Masked Autoencoders to multi-view remote sensing clustering. The proposed clustering-saliency adaptive masking mechanism is a meaningful extension of existing masking strategies, as it explicitly considers the relationship between representation reconstruction and clustering uncertainty.

2.The empirical evaluation is comprehensive. The authors compare the proposed method with a relatively large set of recent multi-view clustering baselines on four benchmarks and provide ablation studies to analyze the contribution of different components. The experimental results demonstrate the effectiveness of the proposed framework in the evaluated scenarios.

Critical weaknesses

1.A main limitation is that the current evaluation is restricted to multi-view remote sensing clustering benchmarks. Although the proposed techniques are motivated by general graph representation learning principles, the current experiments do not provide evidence on whether AM-GMAE can generalize beyond remote sensing data. Additional experiments on more general graph clustering or multi-view representation learning benchmarks would further strengthen the claim of broader applicability.

2.The proposed framework integrates several related research directions, including adaptive masking, graph structure learning, topology regularization, and optimal transport alignment. While the overall integration is carefully designed and empirically effective, the paper would benefit from a clearer positioning of its methodological novelty relative to existing approaches. In particular, further discussion on how DGPO and MOT differ from existing graph refinement and optimal transport-based alignment methods would help better understand their individual contributions.

Questions for authors
  1. The proposed framework integrates three major components: CSAM, DGPO, and MOT. Could the authors provide further analysis on how these components complement each other and clarify the specific role of each component in improving multi-view clustering performance? For example, it would be helpful to understand which limitations of existing GMAE-based approaches are primarily addressed by each module.
  2. The evaluation is mainly conducted on multi-view remote sensing clustering benchmarks. Could the authors provide further discussion on the potential applicability of AM-GMAE beyond this specific domain? In particular, it would be interesting to understand whether the proposed clustering-aware masking and manifold optimization strategies could be extended to more general graph-based representation learning scenarios. 3.AM-GMAE incorporates several techniques related to existing research directions, including adaptive masking, graph refinement, and optimal transport-based alignment. Could the authors further clarify the methodological contribution of AM-GMAE relative to previous approaches in these areas? In particular, what new insights or design principles does the proposed framework introduce beyond the combination of these existing techniques?
6 comments

We sincerely thank you for the constructive feedback and insightful questions. We address your specific concerns in detail below.

W1: Thank you for pointing out the importance of evaluating AM-GMAE beyond remote sensing data. AM-GMAE is designed for multi-view remote sensing clustering, where superpixel nodes share fixed spatial locations across views. When extending AM-GMAE to general non-spatial multi-view clustering benchmarks, each sample directly serves as a graph node and view-specific kk-NN graphs are constructed from the raw feature spaces. Both CSAM and the manifold optimization modules (DGPO and MOT) remain well-defined on general attributed graphs {Z(v),A(v)}v=1V\left\lbrace \mathbf{Z}^{(v)}, \mathbf{A}^{(v)} \right\rbrace_{v=1}^{V}, because first-order clustering sensitivity Si(v)S_i^{(v)}, structural entropy regularization Lstr\mathcal{L}_{\text{str}}, random-walk denoising Ltopo\mathcal{L}_{\text{topo}}, and Wasserstein barycentric alignment LMOT\mathcal{L}_{\text{MOT}} depend only on latent representations H(v)\mathbf{H}^{(v)} and prototype assignments Q(v)∈ΔK−1\mathbf{Q}^{(v)} \in \Delta^{K-1}.

To provide an assessment of where AM-GMAE transfers well and where its domain-specific assumptions become less effective, we evaluated AM-GMAE on seven general multi-view benchmarks (NUS-WIDE, Reuters, HandWritten, Wiki, Hdigit, Fashion, MNIST-USPS, and CUB) against representative baselines:

DatasetMethodReference & VenueACC (%)NMI (%)ARI (%)
NUS-WIDECSOTNeurIPS 202325.0012.163.21
NUS-WIDECOPERICLR 202525.5511.678.45
NUS-WIDEMGBCCAAAI 202528.0415.164.88
NUS-WIDEGCFAggCVPR 202329.0211.834.49
NUS-WIDESGMVCInf. Fusion 202630.8013.795.21
NUS-WIDELLM-DAMVCNeurIPS 202537.1137.5119.42
NUS-WIDEAM-GMAE (Ours)—45.3136.8325.56
ReutersMFLVCCVPR 202252.3139.5830.91
ReutersCOPERICLR 202553.1531.1022.80
ReutersTAMVCESWA 202655.6736.1229.45
ReutersAM-GMAE (Ours)—56.2537.7531.21
ReutersLLM-DAMVCNeurIPS 202557.4038.2031.65
ReutersSGMVCInf. Fusion 202659.0739.8732.34
HandWrittenhubREPCVPR 202589.6583.4081.12
HandWrittenBONEAAAI 202692.9489.1387.36
HandWrittenALPCAAAI 202594.9189.4189.19
HandWrittenDCMVCNeurocomputing 202595.0291.6191.61
HandWrittenAM-GMAE (Ours)—95.2589.8989.72
HandWrittenAEMVCACM MM 202596.9593.6893.45
HandWrittenDMVCSIEEE TKDE 202697.8494.9595.25
HandWrittenPCMSLarXiv 202698.3595.5295.36
WikiDIVIDEAAAI 202454.5050.7040.28
WikiCAMVCAAAI 202455.2053.8843.28
WikiAM-GMAE (Ours)—55.3655.0943.65
WikiDealMVCACM MM 202359.3054.5043.95
WikiSCD-MVCIJCAI 202661.4854.4845.35
WikiBPMLDIEEE TIP 202561.7049.6044.15
HdigitDealMVCACM MM 202390.0093.7583.45
HdigitFastMICEIEEE TKDE 202390.8292.0884.12
HdigitAM-GMAE (Ours)—93.6487.9286.78
HdigitSCD-MVCIJCAI 202694.6788.8588.92
HdigitGCFAggCVPR 202397.2592.6294.09
HdigitSGMVCInf. Fusion 202699.5498.5898.98
FashionSTCMC-URAAAI 202592.3293.2187.46
FashionDIVIDEAAAI 202493.0088.9087.85
FashionAM-GMAE (Ours)—93.4291.5688.64
FashionMFLVCCVPR 202295.1595.7496.03
FashionGCFAggCVPR 202399.2297.9298.28
FashionDCMVCNeurocomputing 202599.8097.2098.80
MNIST-USPSCOMPLETERCVPR 202189.0888.8681.94
MNIST-USPSADMCIEEE TMM 202491.2695.5086.42
MNIST-USPSAM-GMAE (Ours)—95.1893.4691.85
MNIST-USPSDIVIDEAAAI 202497.1092.5093.68
MNIST-USPSMFLVCCVPR 202299.5098.5098.89
MNIST-USPSEMVC-RCLIJCAI 202599.9899.8199.95
CUBhubREPCVPR 202567.0172.4556.78
CUBALPCAAAI 202569.8365.5249.94
CUBICMVCAAAI 202470.1270.4857.88
CUBAM-GMAE (Ours)—74.5075.9664.88
CUBGCFAggCVPR 202379.0074.6164.25
CUBBPMLDIEEE TIP 202585.2082.6074.12
CUBCOPERICLR 202587.3385.5679.48

W1: As shown in the table, AM-GMAE exhibits patterns across different data regimes:

  1. Transferability to noisy, heterogeneous cross-modal and multi-descriptor domains (e.g., NUS-WIDE, Reuters, HandWritten, Wiki): On challenging cross-modal benchmarks with modality discrepancy and noisy initial kk-NN neighborhoods, AM-GMAE achieves leading competitive results. Notably, on NUS-WIDE, AM-GMAE reaches 45.31% ACC and 25.56% ARI, outperforming recent methods such as SGMVC (Cui et al., Information Fusion 2026; 30.80% ACC) and LLM-DAMVC (NeurIPS 2025; 37.11% ACC). On Reuters, AM-GMAE achieves 56.25% ACC, outperforming COPER (Eisenberg et al., ICLR 2025; 53.15%) and TAMVC (Expert Systems with Applications 2026; 55.67%), while remaining slightly below LLM-DAMVC (NeurIPS 2025; 57.40%) and SGMVC (Cui et al., Information Fusion 2026; 59.07%). On HandWritten, AM-GMAE surpasses graph/anchor baselines such as ALPC (Chen et al., AAAI 2025; 94.91%) and DCMVC (Li et al., Neurocomputing 2025; 95.02%), though it trails recent methods tailored for general MVC such as DMVCS (Li et al., IEEE TKDE 2026; 97.84%) and PCMSL (Lu et al., arXiv 2026; 98.35%).

  2. Performance ceiling on homogeneous-modality visual benchmarks (MNIST-USPS, Fashion, Hdigit) and small-sample datasets (CUB) due to neighborhood smoothing and lack of pair-level contrastive repulsion: MNIST-USPS, Hdigit, and Fashion consist of homogeneous visual views where cross-modal negative transfer is mild, and SOTA methods routinely exceed 99% ACC by leveraging instance-level contrastive repulsion to sharpen fine-grained boundaries. In AM-GMAE, however, both GMAE masked reconstruction (Eq. 5) and one-step random-walk smoothing (Eq. 8) are neighborhood-averaging objectives without instance-level negative-pair repulsion, and MOT aligns views only at the coarse KK-prototype distribution level (b∗\mathbf{b}^{\ast}). While this neighborhood averaging is crucial for suppressing sensor noise in remote sensing, on homogeneous visual datasets it slightly blurs boundary samples between visually similar classes (e.g., Shirt vs. T-shirt/Coat in Fashion, or similar handwritten digits in MNIST-USPS/Hdigit). Meanwhile, on CUB, masking 50% of nodes leaves only around 30 visible samples per class during training, making graph message-passing recovery and 2D structural-entropy volume estimation (Eq. 7) statistically sparse. Consequently, while AM-GMAE still outperforms several classical and recent baselines, such as DIVIDE (Lu et al., AAAI 2024; 93.00% on Fashion, 54.50% on Wiki) and hubREP (Xu et al., CVPR 2025; 67.01% on CUB)—it trails methods that directly optimize instance-level contrastive discrimination or generative diffusion on MNIST-USPS (95.18% vs. 99.50% for MFLVC, Xu et al., CVPR 2022 and 99.98% for EMVC-RCL, Wang et al., IJCAI 2025), Fashion (93.42% vs. 99.22% for GCFAgg, Yan et al., CVPR 2023 and 99.80% for DCMVC, Li et al., Neurocomputing 2025), Hdigit (93.64% vs. 94.67% for SCD-MVC, Ma et al., IJCAI 2026 and 99.54% for SGMVC, Cui et al., Information Fusion 2026), and CUB (74.50% vs. 85.20% for BPMLD, Wang et al., IEEE TIP 2025 and 87.33% for COPER, Eisenberg et al., ICLR 2025).

W2: Thank you for encouraging us to clarify the methodological positioning of AM-GMAE relative to existing adaptive masking, graph structure learning, and optimal transport alignment methods. AM-GMAE introduces a unified feedback design in which the current soft cluster assignment Q(v)\mathbf{Q}^{(v)} and view-specific prototypes μ(v)\boldsymbol{\mu}^{(v)} jointly govern node perturbation (CSAM), graph topology refinement (DGPO), and cross-view distributional alignment (MOT). Specifically, DGPO and MOT differ from existing approaches in the following fundamental ways:

  1. How DGPO Differs from Existing Graph Refinement Methods: Conventional graph structure learning (GSL) approaches refine graphs via dense cosine re-weighting as in SUBLIME (Liu et al., WWW 2022) and SAMVGC (Guan et al., AAAI 2025), fixed hard thresholding as in IDGL (Chen et al., NeurIPS 2020), or K-Means hard pseudo-label edge pruning as in DeepCluster (Caron et al., ECCV 2018) and ACGMAE (Shen et al., AAAI 2025), typically paired with standard Laplacian smoothness Tr⁡(H⊤LH)\operatorname{Tr}(\mathbf{H}^\top \mathbf{L} \mathbf{H}) (Kalofolias, AISTATS 2016). However, hard pseudo-label pruning suffers from confirmation bias when early cluster predictions are noisy, whereas Laplacian smoothness over unpurified graphs causes cross-class over-smoothing. In contrast, DGPO is pseudo-label-free: it combines a learnable threshold τth\tau_{\text{th}} with 1D and 2D structural entropy minimization (L1str+L2str\mathcal{L}_{1\text{str}} + \mathcal{L}_{2\text{str}}, Eq. 7). DGPO's specific role is to weaken cross-class message passing so that masked boundary nodes are reconstructed from class-consistent neighbors. As proved in Appendix B.3 (Theorem 4 & Corollary 5), L2str\mathcal{L}_{2\text{str}} formulates a differentiable soft semantic graph cut that penalizes expected outgoing edge weights across cluster boundaries using continuous assignments Q(v)\mathbf{Q}^{(v)}. Moreover, Ltopo\mathcal{L}_{\text{topo}} (Eq. 8) enforces a one-step random-walk denoising target over the purified homophilous neighborhood, which preserves class-conditional means while strictly contracting feature-noise covariance (Theorem 6).

  2. How MOT Differs from Existing Optimal Transport Alignment Methods: Existing OT-based multi-view methods typically adopt either sample-level pairwise OT with an M×MM\times M node-to-node cost matrix across all (V2)\binom{V}{2} view pairs, following Sinkhorn OT (Cuturi, NeurIPS 2013) and latent translation (Maiorca et al., NeurIPS 2023)—or prototype-level pairwise OT that aligns sample-to-prototype distributions between pairs of views without a shared multi-view marginal, as in SwAV (Caron et al., NeurIPS 2020) and DHOT (Luo et al., TPAMI 2023). Sample-level pairwise OT incurs O(RV2M2)\mathcal{O}(RV^2M^2) complexity and forces rigid node-to-node matching across heterogeneous manifolds, while exact VV-marginal OT requires an intractable O(KV)\mathcal{O}(K^V) tensor. Our MOT instead formulates a sample-to-prototype Wasserstein barycentric relaxation coupled through a single shared semantic marginal b∗\mathbf{b}^{\ast} (Eqs. 9–12), following Cuturi & Doucet (ICML 2014) and Benamou et al. (SIAM J. Sci. Comput. 2015), which scales linearly as O(RVMK)\mathcal{O}(RVMK) (Appendix B.5). Crucially, Appendix B.4 (Theorem 9) proves that because transport costs are measured between view-specific nodes hi(v)\mathbf{h}_i^{(v)} and view-specific prototypes μk(v)\boldsymbol{\mu}_k^{(v)}, MOT is strictly invariant to independent view-wise Euclidean isometries (Rvhi(v)+tv\mathbf{R}_v \mathbf{h}_i^{(v)} + \mathbf{t}_v), achieving global semantic consensus without distorting modality-specific geometry.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.