acceptodds
Under review as a conference paper at ICLR 2027

DSMVC: Disentangled Streaming Multi-View Clustering via Purification and Calibration

Abstract

Streaming multi-view clustering has attracted growing attention for its ability to aggregate continuously arriving views without accessing historical raw data. However, emerging views often introduce private variations and unreliable cross-view correspondences, inevitably corrupting the global consensus over time. To address this issue, we propose a novel Disentangled Streaming Multi-View Clustering (DSMVC) framework. Specifically, DSMVC leverages spectral curvature to decouple dominant structures from private residuals, securing reliable knowledge accumulation through two core modules: an Attribution-Guided Suppression Module that suppresses vulnerable components by quantifying feature attribution sensitivity to purify residual representations, and a Discrepancy-Guided Cross-view Alignment Module that calibrates cross-view correspondences via permutation-induced prediction inconsistency, ensuring robust consensus semantics across streaming views. Extensive experiments across benchmark datasets demonstrate that DSMVC achieves substantial advancement over state-of-the-art methods, exhibiting remarkable robustness against continuous view arrivals.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Streaming Multi-view Clustering, Disentangled Representation Learning, Unsupervised Learning, Deep Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_A1J8OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DSMVC addresses clustering with sequentially arriving views without revisiting historical raw data. It uses a curvature-based SVD partition to obtain dominant and residual representations. AGSM suppresses residual reconstruction dimensions according to their contribution to reconstruction non-additivity, while DGAM weights cross-view alignment using prediction-error differences between matched and permuted pairs. Historical embeddings are retained for subsequent alignment and final clustering. Experiments on seven datasets report improvements over the evaluated sequential baselines.

Critical strengths
  1. The paper addresses a relevant problem: clustering sequentially arriving views without revisiting historical raw data.

  2. Table 1 reports consistent gains over the evaluated sequential baselines across seven datasets and three metrics, using five runs.

  3. Gaussian-noise experiments and component ablations support the framework’s robustness and the usefulness of its added objectives.

Critical weaknesses
  1. AGSM’s selection rule needs stronger validation. Eqs. (11)–(12) measure reconstruction non-additivity, which need not identify nuisance features. Existing experiments support robustness and suppression, but do not isolate targeted selection from general regularization. A matched random-mask or uniform-penalty control would help.

  2. DGAM’s weight ablation is confounded by alignment strength. Setting all weights to 1 in Appendix C.3 changes both correspondence weighting and overall loss strength. A mean-matched constant weight or retuned alignment coefficient is needed to isolate the benefit of reliability-based weighting.

  3. Some reproducibility details remain unclear. Algorithm 1 and Appendix C.7 do not clearly specify final buffer-feature computation after mini-batch SVD, gradient detachment for adaptive weights, or input feature extraction and normalization. These choices can affect reproduction of the reported results.

Questions for authors
  1. Does AGSM retain its advantage over suppression-budget-matched random masks or uniform penalties?

  2. Does DGAM improve over a mean-matched constant weight or a uniform-weight baseline with a retuned alignment coefficient?

  3. How are final buffer features computed, are adaptive weights detached, and what feature extraction and normalization are used?

3 comments

[W1/Q1: Validation of the Attribution-Guided Selection Rule] We thank the reviewer for the insightful suggestion to isolate targeted selection from general regularization. To investigate this, we compare AGSM with two baselines: Uniform Penalty (unweighted penalty across all dimensions) and Random Mask (stochastically dropping dimensions matched to AGSM's masking ratio). As shown in Table 1, AGSM consistently outperforms both variants across all datasets. For instance, it exceeds both variants by over 23% in ACC on COIL20. This confirms that indiscriminate masking inevitably corrupts essential consensus features, whereas AGSM identifies and suppresses volatile components, effectively purifying the feature subspace. Detailed content is provided in Appendix D.8 (P18–P19).

Table 1: Clustering Accuracy under Different Masking Settings

SettingCOIL20ALOI100LeavesHandwrittenOut-Scene
Uniform Penalty55.46±1.6779.56±3.7562.64±4.5288.46±4.2255.28±4.99
Random Mask56.09±4.1278.24±2.8864.71±2.2787.69±5.0956.48±3.27
AGSM (Ours)79.71±1.7694.53±0.5690.41±1.3896.67±0.3669.63±4.85

[W2/Q2: Ablation on Alignment Weights] We appreciate the reviewer's advice to decouple reliability-based weighting from overall alignment strength. To achieve this, we compare DSMVC with two variants, including a constant baseline with the average dynamic weight (γw=γˉw\gamma^w = \bar{\gamma}^w) and a uniform baseline (γw=1\gamma^w = 1) with its optimal coefficient. As shown in Table 2, DSMVC achieves superior performance over both comparative baselines across all benchmarks. Even after exhaustive tuning, the uniform baseline still trails DSMVC by substantial margins, such as 14.55% in ACC on COIL20 and 2.42% on Out-Scene, indicating that models relying on uniform scaling remain vulnerable to unreliable associations. By contrast, DGAM dynamically downweights unreliable pairs via permutation-induced prediction inconsistency, preserving reliable semantic consensus across streaming scenarios. We have added the relevant analysis in Appendix D.4 (P16-P17).

Table 2: Clustering Accuracy under Different Alignment Weights

SettingCOIL20ALOI100LeavesHandwrittenOut-Scene
Constant54.79±3.2378.99±4.7164.42±2.9290.90±2.6459.24±2.00
Uniform65.16±4.1093.68±0.3189.88±1.5295.12±0.0667.21±1.17
DGAM (Ours)79.71±1.7694.53±0.5690.41±1.3896.67±0.3669.63±4.85

[W3/Q3: Reproducibility and Implementation Details] We sincerely thank the reviewer for pointing out these critical implementation details. For clarity, we have added these details in Appendix D.12 (P21): 1) Input features across all datasets are pre-processed via row-wise min-max normalization into [0,1][0, 1]. 2) After spectral decomposition, the dominant features are cached in the historical buffer to guide the training of DGAM, while residual features are preserved for final clustering. 3) The adaptive alignment weights γw\gamma^w in DGAM are detached from the computational graph, derived from permutation-induced prediction inconsistency to prevent trivial degeneration during backpropagation. To ensure reproducibility, our code will be publicly released upon acceptance.

We are grateful for the Reviewer's time and rigorous feedback. Your comments prompted us to provide more empirical experiments and implementation details that substantially solidify our innovations and claims. We sincerely hope our thorough responses can resolve your concerns. Please don't hesitate to contact us with any questions, and we would be happy to provide further clarification.

Thank you for your response and the additional experiments. I have read the response, and it addresses my main concerns. I will update my rating to 3.

Thank you very much for your reply and for taking the time to read our response! We are glad it addressed your concerns, and we really appreciate your helpful comments on our work.

Reviewer_o7nAOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies streaming multi-view clustering, where views arrive sequentially and previously observed raw data cannot be revisited. The authors identify two problems: shared semantic information can become entangled with view-specific variation, and unreliable cross-view correspondences may progressively contaminate the consensus representation. They propose DSMVC, which uses singular value decomposition and spectral curvature to separate dominant and residual components. An Attribution-Guided Suppression Module (AGSM) suppresses potentially unreliable residual features based on attribution sensitivity, while a Discrepancy-Guided Cross-view Alignment Module (DGAM) uses prediction differences between naturally matched and permuted pairs to calibrate cross-view alignment. The method is evaluated on seven multi-view clustering benchmarks using ACC, NMI, and ARI, along with ablations, noise-robustness studies, and view-order analyses.

Critical strengths
  1. A meaningful streaming-learning problem. The work focuses on how view-specific variation and unreliable correspondence can accumulate during sequential integration, rather than considering only final clustering quality. The motivation is relevant to settings where historical raw views are unavailable.

  2. Strong experimental results across streaming benchmarks. DSMVC is compared against sequential-view and full-view methods on seven datasets. In the reported comparisons, it outperforms the listed sequential-view baselines in ACC, NMI, and ARI across the seven datasets. Ablations, noise experiments, and view-order analyses provide additional evidence of robustness.

  3. Clear and effective visual presentation. The paper includes well-designed and informative figures, particularly the framework overview in Figure 2.

Critical weaknesses
  1. Insufficient justification for interpreting spectral components as shared semantics. DSMVC assumes that the dominant singular components of a latent representation primarily capture shared semantics, whereas the residual components capture view-specific variation and noise; spectral curvature is used to select the split. However, larger singular values reflect dominant directions of variation and do not indicate cross-view shared semantics.

  2. Limited direct validation of unreliable correspondence handling. DGAM constructs permuted pairs as contrasts and uses prediction inconsistency to regulate alignment. Yet random permutations can produce mismatches that are easier to detect than difficult real-world errors between highly similar but incorrectly associated samples.

Questions for authors
  1. How can the authors establish that the leading components selected by spectral curvature capture shared semantics rather than high-variance view-specific factors? Could a synthetic setting with known shared and private factors, or a direct measurement of semantic correspondence, clarify this assumption?

  2. How does DGAM perform when cross-view mismatches are semantically similar but incorrect? Could the authors evaluate controlled levels of correspondence errors and compare permutation-based calibration with direct or confidence-weighted alignment?

3 comments

[W1/Q1: Reliability of Disentanglement Mechanism] We thank the reviewer for the constructive question about the disentanglement mechanism. To investigate this, we conduct a controlled experiment on a synthetic benchmark with known ground-truth factors. In detail, we construct a two-view dataset by overlaying private stripes (horizontal for View 1, vertical for View 2) onto shared semantic shapes. By training DSMVC on this dataset, the separated components reveal that the dominant subspace effectively preserves the shared shape patterns while relegating view-specific variations into the residual subspace. These results confirm that our spectral partition strategy reliably isolates cross-view shared structures from private variations. We have added the visual evidence to Appendix D.2 (P15-P16).

[W2/Q2: Robustness to Correspondence Mismatches] We appreciate the reviewer's question regarding DGAM's robustness against semantically similar cross-view mismatches. To examine this, we compare DGAM with Direct Alignment and Confidence-weighted Alignment under both fully matched (Clean) and intra-class sample mismatch (Semantic) scenarios. As shown in Table 1, DGAM consistently achieves superior performance across all settings. While the other comparative strategies are still misled by the high similarity between samples on complex datasets like COIL20 and 100Leaves, DGAM leverages prediction inconsistency to dynamically calibrate cross-view correspondences, ensuring robust consensus semantics. Detailed experiments and analysis are provided in Appendix D.11 (P20).

Table 1: Clustering Accuracy under Correspondence Mismatches

(Dataset) MethodCleanSemantic (0.1)Semantic (0.3)Semantic (0.5)Semantic (0.7)Semantic (0.9)
(COIL20) Direct76.08±2.0673.88±3.7277.00±3.4175.04±2.3376.79±3.4978.13±2.76
(COIL20) Confidence78.08±3.0777.71±1.8877.67±2.8077.79±2.3177.50±1.7976.17±3.06
(COIL20) DGAM79.71±1.7677.91±2.3480.00±3.3278.79±3.2479.54±5.0778.29±4.68
(ALOI) Direct93.90±0.3894.76±0.2295.98±0.3396.39±0.5998.24±0.3899.35±0.20
(ALOI) Confidence93.57±0.6594.46±0.5395.85±0.3196.48±0.4498.22±0.3799.35±0.19
(ALOI) DGAM94.53±0.5694.77±0.4996.00±0.7197.48±0.4998.37±0.4199.48±0.28
(100Leaves) Direct88.88±1.0190.43±0.4189.94±0.7689.88±1.7690.69±0.8089.76±0.54
(100Leaves) Confidence88.69±1.4390.59±1.6990.32±1.7690.03±0.9690.31±0.5591.15±1.01
(100Leaves) DGAM90.41±1.3892.26±0.8990.89±2.2091.36±1.3791.61±0.9391.63±0.74
(Handwritten) Direct96.96±0.1595.40±3.3595.11±5.4898.51±0.2098.87±0.2896.49±5.80
(Handwritten) Confidence96.20±0.4796.65±0.6594.97±5.6998.11±0.2798.81±0.3099.49±0.19
(Handwritten) DGAM96.67±0.3696.90±0.4297.51±0.3598.61±0.1499.03±0.1799.53±0.11
(Out-Scene) Direct68.62±0.6270.21±1.8572.95±1.6575.61±1.5579.14±2.1781.06±1.68
(Out-Scene) Confidence67.05±1.3770.19±1.2272.87±1.2174.96±1.8478.87±1.4080.49±2.04
(Out-Scene) DGAM69.63±4.8571.06±1.2774.16±0.6077.24±1.2980.21±1.0881.08±0.87

Thank you again for prompting us to re-examine the core mechanics of DSMVC. We hope these concrete revisions earn your confidence in our work and encourage you to raise the final rating. We would be delighted to address any further questions at any time.

Thank you for the response and additional experiments. The synthetic experiment and comparisons with direct alignment and confidence weighted alignment partly address my original concerns. However, I still have two questions. First, the synthetic experiment shows that the decomposition works in the setting considered, but do the dominant components still preserve shared semantics when variation in the private factors is stronger than variation in the shared factors? Second, mismatches within the same class preserve class identity, so they do not fully test robustness to incorrect correspondences between semantically similar samples from different classes. Please also explain why accuracy on ALOI and Out Scene increases as the mismatch rate rises.

[Q1: Robustness under High Private-Factor Variance] To address the reviewer's concern, we conduct a controlled synthetic experiment where the variation of the private factors exceeds the shared shapes Varprivate=2×VarshareVar_{private}=2\times Var_{share}. As shown in Tab 2, the Dominant feature alone achieves 92.67% clustering ACC (closely matching the unified representation at 96.33%), whereas the Residual feature obtains non-zero performance (36.33%), confirming that the dominant components still preserve shared semantics even under stronger private-factor variation.

Tab 2 Clustering Performance of disentangled components.

SubspacesClustering ACC (%) ↑\uparrow
Unified Feature96.33
Residual Feature36.33
Dominant Feature92.67

[Q2: Semantic Similarity but Mismatches]

  • To further evaluate the robustness of DSMVC against incorrect correspondences between semantically similar samples from different classes, we investigate unconstrained cross-view shuffling (Random) from 10% to 90%. As shown in Tab 3, our DSMVC achieves the best results across all datasets and mismatch rates. Notably, while performance degrades monotonically as inter-class mismatch grows, DGAM consistently outperforms both Direct and Confidence baselines, validating the effectiveness of our method.
  • In our original setup, Semantic mismatch re-paired distinct instances within the same category across views, implicitly averaging intra-class features to compress intra-class variance and push each sample closer to its corresponding category centroids.

Tab 3 Clustering Accuracy under Inter-class Random Mismatches

(Dataset) MethodCleanRandom (0.1)Random (0.3)Random (0.5)Random (0.7)Random (0.9)
(COIL20) Direct76.08±2.0669.75±2.2054.33±2.7640.21±2.0327.84±1.3820.00±1.85
(COIL20) Confidence78.08±3.0769.58±2.7055.96±2.0939.58±1.2327.91±1.5419.87±4.04
(COIL20) DGAM79.71±1.7671.37±4.0456.38±1.8541.42±1.8428.42±1.7920.29±3.22
(ALOI) Direct93.90±0.3885.69±0.2868.21±0.8351.99±1.1234.09±1.6818.65±0.87
(ALOI) Confidence93.57±0.6585.51±0.4468.27±0.9751.75±1.1434.42±0.5318.87±0.51
(ALOI) DGAM94.53±0.5686.08±0.5968.77±0.7252.18±0.9534.50±0.8918.90±0.60
(100Leaves) Direct88.88±1.0180.37±0.2562.89±0.9643.63±1.1026.25±0.6213.96±0.47
(100Leaves) Confidence88.69±1.4380.90±0.9962.68±0.9743.51±0.9826.36±0.5613.85±0.24
(100Leaves) DGAM90.41±1.3881.49±1.8963.44±0.6144.54±0.7426.32±0.8014.16±0.45
(Handwritten) Direct96.96±0.1588.02±0.4666.84±4.2648.85±2.6534.87±1.0918.69±0.70
(Handwritten) Confidence96.20±0.4785.39±5.4368.42±4.2853.38±0.3235.02±0.7718.21±0.95
(Handwritten) DGAM96.67±0.3688.15±0.3270.86±0.2253.80±0.3035.45±0.4118.72±0.46
(Out-Scene) Direct68.62±0.6262.57±3.1151.90±1.9740.62±1.1929.14±0.9819.21±1.95
(Out-Scene) Confidence67.05±1.3762.51±2.8151.62±2.0040.50±0.7728.56±1.5818.69±1.48
(Out-Scene) DGAM69.63±4.8563.04±0.8552.19±0.7441.65±1.1229.84±1.1019.45±1.37

Thank you once again for your guidance and support.

Reviewer_G854OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes DSMVC, a disentangled streaming multi-view clustering framework that processes views sequentially without accessing historical raw data. The method uses spectral curvature to partition each incoming view's latent representation into a dominant component (primary structure) and a residual component (view-private variations). An Attribution-Guided Suppression Module (AGSM) identifies and penalizes feature dimensions that contribute excessively to reconstruction inconsistency via gradient-based attribution. A Discrepancy-Guided Cross-View Alignment Module (DGAM) calibrates cross-view correspondences by comparing discrepancies between naturally matched pairs and randomly permuted counterparts, producing per-instance reliability weights. Experiments on seven benchmarks under the streaming setting report competitive or superior performance against full-access and sequence-access baselines.

Critical strengths
  1. The problem setting is well motivated and practically relevant. Streaming multi-view clustering with no access to historical raw data is a realistic constraint, and the paper correctly identifies that naive cross-view alignment under this setting is vulnerable to spurious associations. The separation of dominant versus residual components is a sensible response, and the curvature-based truncation in Eq. (3)–(6) is a parameter-free way to avoid hand-tuning the partition boundary.

  2. The DGAM is the most original component. Constructing a permutation-based counterfactual to estimate per-instance correspondence reliability, then converting the discrepancy gap into a sigmoid weight in Eq. (17)–(18), is a clean and interpretable mechanism. The ablation in Fig. 10 showing that removing γ^w degrades performance, particularly on COIL20 and Out-Scene, supports the claim that uniform weighting is insufficient.

  3. The empirical evaluation is broad: seven datasets, eleven baselines split into full-access and sequence-access paradigms, three metrics, Gaussian noise robustness (Fig. 4), hyperparameter sensitivity (Fig. 5), view-order sensitivity (Fig. 8), incremental-view stability (Fig. 7), runtime comparison (Table 3), and a parameter-free partition baseline (Fig. 11). The efficiency results in Table 3 are a genuine practical advantage.

Critical weaknesses
  1. The theoretical grounding of the curvature-based partition is thin. Eq. (3)–(6) invoke the scree test (Cattell, 1966) and maximum discrete curvature, but the paper does not establish any property of the resulting partition, such as a bound on the energy retained by the dominant component or a guarantee that the elbow is well-defined. Since k^{v,} directly determines how much information is routed to the noise-prone residual branch, a more formal or at least empirical characterization (e.g., the distribution of k^{v,}/r across datasets and epochs) would strengthen this contribution.

  2. The AGSM attribution is computed from a single backward pass of J^v with respect to X̂_s^v (Eq. (12)), averaged over samples. This is essentially a first-order saliency measure, which is known to be sensitive to input scale and decoder nonlinearity. The paper does not discuss whether the threshold ā^v and the resulting mask M^v are stable across epochs or mini-batches; if M^v fluctuates, the suppression loss in Eq. (14) could oscillate and destabilize training. A diagnostic of mask stability over training would be valuable.

  3. The comparison in Table 1 is framed as "competitiveness against full-access methods," but DSMVC does not surpass DMVCS on several datasets under ACC and NMI (e.g., ALOI100 ACC: 87.85 vs 94.53; Cifar100 ACC: 99.38 vs 96.35 is favorable but Leaves NMI: 92.53 vs 90.41 is close). The text selectively highlights Handwritten and ALOI. A more balanced discussion of where DSMVC loses and why would improve credibility.

Questions for authors
  1. In Eq. (4), the rescaling y_i^v = (e_i^v − e_1^v)/(e_i^v − e_1^v) appears to have an identical numerator and denominator, which would make y_i^v = 1 everywhere and render the curvature in Eq. (5) undefined. Is this a typographical error, and if so, what is the intended normalization? This is important because the entire partition in Eq. (6) depends on it.

  2. In Table 3, DSMVC's inference time (11.12 ms on Handwritten) is higher than DSVC's (5.48 ms) and DFL-Net's (1408.08 ms is training, but inference is not directly comparable). Could the authors decompose the inference cost across the encoder, SVD partition, and DGAM to explain the source of this latency?

  3. The suppression mask M^v in Eq. (13) uses a mean threshold over dimensions. For datasets with very different view dimensionalities (e.g., Handwritten views range from 6 to 240 dimensions per Table 4), a mean-based threshold may behave very differently across views. Did the authors consider a percentile-based or scale-invariant threshold, and does the current choice interact with view dimensionality?

5 comments

[W1: Theoretical and Empirical Analysis of Curvature Partition] To rigorously characterize our curvature-based partition, we establish its theoretical analysis and empirical experiments as follows:

  • Existence of the Elbow: Since singular values of real-world data inherently exhibit exponential or power-law decay, the continuous curvature function 𝑐(𝑥) bounded on [0,1] guarantees the existence of an interior global maximum via the Extreme Value Theorem. This theoretically ensures that the optimal boundary kv,∗k^{v,*} is well-defined and deterministic.
  • Energy Bound of Dominant Components: Due to the orthogonality of singular vectors, the energy of dominant components satisfies 0<E(Zcv)<∣∣Zv∣∣F20 < E(Z_c^v) < ||Z^v||_F^2, which strictly prevents feature scale explosion while reserving a non-trivial capacity for residual components.
  • Empirical Distribution of kv,∗/rk^{v,*}/r: To support the theoretical analysis, we track the dimensional ratio kv,∗/rk^{v,*}/r and its corresponding energy across diverse datasets, showing that the ratio rapidly stabilizes within 12%–25% with 88%–97% of the total spectral energy. This confirms that our curvature partition adaptively isolates primary structures without manual tuning. For clarity, all relevant content has been added to Appendix B (P13-P14).

[W2: Mask Stability in AGSM] To thoroughly investigate AGSM's training dynamics and mask stability, we quantify mask consistency using the Mask Stability Rate, defined as MSR=1−FlipRatioMSR=1 - FlipRatio (Liu et al., ICML 2021), measuring the proportion of unchanged mask values across epochs. As shown in Table 1, MSR consistently exceeds 94.9% (with standard deviation ≤ 1.20%) across five datasets of varying scales. This stability stems from the consistent importance ranking across dimensions, even as gradient magnitudes decay during training. Moreover, we plot the evolution curves of the attribution threshold aˉv\bar{a}^v and the suppression loss across training epochs, both of which decrease smoothly toward convergence, further confirming the robustness of AGSM against aˉv\bar{a}^v and MvM^v. Detailed experiments are provided in Appendix D.6 (P17–P18).

Table 1: AGSM Training Dynamics and Mask Stability across Datasets

DatasetMSR(%)aˉv\bar{a}^v (Initial→Final)Initial LossFinal Loss
COIL2099.92±0.250.320→0.00778288.80327.29
ALOI94.99±1.200.060→0.015755.3719.63
100Leaves97.08±0.880.080→0.015903.4242.74
Handwritten98.34±0.260.220→0.0261777.7976.29
Out-Scene97.75±0.850.150→0.0124850.9827.40

[W3: Discussion against Full-Access Baselines] We appreciate the reviewer's suggestion to present a more balanced discussion against full-access methods. As an offline full-access framework, DMVCS can simultaneously access all views to model global cross-component relevance and incorporates explicit cluster-oriented contrastive objectives to enhance feature separability. In contrast, DSMVC operates under the strictly constrained streaming setting where historical raw data cannot be revisited, and learns unified features without explicit cluster-aware objectives. Despite these limitations, DSMVC still achieves highly comparable performance across all benchmarks, confirming the effectiveness of our framework in streaming scenarios. In future work, we will introduce discrimination-oriented objectives to further boost performance. For clarity, we have revised the title "Competitiveness against Full-Access Methods" to "Comparability against Full-Access Methods" in Section 3.2 (P7).

[Q1: Typographical Error] We thank the reviewer for the careful reading. In our manuscript, Eq. (4) is formulated as yiv=eiv−e1verv−e1vy^v_i=\frac{e^v_i-e^v_1}{e^v_r-e^v_1}. Since erv=1e^v_r=1 and e1v<1e^v_1<1, the denominator is strictly positive. Thus, yivy_i^v scales monotonically from 0 to 1 without degenerating to a constant, ensuring that the discrete curvature in Eq. (5) and the partition boundary in Eq. (6) remain well-defined and stable.

[Q2: Inference Time Decomposition] Following this suggestion, we decompose the inference time into the encoder forward pass, spectral partition, and K-means clustering (note that DGAM is only used during training and does not participate in inference). As reported in Table 2, the inference latency stems almost entirely from the SVD-based spectral partition (>88%>88\%), whereas feature encoding takes less than 0.3 ms. We have added the relevant content in Appendix D.1 (P15).

Table 2: Decomposition of Inference Time (ms)

DatasetEncoderSpectral PartitionK-MeansTotal
Handwritten0.27 (2.3%)11.44 (95.6%)0.26 (2.1%)11.97
Out-Scene0.28 (5.2%)4.82 (88.8%)0.33 (6.0%)5.43

[Q3: Impact of View Dimensionality on Mask Thresholding] We thank the reviewer for the question regarding the threshold choice under heterogeneous view dimensions. To address this, we compared our mean strategy against scale-invariant normalization (MinMax) and percentile variants on Handwritten, where view dimensions vary 40×40\times (from D4=6D^4=6 to D5=240D^5=240). As shown in Table 3, despite this discrepancy, our strategy yields a balanced active mask ratio (41.46%–48.16%) across views, showing that gradient attribution naturally scales with dimensional information density. Meanwhile, the mean strategy achieves high mask stability (98.34%) and superior clustering results without extra tuning. We have included these analyses in Appendix D.7 (P18).

Table 3: Comparison of Threshold Strategies in AGSM on Handwritten

StrategyV1 (216)V2 (76)V3 (64)V4 (6)V5 (240)V6 (47)MSR(%)ACC(%)
MinMax37.18±2.9331.76±3.0528.62±2.6944.12±2.7224.40±2.4134.87±3.9097.02±0.3896.20±0.30
Percentile (0.5)50.00±0.0050.00±0.0050.00±0.00±50.00±0.00±50.00±0.0048.94±0.0098.35±0.5696.21±0.33
Percentile (0.7)30.09±0.0030.26±0.0029.69±0.0033.33±0.0030.00±0.0029.79±0.0097.39±0.8796.37±0.52
Percentile (0.9)10.19±0.0010.53±0.0010.94±0.0016.67±0.0010.00±0.0010.64±0.0096.59±1.4196.25±0.33
Mean (Ours)48.16±0.3442.34±0.8943.50±0.7242.33±1.7141.46±1.0743.62±1.7198.34±0.2696.67±0.36

Altogether, we are very heartened by the Reviewer's deep dive into our manuscript, and your constructive comments have significantly sharpened the manuscript's clarity and technical rigor. These drove us to complete a series of targeted experiments and theoretical analyses, resulting in much deeper insights into our framework. We hope our thorough responses and additions can address your concerns, and we are happy to provide any further clarifications.

Dear Reviewer G854,

Thank you for your valuable insights and the time spent reviewing our work, especially during this busy period. We greatly appreciate it.

As we near the final day of the discussion period, we wanted to check in and see if our responses have addressed your concerns. We welcome your further queries and suggestions and are ready to provide any additional information or clarification you might need.

We understand that reviewing is a time-consuming task and greatly appreciate your commitment to advancing academic discourse. Our team is dedicated to improving our work based on your feedback, believing it will enhance our contribution to the field.

Thank you once again for your guidance and support.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.