acceptodds
Under review as a conference paper at ICLR 2027

Learning What and When to Transfer for Cross-Scene Hyperspectral Clustering

Abstract

Cross-scene clustering of hyperspectral imagery (HSI) asks an unlabeled source scene to help organize an unlabeled target scene into consistent land-cover groups despite scene-dependent spectral shifts. Existing methods typically align scene representations as a whole, although transferable land-cover semantics and scene-specific spectral variation may remain entangled. This makes it difficult to determine both what structure should be transferred and when target evidence should enter the transfer process. We propose disentangled transfer with recurrent admission (DTRA), which treats these two decisions as coupled parts of one recurrent transfer process. To learn what to transfer, it factorizes each representation into semantic and scene-specific branches, tests whether semantic assignments are preserved under cross-scene variation exchange, and summarizes the preserved semantic structure by balanced prototypes. These prototypes are then matched across scenes to initialize the target-oriented transfer state. To learn when to transfer, assignment confidence is combined with the exchange-induced assignment discrepancy to rank target regions and admit them progressively. The terminal target-oriented prototype state is carried into the next training round, so an admission decision also changes the prototype geometry used to judge subsequent transfer content. We further provide an assignment-preservation analysis that connects semantic deviation under cross-scene variation exchange to the assignment discrepancy reused by recurrent admission, and formulate training as recurrent prototype-state refinement under a unified clustering objective. Experiments on six directional tasks from Houston, Pavia, and HyRANK have conducted to validate the effectiveness and superiority of our DTRA method. The code is released at https://github.com/Sky-byte-box/DTRA_code.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Hyperspectral image clustering, Cross-scene learning, Disentangled representation learning, Recurrent transfer

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_X85uOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the problem of unsupervised cross-scene hyperspectral image clustering without any manual annotations. To overcome cluster distortion caused by the entanglement of genuine land-cover semantics and scene-specific spectral variations such as lighting or sensor shifts, the authors introduce DTRA. The proposed method factorizes representations into semantic and scene-specific components, verifies transferable semantic prototypes via cross-scene variation exchange, and scores target samples using assignment confidence and exchange-induced discrepancy to progressively incorporate reliable target evidence for prototype refinement. The refined prototype state is then recurrently fed back into subsequent training rounds to guide future transfer decisions. Experiments across six cross-scene tasks on the Houston, Pavia, and HyRANK benchmark datasets demonstrate that the method consistently improves clustering accuracy over existing approaches.

Critical strengths

The paper is well-structured and clearly motivated. It decomposes cross-scene HSI clustering into “what to transfer” and “when to transfer,” which makes the method easy to follow and the contribution well defined.

The empirical validation is broad and convincing. It covers six directional cross-scene tasks, compares against nine recent methods, and includes component ablation, parameter sensitivity analysis, and mechanism-level visualization.

Critical weaknesses

The reported improvements are not consistently convincing. On several tasks, such as H18→H13, PC→PU, and PU→PC, the gains over the second-best method are only 0.5–1.3 percentage points, while the standard deviations are as large as 4–7 percentage points.

The evaluation is limited to ACC only, with no NMI or ARI, which are standard for clustering. Moreover, ablation and parameter analysis are conducted only on a single task (PU→PC), so it is unclear whether the conclusions generalize to other transfer directions.

Questions for authors

see Weaknesses

3 comments
AuthorsOpenReview ↗

Response to Reviewer X85u

We thank the reviewer for the careful assessment and constructive questions. The following responses address the concerns and questions point by point.

W1: Reliability of small ACC gains under run-to-run variability.

A. Table 1 reports mean ±\pm SD over five independent runs. On H18→H13\mathrm{H18}\rightarrow\mathrm{H13} and PC→PU\mathrm{PC}\rightarrow\mathrm{PU}, DTRA exceeds the second-best methods by 0.63 and 0.49 percentage points in mean ACC, respectively, with standard deviations of 7.04 and 4.32 points. Although the run-to-run variability is relatively large on these two directions, the reported results reflect the observed mean ACC over five independent runs rather than statistically established improvements. On PU→PC\mathrm{PU}\rightarrow\mathrm{PC}, DTRA achieves a 1.28-point improvement with a considerably smaller standard deviation of 0.03 points. Overall, DTRA achieves the highest observed mean ACC across all six directions, with task-dependent improvement margins and run-to-run variability.

W2(a): Clustering evaluation beyond ACC.

A: We added normalized mutual information (NMI) and adjusted Rand index (ARI) for all six transfer tasks and all eleven baselines in Appendix A (Table 3). The complete results are below (%, NMI/ARI in each cell).

Table 3: NMI/ARI (%) comparisons on six directional cross-scene clustering tasks. Each entry lists NMI and ARI separated by ``/'', respectively. Bold indicates the highest value for both metrics in each task.

MethodH13$\to$H18H18$\to$H13PU$\to$PCPC$\to$PUDi$\to$LoLo$\to$Di
SGCNR26.67/13.3554.31/36.7155.46/37.3445.27/23.8530.35/13.2228.16/12.38
S3AGC52.26/33.8912.09/4.4249.13/31.5754.68/33.6234.95/14.5830.38/14.89
BGPC31.75/15.2453.46/33.2257.09/37.3845.45/25.9727.49/13.0629.31/13.25
SAPC47.17/28.1512.09/4.4265.87/49.0857.04/34.2537.98/18.7735.17/17.35
NCSC32.11/15.4538.99/19.7570.07/53.5166.57/47.3344.44/24.0152.27/29.71
SDST34.25/16.6939.07/22.1469.97/53.4267.75/50.2546.75/25.0553.01/31.09
AHSGC31.97/15.2638.91/18.0471.46/53.5667.35/47.3944.51/24.9852.44/31.03
SLCGC34.63/17.8537.42/17.9173.33/54.2563.56/43.8844.31/23.9441.12/19.89
ADSAC37.24/19.1331.93/14.4258.89/41.7239.04/20.4547.95/25.2555.36/32.88
MLUDA35.26/18.6948.67/28.5362.85/44.2159.55/39.6441.02/21.7732.84/17.16
DG-MDC54.55/36.8420.75/8.3963.85/48.6857.22/35.7747.95/25.2536.45/19.78
DTRA65.72/48.3256.43/36.8473.61/56.8670.24/51.1452.09/29.8864.72/45.24

DTRA has the highest NMI and ARI in every direction. For H13→H18\mathrm{H13}\rightarrow\mathrm{H18}, its NMI/ARI are 65.72/48.32 versus the second-best 54.55/36.84. For Lo→Di\mathrm{Lo}\rightarrow\mathrm{Di}, they are 64.72/45.24 versus 55.36/32.88. These permutation-invariant metrics supplement Hungarian-matched ACC and support the clustering results beyond that measure.

W2(b): Generalizability of ablations and parameter analysis.

A. We extended Table 2 and Fig. 4 from PU→PC\mathrm{PU}\rightarrow\mathrm{PC} to H13→H18\mathrm{H13}\rightarrow\mathrm{H18} and Lo→Di\mathrm{Lo}\rightarrow\mathrm{Di}, covering Houston, Pavia, and HyRANK. The expanded ablations are reported below (ACC, %).

Table 2: Component ablations and alternative-strategy comparisons on three cross-scene tasks (ACC, %).

VariantH13$\to$H18PU$\to$PCLo$\to$Di
Full DTRA68.9672.3762.40
Without factor separation53.6262.3657.55
Without variation exchange62.5764.3649.31
Without prototype OT63.6967.9759.66
Without admission ordering57.9261.3452.05
Without recurrent feedback60.5162.3548.17
Without conservative state screening62.6061.5756.66
Confidence-only (β=0\beta=0)62.4769.7452.72
Conventional consistency60.5367.2749.77

Every component removal reduces ACC on all three tasks, with task-dependent effects. Fig. 4 additionally examines τ\tau, ϵπ\epsilon_{\pi}, KK, and β\beta on these same three directions. The favorable regions support the shared defaults τ=0.20\tau=0.20, ϵπ=0.05\epsilon_{\pi}=0.05, K=4K=4, and β=1.0\beta=1.0. Sensitivity to KK is relatively weak on PU→PC\mathrm{PU}\rightarrow\mathrm{PC}. Together, these results provide cross-dataset evidence for the components and parameter choices.

have reviewed the authors' rebuttal. While the additional experiments do provide a clearer demonstration of the proposed method's performance advantages, given the overall novelty of the method, I have decided to maintain my current score.

We thank the reviewer for carefully reconsidering our response and acknowledging that the additional experiments provide clearer evidence of DTRA's performance advantages. We appreciate the reviewer's constructive feedback and thoughtful assessment, which have helped us improve the clarity and presentation of our work.

Reviewer_htgrOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies fully unsupervised cross-scene hyperspectral image clustering, where both source and target scenes are unlabeled and may exhibit scene-dependent spectral variations.

The authors propose Disentangled Transfer with Recurrent Admission (DTRA), a framework that jointly addresses what semantic information should be transferred and when target samples should participate in adaptation. DTRA separates semantic and scene-specific representations, evaluates assignment preservation through cross-scene variation exchange, and employs balanced prototypes with entropy-regularized optimal transport to establish cross-scene correspondence. It further introduces a confidence- and discrepancy-based progressive admission mechanism, with recurrent prototype updates across training rounds.

The method is evaluated on six directional transfer tasks constructed from the Houston, Pavia, and HyRANK datasets. The reported results show the highest mean clustering accuracy among nine compared methods on all six tasks, accompanied by a component ablation study and parameter sensitivity analyses.

Critical strengths
  1. Joint modeling of transfer content and target admission. The paper connects semantic transferability estimation with progressive target adaptation through recurrent prototype feedback. This design is relevant to fully unsupervised cross-scene clustering, where unreliable target assignments may otherwise reinforce incorrect correspondence.

  2. Competitive results across multiple scene pairs. DTRA reports the highest mean ACC on all six directional tasks in Table 1. The improvements are particularly substantial on H13→H18 and Lo→Di. The component ablations on PU→PC also provide evidence that several proposed mechanisms contribute to the reported performance.

Critical weaknesses
  1. Limited theoretical support for semantic transferability. Proposition 1 bounds the assignment discrepancy under a bounded change in normalized semantic representations. While this establishes a useful stability property, it does not by itself guarantee that stable assignments correspond to correct or transferable land-cover semantics. The connection between assignment preservation and semantic reliability therefore remains insufficiently established.

  2. Limited validation of component generality across transfer directions. The main comparison covers six directional tasks, but the component ablations and parameter analyses are concentrated on PU→PC. Moreover, task-specific initialization strategies are used for H13→H18 and PC→PU. Additional cross-task ablations and initialization-controlled comparisons would help determine whether the observed gains consistently arise from the proposed mechanisms rather than particular implementation choices.

Questions for authors

Q1. Proposition 1 establishes an upper bound on assignment discrepancy under bounded semantic perturbations. What additional evidence supports the use of low assignment discrepancy as an indicator of semantically correct and transferable target assignments, rather than merely stable but potentially incorrect predictions? Could the authors provide an empirical analysis relating this discrepancy to target clustering correctness? Q2. The ablation study is conducted on PU→PC only. Do the individual contributions of variation exchange, admission ordering, recurrent feedback, and state screening remain consistent across the other five transfer directions, particularly those exhibiting larger performance variance or smaller gains over the baselines? Q3. The implementation details describe specialized initialization configurations for H13→H18 and PC→PU. How were these configurations selected without using target ground-truth labels? Could the authors report the performance under a common initialization protocol, or quantify the improvement attributable to these task-specific choices?

1 comment
AuthorsOpenReview ↗

Response to Reviewer htgr

We thank the reviewer for the valuable feedback. Our responses to the concerns are provided below.

Q1 & W1: Assignment stability and semantic reliability under variation exchange.

A. Proposition 1 bounds assignment changes under bounded variation exchange. It does not equate low discrepancy with semantic correctness. DTRA therefore combines exchange-induced JS discrepancy and assignment entropy in the admission score (Eq. (10)), followed by recurrent prototype refinement and unlabeled state screening. To isolate the contribution of discrepancy beyond confidence, we added a matched confidence-only control (β=0\beta=0) in Table 2, retaining the same admission schedule and training settings. The results (ACC, %) are:

Below is excerpted from expanded Table 2 of the revised manuscript:

MethodH13$\to$H18PU$\to$PCLo$\to$Di
Confidence-only (β=0\beta=0)62.4769.7452.72
Full DTRA68.9672.3762.40

Full DTRA improves ACC by 6.49, 2.63, and 9.68 percentage points, respectively. The matched control shows that exchange discrepancy adds useful information beyond assignment confidence for target admission across the three datasets.

Q2 & W2. Generalizability of component contributions across transfer directions.

A. We expanded Table 2 from PU$\to$PC to H13$\to$H18 and Lo$\to$Di, covering Houston, Pavia, and HyRANK. The three-direction ablation results (ACC, %) are:

Table 2: Component ablations and alternative-strategy comparisons on three cross-scene tasks (ACC, %).

VariantH13$\to$H18PU$\to$PCLo$\to$Di
Full DTRA68.9672.3762.40
Without factor separation53.6262.3657.55
Without variation exchange62.5764.3649.31
Without prototype OT63.6967.9759.66
Without admission ordering57.9261.3452.05
Without recurrent feedback60.5162.3548.17
Without conservative state screening62.6061.5756.66
Confidence-only (β=0\beta=0)62.4769.7452.72
Conventional consistency60.5367.2749.77

Each component removal reduces ACC on all three directions, although the effect sizes differ. For example, without factor separation, H13$\to$H18 falls to 53.62%. Without admission ordering, PU$\to$PC falls to 61.34%. Without variation exchange or recurrent feedback, Lo$\to$Di falls to 49.31% or 48.17%. These results support the contribution of the proposed components across all three benchmark datasets, rather than only PU$\to$PC.

Q3 & W2. Label-free selection and impact of task-specific initialization.

A. Task-specific configurations were selected using cluster-assignment stability on unlabeled spectral data, without target labels or target ACC (Sec. 4.1). We added matched standard-initialization controls for H13$\to$H18 and PC$\to$PU, the only directions with specialized initialization. Table 4 in Appendix B reports ACC (mean ±\pm SD over five runs):

Table 4: Comparison of standard and task-specific initialization on two transfer directions (ACC, %, mean$\pm$SD).

TaskStandard initializationTask-specific configurationGain (pp)
H13$\to$H1865.05$\pm$1.9468.96$\pm$1.12+3.91
PC$\to$PU64.86$\pm$2.2266.76$\pm$2.32+1.90

All other training and admission settings remain unchanged. H13$\to$H18 uses eight cluster-specific sub-prototypes with soft-3 aggregation. PC$\to$PU uses a 12-dimensional representation with three-way over-clustered initialization. The PC$\to$PU control tests this complete configuration, not over-clustering alone. The other four directions use standard initialization.

Reviewer_SgyEOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents DTRA, a framework for unsupervised cross-scene hyperspectral clustering. The authors investigate two related questions: what information should be transferred between scenes and when target samples should be incorporated into training. The proposed method separates semantic and scene-specific representations and uses cross-scene variation exchange to assess the consistency of cluster assignments. Optimal transport is employed to align prototypes across scenes, while a progressive sample selection strategy and recurrent prototype updates are used to guide adaptation. Experiments are conducted on six transfer tasks constructed from three hyperspectral datasets.

Critical strengths
  1. The paper addresses an important problem in hyperspectral clustering. The distinction between transferable semantic information and potentially unreliable target-domain predictions provides a reasonable motivation for the proposed approach.
  2. The method considers both cross-scene feature consistency and the reliability of target samples. Their integration into an iterative adaptation procedure is an interesting aspect of the work.
  3. DTRA achieves the highest mean clustering accuracy on all six evaluated transfer tasks (Table 1). The improvements are substantial on H13→H18 and Lo→Di, and the ablation results indicate that several proposed components contribute to the performance.
Critical weaknesses
  1. The methodological novelty is not sufficiently clear. Feature disentanglement, consistency regularization, optimal transport, and progressive pseudo-label selection have been extensively studied. The paper would benefit from a more precise explanation of what is fundamentally new in the variation exchange and recurrent feedback mechanisms.
  2. The comparison with related transfer learning methods is limited. Most baselines are hyperspectral clustering methods, while relatively few explicitly address cross-scene adaptation. Comparisons with representative unsupervised domain adaptation methods would help establish the benefits of DTRA more convincingly.
  3. The ablation study does not fully support the main contributions. Table 2 reports results only for PU→PC. It remains unclear whether the proposed mechanisms are equally effective on other transfer tasks. Comparisons with simpler alternatives, such as conventional consistency regularization and confidence-based sample selection, would also be informative.
  4. The performance gains are inconsistent across tasks. Although DTRA obtains the best mean ACC in all six cases, its improvements on H18→H13 and PC→PU are only 0.63 and 0.49 percentage points, respectively. The relatively large standard deviations on these tasks make it difficult to assess the reliability of these gains.
  5. Some experimental settings and practical assumptions need further examination. The paper uses task-specific initialization strategies for certain transfer directions. Their selection criteria should be clarified. In addition, the experiments assume shared categories across scenes, while robustness to class imbalance and partially overlapping categories remains unexplored.
Questions for authors
  1. What distinguishes the proposed variation exchange mechanism from existing feature consistency and domain-invariant representation learning approaches? Similarly, what is the specific advantage of recurrent prototype feedback over conventional iterative pseudo-label refinement?
  2. Can the authors include comparisons with representative cross-domain clustering or unsupervised domain adaptation methods under the same experimental protocol?
  3. Can the authors extend the ablation study to additional transfer tasks and compare the proposed admission strategy with standard confidence-based sample selection?
  4. How were the task-specific initialization settings determined? Were all configurations selected without access to target labels?
  5. Proposition 1 provides a bound on assignment changes under small representation perturbations. How does this result justify the proposed semantic transfer mechanism more directly?
5 comments
AuthorsOpenReview ↗

Response to Reviewer SgyE—Part I

We sincerely thank you for your constructive feedback and expert insights. Based on your feedback, we have made the following responses:

Q1 & W1: Novelty of variation exchange and recurrent feedback.

A. DTRA introduces a transferability-guided recurrent mechanism that couples what to transfer with when to transfer, rather than merely combining existing techniques. Unlike conventional feature consistency or domain-invariant representation learning, Eq. (3) exchanges scene-specific codes across unlabeled scenes while retaining the original semantic code as input to the decoder. After re-encoding, DTRA evaluates whether the cluster assignments defined by Eq. (4) remain stable under such variation using Jensen–Shannon discrepancy. Crucially, this discrepancy serves both to assess assignment preservation and to regulate target admission in Eq. (10). Unlike conventional iterative pseudo-label refinement, admitted target evidence updates the target-oriented prototypes through Eqs. (11)–(13), and the resulting state is carried into the next training round when the unlabeled screening criterion is satisfied (Sec. 3.4). Thus, previous admission decisions directly influence subsequent transfer-content estimation. Table 2 further supports these mechanisms: removing variation exchange or recurrent feedback reduces ACC from 72.37% to 64.36% or 62.35%, respectively, on PU→PC\mathrm{PU}\rightarrow\mathrm{PC}. This coupling between transfer-content evaluation and recurrent target admission constitutes the central methodological distinction.

Q2 & W2. Cross-domain comparisons.

A. We have expanded Table 1 from nine to eleven comparison methods by adding two independently implemented label-free adaptations inspired by MLUDA and DG-MDC, alongside the existing ADSAC baseline. All eleven baselines are evaluated on the same six directional tasks without using source or target labels for training or model selection. The newly added results, together with ADSAC and DTRA for reference, are summarized below (ACC, mean ±\pm SD over five runs).

Below is excerpted from expanded Table 1 of the revised manuscript:

MethodH13$\to$H18H18$\to$H13PU$\to$PCPC$\to$PUDi$\to$LoLo$\to$Di
ADSAC45.80$\pm$5.3734.46$\pm$0.6961.48$\pm$3.3143.75$\pm$0.5247.38$\pm$3.1853.84$\pm$1.06
MLUDA43.38$\pm$5.7747.32$\pm$3.4664.11$\pm$0.1660.72$\pm$0.9342.55$\pm$1.7839.57$\pm$2.58
DG-MDC61.04$\pm$0.0525.79$\pm$1.0467.77$\pm$6.2857.78$\pm$3.4847.38$\pm$6.3842.73$\pm$1.32
DTRA68.96$\pm$1.1252.21$\pm$7.0472.37$\pm$0.0366.76$\pm$4.3251.84$\pm$1.6462.40$\pm$1.89

DTRA achieves the highest mean ACC across all six tasks among the eleven baselines. Specifically, on H13→H18\mathrm{H13}\rightarrow\mathrm{H18} and Lo→Di\mathrm{Lo}\rightarrow\mathrm{Di}, DTRA obtains 68.96% and 62.40% ACC, exceeding the corresponding second-best results of 61.04% (DG-MDC) and 53.84% (ADSAC), respectively. We have also added two clustering maps to Fig. 3 for qualitative comparisons with MLUDA and DG-MDC on PU→PC\mathrm{PU}\rightarrow\mathrm{PC}, providing complementary visual evidence of the clustering results.

AuthorsOpenReview ↗

Response to Reviewer SgyE—Part II (continued)

Q3 & W3. Ablations and simpler alternatives.

A. We have expanded Table 2 from PU→PC\mathrm{PU}\rightarrow\mathrm{PC} to H13→H18\mathrm{H13}\rightarrow\mathrm{H18} and Lo→Di\mathrm{Lo}\rightarrow\mathrm{Di}, covering Houston, Pavia, and HyRANK. We also compare DTRA with confidence-only selection (β=0\beta=0 in Eq. (10), retaining the same admission schedule) and conventional consistency based on assignments from two augmented views. The complete results (ACC, %) are shown below.

Table 2: Component ablations and alternative-strategy comparisons on three cross-scene tasks (ACC, %).

VariantH13$\to$H18PU$\to$PCLo$\to$Di
Full DTRA68.9672.3762.40
Without factor separation53.6262.3657.55
Without variation exchange62.5764.3649.31
Without prototype OT63.6967.9759.66
Without admission ordering57.9261.3452.05
Without recurrent feedback60.5162.3548.17
Without conservative state screening62.6061.5756.66
Confidence-only (β=0\beta=0)62.4769.7452.72
Conventional consistency60.5367.2749.77

All six component removals and both simpler alternatives reduce ACC on each task. Compared with confidence-only selection, full DTRA improves ACC by 6.49, 2.63, and 9.68 percentage points on H13→H18\mathrm{H13}\rightarrow\mathrm{H18}, PU→PC\mathrm{PU}\rightarrow\mathrm{PC}, and Lo→Di\mathrm{Lo}\rightarrow\mathrm{Di}, respectively. The corresponding improvements over conventional consistency are 8.43, 5.10, and 12.63 points. These results support the contribution of exchange-induced discrepancy beyond assignment confidence and demonstrate that conventional view consistency does not recover the same gains.

W4. Performance variability.

A. Table 1 reports mean ±\pm SD over five independent runs. On H18→H13\mathrm{H18}\rightarrow\mathrm{H13} and PC→PU\mathrm{PC}\rightarrow\mathrm{PU}, DTRA exceeds the second-best methods by 0.63 and 0.49 percentage points in mean ACC, respectively, with DTRA standard deviations of 7.04 and 4.32 points. These standard deviations characterize run-to-run variability, whereas the reported gains quantify differences between method means. Neither statistic alone establishes statistical significance. We therefore report these as observed mean improvements without claiming statistically established superiority on these two transfer directions. Across all six tasks, DTRA achieves the highest observed mean ACC, although the improvement margins and run-to-run variability differ by transfer direction.

AuthorsOpenReview ↗

Response to Reviewer SgyE—Part III (continued)

Q4 & W5. Initialization and label-free selection.

A. The task-specific initialization configurations were determined based on unlabeled spectral characteristics using a predefined label-free clustering-stability criterion, without accessing target labels or target ACC. Specifically, H13→H18\mathrm{H13}\rightarrow\mathrm{H18} uses eight cluster-specific sub-prototypes with soft-3 aggregation to accommodate intra-class spectral variation, while PC→PU\mathrm{PC}\rightarrow\mathrm{PU} adopts a 12-dimensional representation with three-way over-clustered initialization to capture local cluster structure. The other four tasks use standard initialization. Ground-truth labels are used only after prediction for Hungarian-matched ACC and visualization (Sec. 4.1).

W5(a): Robustness to class imbalance.

A: We further evaluate DTRA against five representative baselines (ADSAC, MLUDA, DG-MDC, SDST, and SLCGC) under target-class imbalance, using the same sampled subsets for all methods (Appendix C, Table 5). The results (ACC, %) are:

Table 5: Class-imbalance ACC (%) for PU$\to$PC and Lo$\to$Di at four target-class max:min ratios. The results refer to controlled target subsets, rather than the original full-scene test sets.

TaskRatioADSACMLUDADG-MDCSDSTSLCGCDTRA
PU$\to$PC1:11{:}160.9263.6267.3270.2970.4571.92
3:13{:}158.6761.9664.9567.9567.6269.77
5:15{:}155.4358.6461.7664.1762.8766.83
10:110{:}150.8754.7857.3757.7454.3960.20
Lo$\to$Di1:11{:}153.2839.2542.5252.9844.2961.87
3:13{:}150.8238.4240.7950.0742.1559.12
5:15{:}147.9835.1139.4147.4639.4755.93
10:110{:}144.3132.7336.5842.6335.8048.37

DTRA attains the highest ACC across all eight settings. From 1:11{:}1 to 10:110{:}1, its ACC decreases from 71.92% to 60.20% on PU$\to$PC and from 61.87% to 48.37% on Lo$\to$Di. These results support competitive overall clustering under class imbalance, without asserting minority-class robustness from ACC alone.

W5(b): Robustness to partially overlapping categories.

A: We additionally test partial category overlap against the same five baselines (Appendix D, Table 6), with the original cluster counts fixed at C=7C=7 (PU$\to$PC) and C=12C=12 (Lo$\to$Di). The results (ACC, %) are:

Table 6: ACC (%) under controlled category overlap on PU$\to$PC and Lo$\to$Di. ∣Cs∩Ct∣|\mathcal C_s\cap\mathcal C_t| denotes the number of shared classes.

TaskShared (source/target)ADSACMLUDADG-MDCSDSTSLCGCDTRA
PU$\to$PC7 (7/7)61.4064.2667.6970.6771.0572.28
5 (6/6)55.9358.3561.8263.4364.8965.48
3 (5/5)45.8044.2852.1349.9651.7650.85
Lo$\to$Di12 (12/12)53.7339.8442.9753.2544.1762.25
9 (10/11)47.8836.9340.5746.8041.2854.15
6 (9/9)42.2929.1235.9238.7632.0741.62

DTRA leads under full and moderate overlap. At the lowest overlap, DG-MDC exceeds DTRA on PU$\to$PC (52.13% vs. 50.85%), while ADSAC exceeds DTRA on Lo$\to$Di (42.29% vs. 41.62%). These results identify a limitation under severe category mismatch rather than establishing open-set clustering. In both robustness studies, labels are used only to create evaluation subsets and compute ACC, not to train or select the models.

Q5. Connection between Proposition 1 and semantic transfer.

A. Proposition 1 establishes that, under bounded changes in the normalized semantic representation induced by cross-scene variation exchange, both the assignment deviation and its Jensen–Shannon divergence are bounded (Eq. (5)). This provides a theoretical basis for evaluating assignment stability under scene-specific variation. Specifically, Eq. (3) exchanges scene-specific factors while retaining the original semantic factor, and Eq. (4) measures the resulting assignment discrepancy. DTRA directly incorporates this discrepancy into the admission score (Eq. (10)), delaying samples whose assignments are sensitive to scene-specific changes. The admitted samples then progressively refine target prototypes through Eqs. (11)–(13). Thus, Proposition 1 connects assignment stability under variation exchange to the progressive selection and refinement of transferable semantic structure.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.