- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper addresses single-source domain generalization for cross-scene hyperspectral image (HSI) classification and proposes a transformation-conditioned discrete self-supervision framework (TCDS). To overcome the potential loss of discriminative information caused by uniformly enforcing view consistency in conventional self-supervised learning, the authors compute a “semantic preservation score” for different transformed views based on the prediction divergence (JSD) of the source-domain classifier and the spectral angle distance (SAM). The model then encodes the spectral and spatial views separately through discrete codebooks, aligns them at the semantic-index level using the Sinkhorn algorithm, and performs preservation-weighted consensus. In addition, the framework designs a preservation-dependent residual gating branch to supplement features not captured by the discrete codebooks, and introduces Lagrangian constraints to align semantics and balance codebook usage.
Critical strengths
- The motivation—that uniform consistency may suppress class-specific discriminative information—has a clear physical grounding. Figure 1 provides an intuitive explanation through the different effects of adjacent-band mixing and spatial smoothing on two types of materials, and the proposed preservation-weighted consensus is well targeted.
- The basic rigor of the experimental setup is commendable. On three cross-scene HSI benchmarks, target-domain labels are strictly isolated and do not participate in hyperparameter tuning. The dual codebooks and the residual branch yield considerable ablation gains on the PaviaU dataset.
Critical weaknesses
- The hyperparameter matrix in this paper raises suspicions of dataset overfitting. In the Houston13→Houston18 task, the training-time parameters change abruptly during inference through an unjustified “source-only calibration” (e.g., changes from 1.15 to 1.80).
- The core “semantic preservation score” relies heavily on the JSD divergence of the source-domain classifier . In domain generalization, inevitably carries source-domain-specific bias. If a transformation effectively removes source-specific noise and exposes truly cross-domain shared features, may instead assign a high JSD penalty because its prediction distribution changes, causing this high-quality transformation to be downweighted. This circular definition, at the underlying logic level, implicitly encourages the model to fit source-domain bias.
- The design of the residual gate has a structural disconnect in information flow. The gate is determined jointly by the preservation scores of all transformed views, but the residual feature it controls is extracted only from the identity view. This means that when the preservation of transformed views is very low, the gate amplifies the non-shared features of the source image itself. Figure 6(d) also confirms that in the interval with the highest preservation (smallest ), removing the residual still causes a 5.25% increase in error, indicating that the gate fails to achieve the expected dynamic compensation.
- The Lagrangian constraints proposed in the paper lack empirical support and have extremely loose boundaries. Taking the codebook balance constraint as an example, the authors set and , which in practice only requires the activation entropy to be greater than 2.56. This is equivalent to allowing the number of effective non-zero activation codes to shrink to about 13, and thus cannot substantially prevent codebook collapse.
Questions for authors
- In the Houston13→Houston18 task, why can the model abruptly change the training-time value to during inference through the so-called “source-only calibration”?
- According to Eq. (7), the gate is determined by the preservation scores of all transformed views, but the residual it controls comes only from the identity view. Does this mean that when the transformed views are severely distorted, the gate instead amplifies the features of the source image itself?
- Figure 6(d) shows that in the interval with the highest preservation, removing the residual still leads to a 5.25% increase in error. Please explain why this phenomenon contradicts the dynamic compensation design you claim.
Response to Reviewer a72q
Thank you so much for your positive and encouraging feedback. Below, we address your comments and suggestions in details:
Q1 & W1: Houston inference-time residual-scale change and source-only calibration.
A. The training-time value γr=1.15 is not fixed at inference because γr controls residual fusion rather than learned model weights. Its inference value of 1.80 is jointly selected with ρ and Tc using held-out Houston13 source data, without accessing Houston18 or retraining the model (Appendix F, Table 7).
Q2 & W3: Transformed-view preservation and identity-view residual gating.
A. The preservation scores assess how reliably transformed views support the discrete semantic consensus, whereas the identity-view residual retains discriminative cues omitted by the discrete codes. When transformed views preserve less class evidence, the gate therefore increases reliance on the current input's untransformed residual to compensate for the less reliable consensus evidence. At inference, this residual is computed from the current target sample, not a stored training-source image. The residual-separation constraint discourages overlap with shared semantics, while clipping bounds the gate coefficient. The gate thus adjusts the contribution of complementary information without assuming that every residual feature is domain-invariant.
Q3 & W3: Residual removal in the highest-preservation quartile and dynamic compensation.
A. High transformation preservation does not imply that discrete codes capture all discriminative information. Moreover, the gate decreases with preservation but is not forced to zero (βmin=0.03) on Pavia. Fig. 6(d) measures the effect of removing the entire residual branch within each preservation quartile, not the gate coefficient itself. The observed error increases upon removal are 15.49, 12.15, 6.14, and 5.25 percentage points across (Q1)–(Q4), respectively. Thus, the residual remains useful even in (Q4), whereas the observed benefit is larger in lower-preservation groups. This pattern is consistent with relative compensation, which does not require residual benefit to vanish at high preservation.
W2: Source-classifier JSD, beneficial transformations, and source-specific bias.
A. A change in source-classifier predictions may increase JSD and lower a transformation's preservation weight, but TCDS does not treat JSD alone as a rejection criterion. Eq. (2) combines prediction JSD with spectral-angle discrepancy to produce a continuous weight (aim), rather than discarding views outright. Fig. 6(a) evaluates this score with a separate source hold-out probe: balanced transformed-view error decreases from 14.73% in (Q1) to 1.40% in (Q4). On PaviaU$\rightarrow$Pavia Center, Full TCDS achieves 66.32% OA versus 65.3% with uniform consensus weights (Table 6). The score therefore has support from held-out discriminative evidence and target-domain performance, not merely from agreement with its source scoring classifier.
W4: Empirical support for Lagrangian constraints and codebook-collapse prevention under the relaxed code-balance bound.
A: The Lagrangian constraints target semantic agreement, residual separation, and code-use diversity, rather than uniform code activation. The code-balance bound (H(cˉ)≥logK−η) imposes a minimum entropy on aggregate code usage. With (K=96) and (η=2), it requires (H(cˉ)≥2.56), or an effective code count (exp(H(cˉ))≳13) when satisfied. It therefore excludes single-code collapse under that condition while allowing nonuniform codebook usage. It does not ensure that all 96 codes remain active. Table 8 shows that removing code balance reduces mean OA by 2.85/2.20/0.94 percentage points on Houston/Pavia/HyRANK.
Table 8. Cross-task component ablations (target OA, %). Results are mean$\pm$SD over three runs. Each variant follows the corresponding task-specific configuration.