- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
CoMask addresses strong correlations between neighboring hyperspectral bands by moving self-supervised masking from input space to spectral components. It fits and fixes a spectral basis on unlabeled pretraining data, removes selected component coefficients at chosen spatial locations, and recombines the remaining components into a degraded input retaining all wavelength channels. An encoder–decoder reconstructs the original normalized spectra.
Critical strengths
-
A well-motivated use of low-dimensional spectral structure for self-supervised learning. Figure 1 directly supports the motivation that masked hyperspectral bands remain readily predictable through simple cross-band relationships. CoMask uses spectral decomposition to redefine what information the pretext task removes while retaining wavelength coverage. The decomposition determines the input corruption, while the reconstruction target remains the original spectrum. This simple design translates domain knowledge into a meaningful pretraining strategy that integrates readily into existing masked autoencoding frameworks.
-
Controlled experiments across data scales support the method’s practical value. The evaluation spans a single airborne scene, a satellite pretraining collection, and SpectralEarth with approximately 8.8 billion pixels. In Table 3, CoMask improves over the in-house MaskedSST-Tube baseline on all six tasks by 0.94–2.58 percentage points in micro-IoU, using the same ViT-Tiny backbone, pretraining data, training budget, and segmentation head. This controlled comparison is my primary evidence supporting acceptance. Table 4 further investigates the gains through matched input degradation and controls using random bases, band-permuted bases, and noise perturbations.
Critical weaknesses
-
Dominant masking lacks a clearly specified treatment of basis scaling. The dominant strategy ranks components by their mean coefficient magnitude, interpreting larger coefficients as stronger spectral contributions. However, NMF permits reciprocal rescaling of a basis vector and its coefficients without changing the reconstruction, potentially changing this ranking. Unless basis scales are standardized, coefficient magnitude need not reflect actual spectral contribution. The paper describes input normalization but does not clearly specify corresponding normalization of the basis vectors. This leaves the interpretation and scaling sensitivity of dominant masking unresolved, particularly for the Houston and EnMAP experiments that use this strategy.
-
The roles of spectral component learning and spatial completion remain insufficiently distinguished. CoMask shares one component mask across all corrupted locations within a sample while retaining intact spectra elsewhere. Appendix D shows that corrupting every location places most Houston configurations and all tested EnMAP configurations below the randomly initialized linear-probe baseline, suggesting that intact spectra are important to the current method. Although the authors report this dependence, its mechanism remains unclear. Because the same components are missing across corrupted locations, intact spectra may provide essential spatial cues for their recovery. The analysis does not yet distinguish learning relationships among spectral components from spatial completion using intact spectra, or establish their respective contributions. The standard MAE practice of retaining visible inputs does not, by itself, explain this dependence.
Questions for authors
- Are basis vectors normalized, with compensating coefficient rescaling, before dominant selection? Is selection invariant to reciprocal basis–coefficient rescaling? If not, why should mean coefficient magnitude represent spectral contribution?
- Spatial masking: Why share one component mask across spatial locations? Is there a comparison with independently sampled masks at comparable information retention that could clarify the role of intact spectra and the performance drop when every location is corrupted?
- Scope of transfer: How would the spectral basis, component count, and masking strategy adapt to different band counts, wavelength ranges, or missing bands? Would this require pretraining on target data, or could existing encoder weights be reused directly? Please distinguish the reusability of the masking strategy from cross-sensor transfer of pretrained representations.
- Suggested mechanism controls: Have you compared reconstructing the original spectra from full low-rank approximations without component masking, and reconstructing the full low-rank approximations from component-masked inputs? These controls could help separate the contributions of component completion and decomposition-residual recovery.
3 comments
Response to Reviewer brbZ (2/3): Q2
Q2 (W2).
Spatial masking: Why share one component mask across spatial locations? Is there a comparison with independently sampled masks at comparable information retention that could clarify the role of intact spectra and the performance drop when every location is corrupted?
Sharing one mask within a sample is a design choice, not a CoMask requirement. For the results reported in the paper, every sample draws its own component mask, but within a sample, every location removes the same components. However, CoMask supports independent masking at each location too. We ran that version at matched input NRMSE (0.803 against 0.796). The difference is within seed noise, so CoMask works with either rule:
| Houston2018, ViT-Tiny, 8 pretraining seeds | LP-OA |
|---|---|
| Shared mask per sample (paper) | 56.16 ± 3.00 |
| Independent mask per location | 55.43 ± 2.62 |
| Input-space control | 49.46 ± 2.86 |
Independent masks are 0.73 points below the shared mask (3 of 8 seeds above it) and 5.97 points above the input-space control (7 of 8 seeds).
On the role of intact spectra, your weakness raises three points.
Intact spectra may provide essential spatial cues for their recovery.
They do. From a corrupted pixel alone, a linear predictor recovers CoMask's removed content at R² 0.66. With the mean of the tile's intact pixels and the nearest intact pixel, it reaches 0.94.
Because the same components are missing across corrupted locations.
This is not the reason. Independent masks remove different components at each location and score the same as the shared mask (table above), so the completion does not depend on neighbours missing the same components.
The analysis does not yet distinguish learning relationships among spectral components from spatial completion using intact spectra.
The full-corruption runs separate them. With every location corrupted (s = 1.0, 3 pretraining seeds), independent masks probe at 47.94 ± 2.04 and the shared mask at 40.96 ± 6.09, against a random-init floor of 46.45 and 58.22 ± 1.19 for the fixed configuration with intact locations. Without intact spectra, neither pattern learns a useful representation, so spatial completion is necessary for the pretext task. It is not what distinguishes CoMask from input-space masking. The input-space baselines keep the same fraction of locations intact, and at the same NRMSE CoMask exceeds them by 6 to 7 percentage points.
These runs rule out two explanations of the mechanism without establishing it. The dependence on intact spectra is real, it does not come from the shared mask pattern, and reconstruction is not the bottleneck, since the full-corruption models reconstruct well (validation loss 0.0047 and 0.0075, against 0.0027 for the fixed configuration) and their probe accuracy is at or below the floor. Why intact anchors are needed for the representation to transfer remains open, as you note, and the revision adds both tables to Appendix D and states this.
Response to Reviewer brbZ (3/3): Q3, Q4
Q3.
Scope of transfer: How would the spectral basis, component count, and masking strategy adapt to different band counts, wavelength ranges, or missing bands? Would this require pretraining on target data, or could existing encoder weights be reused directly? Please distinguish the reusability of the masking strategy from cross-sensor transfer of pretrained representations.
This is an important question for the next steps of CoMask. The goal of this paper was to show the concept and the harness for masking in component space, so every experiment was built around that. The scope of transfer splits into the two parts you name, and they have different answers.
The masking strategy transfers as it is. One configuration (NMF, K = 8, dominant, c = 0.75, s = 0.70) runs on Houston2018 (48 bands, visible and near infrared) and on EnMAP (200 bands, 420 to 2450 nm), and the same harness runs on Spectral Earth with K = 10 and random selection. What changes between sensors is the basis, because the basis is tied to the band grid. A new sensor needs a basis fitted on its own unlabeled pixels, which costs a small share of pretraining (Section 4), and K can be set from reconstruction error on those pixels, as we did for Spectral Earth. For missing bands, the basis can be restricted to the available bands and the coefficients recomputed. We have not tested that case.
Pretrained representations do not transfer across sensors, and that is a property of the encoder, not of the masking. The weights are tied to the tokenizer's band grid, as in any spectral ViT, so a new sensor needs pretraining on its own data. We do not test cross-sensor reuse of weights and make no claim about it. Within one sensor, the paper does test transfer of the pretrained representation to a new region (EnMAP, pretrained on 90 European scenes and finetuned on Mexico City), and CoMask exceeds the input-space baseline there at every encoder scale (Table 2). The revision states this distinction in Section 7.
Q4.
Suggested mechanism controls: Have you compared reconstructing the original spectra from full low-rank approximations without component masking, and reconstructing the full low-rank approximations from component-masked inputs? These controls could help separate the contributions of component completion and decomposition-residual recovery.
We ran both, and they separate the two contributions cleanly. Component completion accounts for nearly all of CoMask's gain, and recovery of the decomposition residual accounts for almost none of it:
| Houston2018, ViT-Tiny, 8 pretraining seeds | Input | Target | LP-OA |
|---|---|---|---|
| CoMask-NMF (paper) | component masked | original spectrum | 56.16 ± 3.00 |
| Low-rank target | component masked | low-rank approximation | 54.52 ± 2.20 |
| Residual recovery only | full low-rank approximation, nothing masked | original spectrum | 50.01 ± 3.14 |
| Input-space control | zeroed tokens | original spectrum | 49.46 ± 2.86 |
The low-rank target removes the residual from the task. The encoder still has to fill in the masked components, and it probes at 54.52, which is 1.64 points below the paper's version and within seed noise (3 of 8 seeds above it). It is still 5.06 points above the input-space control at 8 of 8 seeds. The second control removes the masking and keeps only the residual. The encoder gets an almost intact spectrum (NRMSE 0.012) and has to reconstruct what the rank-8 approximation dropped. That gives 50.01, the same as the input-space control (+0.55, 6 of 8 seeds). So the residual on its own does not teach anything useful, and seeing a low-rank-filtered spectrum does not help either.
This confirms what we claim the pretext task does. The gain comes from removing components and training the encoder to reconstruct them. Whether the target has the residual in it or not changes little. Reviewer YJ18 asked for a related control, low-rank input with ordinary band masking. That one gives 48.82 ± 0.67, again level with the input-space control, so both sets of controls give the same result. We will add all three arms to Table 4.
Thank you again for the questions. They led to controls that make the attribution stronger than it was in the submission.
Response to Reviewer brbZ (1/3): Q1
Thank you for the careful reading and for the questions. We ran the controls you proposed: a new selection rule for Q1, two masking arms for Q2, and both targets from Q4, all on Houston2018 at ViT-Tiny with 8 pretraining seeds at the paper's input NRMSE of 0.79. In every arm CoMask exceeds the input-space control.
Q1 (W1).
No to both, and thank you for catching this. The finding changes how the paper describes dominant selection, and nothing else. CoMask's contribution is to mask in a fitted component basis instead of in input space, and every result in the paper was produced by one fixed rule on one released basis, so every number is reproducible as printed. The rule itself does not need to identify the components with the most physical signal. It needs to be consistent, so that the degradation is reproducible and its amount is set by the configuration (c, s), not by the rule. Dominant by magnitude is one rule the harness implements. Random selection is another (used for Spectral Earth), scale-invariant dominant is now a third, and further rules can be added for a given application without changing the method.
The gain does not depend on the rule. A scale-invariant version, ranking by |a_k| times the norm of basis vector k, exceeds the input-space control by 4.50 percentage points at the same input NRMSE (0.790 against 0.796):
The two rules select the same components in only 13.4% of Houston tiles. The 2.21-point difference between them is smaller than the seed standard deviation (paired t = -1.4, 3 of 8 seeds positive), and the invariant rule exceeds the control at 7 of 8 seeds.
On the paper's wording, the NMF basis vectors are not normalised (norms span a factor of 9.5 on Houston2018 and 36 on EnMAP), so the paper's rule ranks by coefficient magnitude under the solver's scaling, and that ranking is not invariant to reciprocal rescaling. Coefficient magnitude therefore does not represent spectral contribution. The revision describes the rule as ranking by coefficient magnitude under the released basis, and reports the invariant variant beside it.