- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Authors train a spectral frontend to map multispectral observations into pseudo-RGB images to use in frozen vision models, with extensions to segmentation, additional spectra and multimodal inputs.
Critical strengths
The classification experiments which the authors proposed, capacity RGB controls, and NIR interventions provide reasonable evidence that the frontend exploits additional spectral information.
Critical weaknesses
First, the contribution beyond existing spectral adaptation methods remains insufficient. As per my knowledge, previous established methods and models already combine spectral adapters with frozen visual backbones(many other variations of existing large/small models, including spectraDino). However, the main comparisons are against RGB probing and linear projection. Without matched comparisons to the closest methods. And the results demonstrate the usefulness of additional spectral information and trainable processing more than the actual merits of such design, which is a digression.
Second, the major segmentation gains can potentially be attributed to other architectural changes. Authors claimed a 19.9-point PASTIS-HD improvement introduces both PFU and VPT-Deep, without RGB+VPT-Deep control set. Plus the approximately 23.6-point PFU-I gain with the SAT backbone also replaces a linear decoder with U-Net (Tables 2 and 4). The comparisons cannot isolate the interface’s contribution from these confounds.
Another problem is that the proposed interface’s generality was not sufficiently backecd. PFU-D modifies the patch projection, departing from the external pseudo-RGB interface. Cross-backbone results are also inconsistent: on BEN-MM, the SAT backbone performance drops with PFU. Separately trained variants with different adaptation mechanisms do not demonstrate that one interface transfers reliably across backbones and sensors.
Questions for authors
Do you consider to include the comparisons with some close peer works that developed spectral adaptation methods? Can you fix the decoder and check how much is the improvement?