acceptodds
Under review as a conference paper at ICLR 2027

CRED: Class-Specific Reasoning Over Typed Evidence with Reliability-Hardness Prioritized Knowledge Distillation for Multisource Remote Sensing Image Classification

Abstract

Joint classification of hyperspectral imagery (HSI) and LiDAR/SAR data demands the effective exploitation of complementary information across heterogeneous sensors. However, existing fusion methods primarily model modality-specific features or fused representations, while conventional distillation methods directly use teacher predictions as supervision, making it difficult to identify discriminative information and effective distillation signals for different classes and samples. To address these limitations, we propose CRED, a framework for Typed Reasoning over Multimodal Evidence with Reliability- and Hardness-Prioritized Knowledge Distillation. Specifically, Typed Multimodal Evidence Construction (TMEC) separately encodes spectral, spatial, and complementary modality information, organizes heterogeneous sensor representations into explicitly typed evidence, rather than directly collapsing them into a single fused feature representation. Class-Specific Typed Evidence Budgeting (CSTEB) models interactions among evidence types using Mamba, employs type-specific class queries to assess their support for each candidate class, and combines the resulting scores using class- and sample-specific weights. Finally, Reliability- and Hardness-Prioritized Distillation (RH-PD) constructs teacher reliability and student hardness indicators from spatial-block held-out prediction probabilities, and prioritizes distillation samples accordingly, transferring trustworthy teacher knowledge to difficult samples that benefit most from additional supervision. Experiments on three public HSI–LiDAR/SAR datasets demonstrate that CRED consistently outperforms existing methods across multiple evaluation metrics, achieving OA values of 94.06%, 80.60%, and 92.77% on Houston2013, Berlin, and Augsburg, respectively.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
multimodal learning, representation learning, hyperspectral image classification, remote sensing, knowledge distillation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_owekOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

To address the limitation of existing distillation methods that directly adopt teacher predictions as supervision, the proposed method seeks to identify discriminative information and assign tailored distillation signals across different classes and samples. The reported results demonstrate advantages over methods that do not leverage foundation models.

Critical strengths

This paper investigates how different modalities contribute to individual predictions and which teacher signals ought to be transferred to the student model. This work is beneficial for distillation research that applies foundation models to modality-fusion tasks. The experimental results demonstrate the advantages brought by the foundation model and the designed distillation scheme.

Critical weaknesses
  1. Only one foundation model is adopted in the experiments, while the other competing methods do not rely on foundation models. The authors are suggested to evaluate the proposed distillation scheme on multiple foundation models to better demonstrate its effectiveness.
  2. Compared with other distillation methods should be included.
  3. Comparisons of model size, training speed and inference speed should be supplemented.
  4. Can the proposed distillation scheme be extended to other modality-fusion tasks?
  5. The manuscript requires further revision. In particular, several critical technical issues are not clearly described
Questions for authors

Compared with so many distillation methods, what are the main distributions of the paper? The theoretic discussion and experimental comparison should be provided.

Reviewer_PugPOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper presents an algorithm for pixel-wise land-cover classification from remotely sensed HSI data when fused with coregistered LiDAR or SAR data. Also there is some knowledge distillation. This paper introduces a system for this training called CRED and uses some ideas called TMEC, CSTEB, and RH-PD that supposedly make things work better.

Critical strengths
  • The method claims to perform better than some other methods.
Critical weaknesses
  • The large system of components claims to work better but the reasoning why any of it should work or provide some benefit is unclear
  • Also there is knowledge distillation from a foundation model or something, but its not clear what this foundation model is.
Questions for authors
  • What is the foundation model that is being distilled in this approach?
Reviewer_MpRTOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper studies multi-modal fusion (combining different remote sensing modalities) to classify land cover, with the added capability of learning from pretrained teachers. The key idea is that different information sources are useful for different land cover classes. The paper also proposes a distillation procedure that prioritizes examples where pretrained teachers appear reliable and the student struggles. Three land cover benchmarks are evaluated, and performance improvements range from under one percentage point to 2-3% over baselines.

Critical strengths
  1. The problem is well motivated: multi-sensor distillation and fusion is an important problem in EO, but how useful a given sensor is can vary across locations on the Earth and land-cover types. Explicitly learning when a sensor helps is interesting. I also think the teacher guidance experiments are interesting.
  2. The paper is generally well-written.
Critical weaknesses

I find several weaknesses that prevent me from recommending acceptance at this stage. I sort these by novelty and lack of significance of results.

  1. I am reasonably certain that the proposed method (CRED) is mostly combining methods used in previous work: separate feature encoders + mamba, learned fusion weights, selective distillation, etc. Moreover, this makes the method incredible complex (relevant for point 2). From a quick search, adaptive modality distillation already exists for the same modalities the authors work with (HSI + LiDAR) [1], and cross-fitting has been proposed for distillation before. It is challenging to see what is truly new to the paper, other than this combination and perhaps a minor addition.

  2. The comparisons in the paper's table do not seem to differentiate itself from prior work. MPT-MKD ([2]) is a very closely related method that authors cite and discuss, which is good. But the main benchmark table omits the baseline. I see that the ablation with spatial fold is present in Figure 4, but the ablations don't show whether this added complexity is necessary. The authors don't compare spatial splits against ordinary random splits, or their class-wise fusion against a simple learned weighting method. This is also reinforced in Table 2 where all this complexity oftentimes leads to very incremental gains (often under 1%).

[Minor]:

  1. While I find the writing to be generally understandable, the quality of the figures (and sizing) is very poor for ICLR. I recommend rejection at the moment, but am willing to increase my score if the authors are open to (i) clearly justifying if the proposed method is novel and important, and (ii) justifying the complexity in the method.
  2. Given the very incremental improvements, I think it would be beneficial to see some notion of variation in the results over different random seeds or another important axis.

[1] Gordon, Lucia, et al. "Multimodal Fusion Strategies for Mapping Biophysical Landscape Features." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024. [2] Kong, Yi, et al. "Multi-modal Prompt Tuning with Modified Knowledge Distillation for the Joint Classification of Hyperspectral Image and LiDAR Data." IEEE Transactions on Geoscience and Remote Sensing (2026).

Questions for authors
  1. In figure 4, do the authors have any insights on the dip in accuracy as the number of folds are increased?

  2. Can the authors provide more information on these benchmark datasets used? I have never heard of the public multi-modal datasets mentioned in Table 1 before. Are these just different land cover rasters of varying sizes? Why these specific regions? Why not any established land cover classification dataset in the machine learning for remote sensing community? Is there a citation for these datasets?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.