acceptodds
Under review as a conference paper at ICLR 2027

Decomposing Cross-Modality Failure in Diabetic Retinopathy Grading

Abstract

Colour and near-infrared (NIR) fundus imaging depict the same retina through different image representations. A diabetic-retinopathy (DR) grader trained on colour photographs must therefore preserve disease knowledge while accommodating a new channel structure and intensity distribution. Across eight encoder configurations and five paired seeds, zero-shot colour-to-NIR transfer degrades sharply without a corresponding loss of confidence. We factorise the source intervention into channel representation and marginal intensity alignment: each recovers part of the loss, while their combination raises ResNet-50 external QWK from 0.08 to 0.54 and improves every tested backbone. A frozen-model control and layer-wise probes show that much of the failure can be reduced without learning a cross-domain mapping. Predictive uncertainty remains weak for shift detection, whereas Mahalanobis distance reaches OOD AUROC 0.93 and remains informative for residual grading errors. The resulting decomposition separates an input-addressable component from what remains for adaptation, acquisition, or new data.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
cross-modality transfer, domain shift, diabetic retinopathy, retinal imaging, representation mismatch, out-of-distribution detection, medical imaging, domain adaptation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jHUDOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper investigates cross-modal failures in diabetic retinopathy classification from color fundus to near-infrared images. A normalization strategy combining channel adjustment and intensity alignment is proposed, with experiments across eight encoder architectures and additional analyses of feature shifts and lesion retention.

Critical strengths
  1. The paper addresses an important practical challenge in transferring medical image classification models across imaging modalities

  2. Extensive experiments across eight encoders, multiple random seeds, and an external dataset, together with uncertainty and feature analyses, provide strong empirical support.

Critical weaknesses
  1. The methodological novelty is limited to me, as the proposed normalization mainly combines established preprocessing techniques. Its differences from existing methods require further clarification. 2.The ablation studies demonstrate the effects of individual transformations but do not fully establish the claimed mechanisms of cross-modal failure.
  2. The comparisons with domain adaptation methods are insufficient, as recent cross-modal adaptation approaches are not adequately evaluated.
  3. The generalizability of the conclusions remains unclear, as experiments focus on a single cross-modal transfer scenario.
  4. Some lines overlap with graphical elements, affecting the clarity of module boundaries and information flow.
Questions for authors
  1. How are the effects of channel representation and intensity distribution distinguished from other changes introduced by the transformation?
  2. What are the main methodological contributions beyond existing preprocessing techniques?
  3. Why are recent cross-modal adaptation methods not included, and how is the competitiveness of the CycleGAN baseline ensured?
Reviewer_iNsvOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates why diabetic-retinopathy graders trained on colour fundus images perform poorly on NIR images. It separates the shift into channel-representation and intensity-distribution effects and proposes a simple harmonisation procedure. The experiments show that both factors contribute to the failure and that their combination substantially improves NIR performance across multiple backbones.

Critical strengths
  1. The factorised analysis is clear and more informative than treating the problem as generic domain shift.
  2. The evaluation is comprehensive, including multiple backbones, five seeds, adaptation baselines, a frozen-model control, and an independently graded external dataset.
  3. The paper appropriately distinguishes shift detection from error detection.
  4. The method is simple, transparent, and appears to preserve pathological features better than learned image translation in the reported audit.
Critical weaknesses
  1. The method requires representative unlabelled target images. Although the appendix evaluates references estimated from 50, 200, and all target-training images, robustness to very small, non-representative, or temporally drifting samples remains unclear.

  2. The source and target datasets differ in population and acquisition device as well as modality. Therefore, the experiments cannot fully establish that modality alone causes the observed performance gap. A paired colour/NIR cohort would provide stronger evidence.

  3. Performance remains limited after harmonisation, particularly for mild DR, so the current results do not establish clinical readiness.

  4. The technical novelty of the transformation is modest; the main contribution is the diagnostic analysis rather than a new adaptation method.

Questions for authors
  1. How robust is the reference histogram to temporal drift, device changes, or a non-representative target sample?
  2. Could the authors report variability over multiple randomly sampled 50-image reference sets?
  3. Can the authors clarify which conclusions concern the controlled input transformations and which concern the imaging modality itself?
  4. How well calibrated is the model on NIR images after harmonisation?
Reviewer_5K5xOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies why a diabetic retinopathy grader trained on RGB fundus photographs fails when applied to near-infrared fundus images. The authors decompose the cross modality shift into two input level factors: channel representation and marginal intensity distribution. They replace RGB with a replicated green channel and align source image intensities to an unlabeled NIR reference distribution. These interventions are evaluated with several encoder architectures and pretraining schemes, including ImageNet, self supervised, and retinal foundation models. The experiments use EyePACS for source training, OLIVES for unlabeled target statistics and proxy analysis, and a locked NIR-EXTERNAL cohort for external evaluation. The authors also compare against preprocessing, test time adaptation, domain adaptation, and CycleGAN baselines. They find that the proposed deterministic harmonisation improves external QWK substantially, from 0.08 to 0.54 for ResNet-50, and reaches performance comparable to CycleGAN. They further show that predictive confidence is a weak signal for this modality shift, while feature space distances, especially Mahalanobis distance, are more useful for shift and error detection.

Critical strengths

The analysis is fairly complete. The paper does not only report a final score, but separates channel representation, intensity alignment, architecture, pretraining, confidence, feature distance, and adaptation baselines. The frozen model control and the external NIR cohort make the main diagnosis more convincing. The finding that confidence can remain high under a serious cross modality failure is useful. The comparison between output based uncertainty and feature space distance is also informative for evaluating silent failures.

Critical weaknesses
  1. The paper mainly provides an analysis rather than a new method. The central intervention is a simple input preprocessing procedure based on channel replication and intensity quantile alignment. The other methods used in the paper are existing adaptation, normalization, translation, or uncertainty techniques. The paper gives a useful account of the failure, but it does not provide a substantial new learning method or a general adaptation algorithm. The paper should make a clearer case for why this analysis is sufficiently novel and important for the main conference. In particular, it should discuss whether the proposed harmonisation is intended as a practical solution or mainly as a diagnostic tool. If it is intended as a practical solution, the paper should provide stronger evidence that it works in realistic deployment settings.

  2. The analysis is somewhat direct and largely demonstrated through a closed evaluation setup. The two factors studied by the paper, channel representation and marginal intensity statistics, are intuitive explanations for a shift from color to NIR images. The experiments show that manipulating these factors helps on the selected cohorts, but they do not establish that the same decomposition is generally valid for cross modality medical imaging. Most of the conclusions are drawn from a fixed source dataset, a fixed target modality, and a small number of target cohorts. The target informed transformation also requires access to an unlabeled sample from the target acquisition. It is therefore unclear how well the conclusions would hold under a new device, a different hospital, a changing target distribution, or another modality transition. The paper would be stronger with an independent external setting, a second modality transition, or an evaluation designed to test whether the decomposition transfers beyond this particular RGB to NIR case.

Questions for authors

Do the authors view deterministic harmonisation as a practical deployment method or primarily as an analysis tool? If it is intended for deployment, what happens when the target device or target intensity distribution changes after the reference histogram is estimated?

Can the authors evaluate whether the proposed decomposition generalizes beyond the current RGB to NIR setting? For example, can the same analysis be applied to another device, hospital, disease dataset, or cross modality retinal task?

How much does the method depend on the availability and representativeness of the unlabeled OLIVES target sample? Please report sensitivity to target sample size and to a mismatch between the reference sample and the deployment distribution.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.