acceptodds
Withdrawn from ICLR 2027

More Should Be Better: Incremental Self-Distillation Multimodal Brain Tumor Segmentation with Missing Modalities

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multimodal magnetic resonance imaging provides complementary evidence for brain tumor segmentation, yet one or more modalities are frequently unavailable in clinical practice. A practical model must therefore remain effective for any available modality subset. Existing missing-modality methods largely optimize robustness to modality removal, but overlook the converse question: does the model reliably benefit when an additional modality becomes available? We argue that robustness alone is insufficient and introduce the principle of non-negative modality utility, i.e., enriching a valid modality subset should not worsen its information content for segmentation. Based on this principle, we propose a complementarity-guided incremental self-distillation framework. Specifically, it reformulates robust multimodal representation learning with missing modalities as a progressive multimodal self-distillation problem. Based on the non-negative modality-utility hypothesis, we organize modality subsets into a distillation path along which modality utility progressively increases, allowing representations learned from fewer modalities to gradually approach the representational capacity of the complete-modality setting. Compared with directly learning a joint representation from a restricted modality subset, our incremental self-distillation strategy better preserves modality-specific discriminative information and improves adaptation to diverse missing-modality configurations. Extensive experiments across diverse missing-modality settings demonstrate the effectiveness and robustness of the proposed framework.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multimodal Brain Tumor Segmentation, Missing Modalities, Medical Image Segmentation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_kK1qOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies brain tumor segmentation from arbitrary non-empty subsets of four MRI modalities, focusing on whether adding a modality benefits the same learned model. CISD combines shared and modality-specific encoding, a greedy path of nested subsets, adjacent feature distillation, full-to-partial distillation, and a supervised modality-utility penalty, with lesion-presence prediction and mask gating included in the final system. The paper provides an optimal-risk ordering argument and evaluates Dice and HD95 across all 15 modality subsets on BraTS 2024 and BraTS 2018, supplemented by component ablations and qualitative comparisons.

Critical strengths
  1. The utility regularizer gives an enriched state a corrective gradient when its segmentation loss exceeds the detached loss of its predecessor. This adaptation ties regularization directly to supervised segmentation quality and specifies which state receives the direct gradient.
  2. On BraTS 2024, the reported mean Dice improvements over MSTKDNet are 1.41, 8.90, and 9.28 percentage points for WT, TC, and ET across all 15 modality configurations. The final system also has the lowest regional mean HD95 among the compared methods, providing evidence of improvements in both overlap and the reported distance metric.
Critical weaknesses
  1. The central claim that MU promotes beneficial modality enrichment lacks a direct empirical test. Eq. 2 establishes an ordering of optimal predictors, while Table 2 reports aggregate segmentation metrics without comparing held-out risk ordering with and without MU. Calculating from Table 1, adding T2 to {FLAIR, T1, T1c} reduces the three-region mean Dice. This thresholded-Dice decrease does not directly contradict the soft-Dice-plus-BCE risk hypothesis, but the missing risk measurements leave the intended property unverified.
  2. CCSD (Xie et al., 2026) already combines shared-specific encoding, HMSD, cosine-guided nested subsets, and adjacent distillation from richer to poorer states. The proposed CISD changes deletion-based path construction to greedy addition, uses Bernoulli alignment at the bottleneck, and adds MU. These are identifiable differences, but Table 2 neither isolates the addition rule against CCSD’s deletion rule nor tests random or fixed paths, leaving the claimed benefit of the new complementarity-guided path unsupported.
Questions for authors
  1. At 400 epochs, what improvement does the complete objective in Eq. 12 provide over SEG when both configurations use identical, explicitly specified lesion-presence supervision and gating?
  2. What are the held-out mean differences in the loss of Eq. 9, with uncertainty estimates, for all 28 single-modality additions between non-empty subsets under matched models trained with and without MU?
  3. With the network, distillation losses, MU, and training budget held fixed, how does the greedy addition rule compare with CCSD’s deletion rule and a random ordering?
Reviewer_i2fvOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper addresses brain-tumor segmentation with missing MRI modalities and introduces the concept of non-negative modality utility, proposing that enriching a valid modality subset should not degrade segmentation performance. It proposes Complementarity-Guided Incremental Self-Distillation (CISD), which constructs a sample-adaptive modality-enrichment path using feature cosine distance and progressively distills representations between adjacent modality subsets. The method is evaluated on BraTS 2024 and BraTS 2018, reporting improvements across missing-modality scenarios.

Critical strengths
  1. Highlighting the converse of missing-modality robustness and ensuring that adding a modality reliably benefits the model rather than causing multimodal negative transfer is a compelling and useful perspective.

  2. Handling all non-empty modality combinations within a single unified network without subset-specific branches or inference-time teachers is practically useful for clinical deployment.

  3. Reporting results across all 15 non-empty combinations of four modalities provides clear insight into how different modality compositions affect Whole Tumor (WT), Tumor Core (TC), and Enhancing Tumor (ET) segmentation.

Critical weaknesses
  1. The framework largely stitches together shared/specific encoders, hierarchical feature pyramids, self-distillation, and ranking losses. The technical components are heavily reused from prior progressive/hierarchical distillation literature (e.g., HMSD and ISD). This makes the contribution incremental.

  2. The authors formulate non-negative modality utility R∗(S)≤R∗(T)R^{\ast}(S) \le R^{\ast}(T) based on optimal population risk over hypothesis spaces HS\mathcal{H}_S versus HT\mathcal{H}_T. However, their proposed Modality-Utility (MU) loss operates as a sample-wise surrogate along selected adjacent transitions during training. Consequently, it does not mathematically guarantee monotonicity across arbitrary nested subsets T⊂ST \subset S.

  3. Using feature cosine distance between deepest modality features as a proxy for "complementary task information" is heuristic and poorly justified. Features can diverge widely simply due to modality-specific appearance shifts (T1c contrast enhancement vs. T2 hyperintensity) rather than containing actual complementary tumor information.

  4. The experiments mainly ablate wrt to SEG, HMSD, ISD, HMSD+ISD, and ISD+MU. It does not convincingly compare the proposed complementarity-guided path against random or fixed paths. The core path-selection mechanism remains unvalidated.

  5. Although 14 comparisons are included, many are relatively old, and several closely related recent arbitrary-missing-modality and self-distillation approaches are absent from the primary comparison.

  6. Evaluating across BraTS 2018 and BraTS 2024 does not prove generalization, as both share identical protocols, preprocessing, and tumor pathologies (glioma MRI setting).

  7. The reported standard deviations are across modality configurations rather than independent training runs, and the primary results appear to rely on a single run/split.

  8. In a batch-training setup with batch size 1, dynamically changing distillation graphs per sample/iteration introduces massive gradient variance and indexing complexity. If the path changes per sample, how are the adjacency losses batch-computed efficiently without severe memory bottlenecks?

  9. The final system relies on auxiliary lesion-presence prediction to suppress complete tissue masks. Because false-positive/empty mask mismatches incur massive HD95 penalties, improvements cannot be cleanly attributed to CISD without a direct gating ablation.

Questions for authors
  1. What percentage of all nested pairs T⊂ST \subset S actually achieve improved Dice or decreased loss when evaluated post-training, both with and without MU regularization?

  2. How does the cosine-distance path perform compared to random or fixed subset-enrichment orderings?

  3. What are the exact Dice and HD95 scores of CISD without the lesion-presence inference gating/suppression module, especially for TC and ET?

  4. Given that the distillation path is sample-adaptive, how does this affect batch training stability, and what is the training time complexity?

  5. How does CISD compare with the most recent closely related missing-modality/self-distillation methods under an identical backbone and training protocol?

  6. Please report results over multiple random seeds or data splits with confidence intervals or mean ± standard deviation, rather than variability across modality configurations.

Reviewer_pvLdOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies brain tumor segmentation under arbitrary missing MRI modalities, argues that prior work is removal-centric, and introduces a "non-negative modality utility" principle (enriching the input should not increase task risk). It proposes CISD: a shared–specific encoder with one shared decoder; a sample-adaptive nested modality path built greedily from cosine distances between deepest-level features; incremental self-distillation (ISD) between adjacent path states via a Bernoulli KL on bottleneck features; a modality-utility hinge regularizer (MU) on per-sample task loss; and an HMSD term adapted from CCSD. On BraTS 2024 (1350 cases, 80/20 split) it reports mean Dice over the 15 modality subsets of 83.32/62.94/62.51 (WT/TC/ET) versus 81.91/54.04/53.23 for the strongest of 14 baselines, together with the lowest mean HD95, plus BraTS 2018 results in the appendix.

Main concern behind my recommendation: a data-integrity problem, not merely an under-documented protocol. Several baseline results in Tables 1–4 appear not to have been produced by running the published methods, and the paper gives no account of how they were obtained — it never states epochs, crops, splits, code or post-processing for the 14 baselines, nor whether the numbers were retrained or copied. The tables show several red flags. (i) On BraTS 2018, RFNet obtains 69.83/64.88/46.54 average Dice (WT/TC/ET), roughly 16/12/11 points below the 85.67/76.53/57.12 reported in RFNet's official repository (average over its three released BraTS 2018 splits); RFNet is the weakest of all 14 methods on WT here, so the claimed margins are measured against a handicapped baseline. (ii) In Table 3 (HD95, BraTS 2024) the TC and ET rows of several baselines are almost identical (M2FTrans 14/15 entries, RFNet 13/15, MSTKDNet 11/15), including RFNet, which is the reference for the headline "−12.73 mm ET" claim. (iii) Several baselines show clinically implausible modality behaviour that even flips between datasets (e.g. MSTKDNet TC Dice with T1 alone vs. T1c alone: 34.84 vs. 65.76 on BraTS 2024, but 83.94 vs. 68.32 on BraTS 2018). Any one of these could be dismissed as an undocumented protocol, but taken together — numbers far below a baseline's own released results, near-duplicate HD95 rows across regions with very different boundary statistics, and modality asymmetries that reverse between datasets — they point to results that were not obtained by executing the methods as described. Since the paper's central evidence is the comparison with these baselines, I cannot treat the reported state-of-the-art gains as established. Further concerns (ablation does not test the final model; an undocumented inference-time gating step; missing direct competitors) are detailed below.

Critical strengths

Missing-modality work is indeed removal-centric, and the utility-monotonicity viewpoint is, to my knowledge, new for this problem and clinically meaningful (a model that gets worse when a contrast scan arrives is a deployment hazard).

The nested path reduces subset coverage to M − 1 adjacent transfers with no external teacher, no subset-specific branches, and no inference-time overhead.

Recomputing all 28 adjacent nested transitions from Table 1, CISD violates non-negative Dice utility in only 6 of 84 region-level transitions, all ≤ 0.58 points (within test noise), whereas several baselines violate it often and severely (e.g. MouseGAN++ 17/84, worst −15.35; ShaSpec 11/84, worst −15.0). This is the most direct evidence for the title claim, yet the paper never computes it.

Critical weaknesses

1.What the paper says. Only three statements bear on this: an 80%/20% partition "following existing methods (Zhu et al., 2026)", "the public preprocessing protocol of existing methods (Zhu et al., 2026)", and, for BraTS 2018, "trained on 228 cases and evaluated on 57 grade-stratified holdout cases, using a 160×192×128 center crop" (Appendix C). The training details in Appendix A (400 epochs, Adam, lr 1e-4, batch size 1) are given only for CISD. Nothing is said about the epochs, crops, augmentation or checkpoint selection of the 14 baselines, whether their numbers were retrained under this split or copied from prior papers, or whether the public splits released by the baseline authors (e.g. RFNet releases three BraTS 2018 splits) were used. The 228/57 holdout is a new split. Evidence of a problem. RFNet on BraTS 2018. Table 4 gives RFNet 69.83/64.88/46.54 average Dice, versus 85.67/76.53/57.12 in RFNet's official repository (mean of its three released splits): 16/12/11 points lower, which a different random split of the same dataset can hardly explain. RFNet is the lowest of all 14 methods on WT average here (below ShaSpec 72.36, MouseGAN++ 73.92, Hyper-GAE 73.94), although it is generally a strong baseline on BraTS 2018. CISD's own BraTS 2018 averages (85.34/75.43) are at or below RFNet's published WT/TC averages (splits differ, so indicative only), so the apparent superiority on this dataset may largely reflect a weak RFNet reproduction. Duplicated TC/ET rows (Table 3, HD95, BraTS 2024). The TC and ET rows share 14 of 15 values for M2FTrans, 13/15 for RFNet, 11/15 for MSTKDNet, 8/15 for ACN, 6/15 for Hyper-GAE and 5/15 for RMBTS (Table 1 also shows 5/15 for M2FTrans). TC and ET have different boundary statistics and very different empty-mask rates, which under the 373.13 mm empty-mask penalty should push these rows apart. RFNet is the reference for the headline "−12.73 mm ET" claim. Implausible modality asymmetries. On BraTS 2024, ACN (58.74 vs. 24.66), SMUNet (64.26 vs. 30.32) and LCKD (35.91 vs. 27.51) report much higher TC Dice with T1 alone than with T1c alone, contrary to the well-established role of T1c for the tumor core, and MSTKDNet's T1/T1c asymmetry flips between BraTS 2024 and 2018 (TC: 34.84 vs. 65.76 → 83.94 vs. 68.32). This is consistent with modality-column mix-ups or numbers copied from sources with different conventions. Without a clear protocol, these tables cannot support a state-of-the-art claim. The authors would need to provide the provenance of every baseline number (retrained vs. copied), identical split / preprocessing / post-processing / HD95 empty-mask convention and training budget for all methods, and the baselines' results on their own public splits.

2.Table 2 is run for 200 epochs, CISD in the main tables for 400, and the baselines' epochs are never stated (W1). The final model is the full Eq. (12) objective (SEG+HMSD+ISD+MU) plus presence gating (Fig. 3 shows "+ISD+MU" and "Ours" as distinct columns), but this configuration is not among Table 2's five rows. The best ablation row (61.10 Dice / 61.33 mm) is far from the final model (69.59 Dice / 41.53 mm, from Tables 1 and 3) and would not even beat the baselines (MSTKDNet 63.06 / 52.18 mm; SMUNet 61.20 Dice), so epochs, gating or the unablated full combination account for the jump to "state of the art" without attribution. The paper's own numbers also show HMSD+ISD is worse than either term alone, yet the final objective keeps HMSD at the largest weight (λ_H = 1).

  1. CCSD (Xie et al., 2026; the source of HMSD), "No Modality Left Behind" (Zhu et al., 2026; whose protocol the paper follows) and M3AE (Liu et al., 2023) are cited but absent from all comparison tables, while older and weaker methods are included.

4.Appendix D calls the Fig. 4 curves "illustrative" and "constructed to peak … at the configured value", uses the future tense ("will be averaged"), and the Fig. 4 values at the configured setting (Dice ≈ 83/50/50, HD95 ≈ 11/19/19 mm) do not match the main results (83.32/62.94/62.51; 5.32/60.19/59.09). This may only be a wording or reduced-budget issue, but as written it is unclear whether Figs. 4–6 are measured results and they cannot justify λ_H, λ_C, λ_U.

5.Single-seed results with no per-case confidence intervals or paired tests, so differences such as 60.44 vs. 60.50 (Table 2) carry no significance. No validation split or checkpoint-selection rule is described, so it is unclear how hyperparameters and thresholds were chosen. On BraTS 2018 CISD is not best on TC Dice (75.43 vs. M2FTrans 77.64) or WT HD95 (8.54 vs. SMUNet 7.00), so "state of the art" should at most be scoped to BraTS 2024.

Questions for authors

Please refer to the weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.