acceptodds
Withdrawn from ICLR 2027

Hierarchical Gating Mixture-of-Experts for Mixed-Domain Medical Image Segmentaiton

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Learning a unified model that effectively captures domain-specific segmentation patterns remains challenging in mixed-domain medical image segmentation (MD-MIS), where images collected from diverse domains exhibit substantial heterogeneity despite sharing consistent anatomical structures. While Mixture-of-Experts (MoE) holds strong potential for mixed-domain medical image segmentation through expert specialization, existing MoEs suffer from limited feature expressiveness and unstable expert allocation, leading to inadequate domain-discriminative characterization of implicit task heterogeneity and intensified expert competition or collapse. To address the aforementioned challenges, we propose Hierarchical Gating Mixture-of-Experts (HG-MoE), introducing a hierarchical gating framework comprising two complementary mechanisms for effective expert specialization. Firstly, Micro-level Feature Gating is proposed to enhance feature expressiveness through an input-adaptive non-linear gating modulation, thereby enhancing domain-discriminative characterization of implicit task heterogeneity. Secondly, Macro-level Expert Gating is introduced to promote stable task-specific expert allocation through Top- routing guided by domain-aware expert load distributions, thereby mitigating expert competition and collapse. Extensive experiments on three public datasets across diverse domain scenarios demonstrate the consistent superiority and versatility of HG-MoE. HG-MoE improves Dice by up to under the mixed-domain setting, while achieving improvements of in Dice and in region-level F1 under the single-domain setting, respectively. Source code is publicly available at https://***/HG-MoE.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Mixture-of-Experts, Gating Mechanism, Mixed-Domain Medical Image Segmentation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_LGrAOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes Hierarchical Gating Mixture-of-Experts (HG-MoE) for joint medical image segmentation across heterogeneous, known domains. Built on Swin-U-Net, the intended approach combines input-adaptive feature modulation with domain-guided expert allocation. The first component aims to improve the representations available to the experts, while the second uses domain-related information to select and combine experts. The method is evaluated on mixed-domain organ segmentation using FLARE22 and AMOS22, and separately on lesion segmentation using MSWAL. The evaluation includes comparisons with existing methods, ablations of the two gating components, and additional two-dimensional and three-dimensional visualizations.

Critical strengths

Learning a shared segmentation model while accommodating differences between data sources is a meaningful problem. The paper considers feature representation and expert allocation together, rather than treating routing as an isolated operation. The idea that better representations may support more useful expert selection is plausible and worth investigating.

Critical weaknesses
  1. The mixed-domain experiments focus primarily on abdominal organ segmentation, with substantial overlap in anatomical structures and segmentation targets across the selected datasets. Although the datasets differ in imaging modality and provenance, the evaluation does not clearly establish what specific difficulties these differences create for a shared segmentation model or how the proposed domain-aware mechanism addresses them. The current benchmark therefore provides limited evidence for the necessity of the proposed design and its ability to handle domain heterogeneity.
  2. The approach primarily augments an existing segmentation backbone with domain-aware processing and expert allocation, followed by joint training on mixed-domain data. While this is a reasonable design, it currently appears more like a combination and extension of established mechanisms than a clearly differentiated methodological contribution. The experiments also do not establish what substantial advantage the two-level gating provides over simpler domain-conditioned alternatives. Further evidence is therefore needed to show that the added mechanisms deliver benefits commensurate with the paper’s claimed contribution.
Questions for authors
  1. How should the Top-KK aggregation in Eqs.(7)-(8) be reconciled with the all-expert aggregation described in Appendix~A.3.1? Please specify the value of KK, whether routing is performed per token or per sample, and whether the router uses HH or H^\hat{H}. A consistent forward-pass description should also explain the roles of the bottleneck task gate and task-specific segmentation heads, together with the complete training objective.
  2. What benefit does the proposed hierarchy provide over a shared Swin-U-Net, a standard Top-KK MoE with conventional load balancing, and a simpler domain-conditioned router? Please clarify whether initialization, training data, domain supervision, output heads, and model capacity are controlled in Table~5. The discussion should also explain the operational differences from M4oE\mathrm{M}^{4}\mathrm{oE} and SegMoTE. Direct comparisons with SegMoTE's reported scores would require matching the prompting and evaluation conditions.
  3. The macro-level router relies on a predicted domain identity at inference. Please report domain-recognition accuracy and explain how uncertain or incorrect predictions affect expert selection. Comparing routing based on ground-truth and predicted domain identities would help determine whether the segmentation gains depend on reliable domain recognition and how much performance is lost when that assumption fails.
Reviewer_ajHSOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes HG-MoE for mixed-domain medical image segmentation. It combines a Swin-UNet backbone with Micro-level Feature Gating for feature modulation and Macro-level Expert Gating for domain-conditioned Top-K expert selection. Experiments cover joint training on FLARE22 and AMOS22, single-domain evaluation on MSWAL, and ablations of the two gating modules.

Critical strengths

Training a unified model on heterogeneous medical images is practically relevant. The proposed modules are conceptually simple, and the paper includes both mixed-domain and single-domain evaluations.

Critical weaknesses
  1. The motivation is weak, and the problem setting is inconsistent —The paper presents the absence of sample-level domain labels as the core difficulty of MD-MIS (Intro, Fig. 5), yet the domain classifier is itself trained with domain labels (Eq. 6). In practice, modality (CT vs. MRI) and source are essentially free information, so the setting is closer to multi-dataset / partially-labeled universal segmentation. No comparison with per-domain models is given, so the need for a unified model is not established. The mixed-domain evidence also rests on a single pair of abdominal datasets (FLARE22 + AMOS22), which is not sufficient to support the general MD-MIS claim.

  2. Limited novelty and insufficient analysis of the MoE claims: the modules resemble gated FFNs and domain-conditioned MoEs, and their distinctions are unclear. No direct evidence supports improved domain discrimination, balanced expert utilization, or collapse prevention.

  3. Insufficient experiments and metrics; the main results appear not comparable. In the mixed-domain setting, Dice is the only metric compared against baselines; Recall and mIoU are reported only for HG-MoE. No boundary metric (HD95, NSD) is given, although improved boundaries are claimed repeatedly. Key baselines are missing (nnU-Net under the mixed setting, MultiTalent, a multi-head Swin-UNet without MoE, a Swin-UNet with standard MoE routing). Full per-organ results are not provided anywhere, including the appendix, so the reader cannot verify how the averages were computed.

Questions for authors

Q1. Fig. 5 describes training data without sample-level domain annotations, whereas Eq. 6 requires domain supervision. What defines a domain, and which domain labels are available during training and inference? How does HG-MoE compare with models trained independently per domain?

Q2. The Micro-level Feature Gating closely resembles a GLU / gated FFN, while the Macro-level Expert Gating resembles domain-conditioned MoE routing. What are the essential differences from these existing designs?

Q3. Which routing mechanism was actually used in the experiments: Top-K selection in Sec. 3.2 or aggregation over all experts in Appendix A.3.1? Please report K and explain how encoder MoE layers obtain domain information from the bottleneck classifier.

Q4. Please provide direct evidence for the MoE claims: per-domain expert-load statistics, feature-separability analysis, and a comparison with standard routing plus load-balancing loss.

Q5. Please provide full per-organ Dice for all methods on FLARE22 and AMOS22, and reconcile “mean Dice,” “FG Dice,” “Macro Dice,” and “Overall Average” across Tables 1–4. Are all methods evaluated using the same metric definitions and aggregation?

Reviewer_o3PLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Hierarchical Gating Mixture-of-Experts (HG-MoE) for mixed-domain medical image segmentation, aiming to better capture domain-specific patterns while reducing expert competition and collapse. The method introduces two components: a micro-level feature gating module that uses nonlinear transformations and input-adaptive modulation to enhance feature representations, and a macro-level expert gating module that uses domain-aware Top-K routing to select specialized experts. The framework is built on a Swin-Unet backbone and evaluated on three medical image segmentation datasets, covering both mixed-domain and single-domain settings. The experimental results show improvements over the selected baselines, and the ablation studies suggest that both gating components contribute to the overall performance.

Critical strengths
  1. The paper addresses a relevant challenge in medical image segmentation, where images from different domains may have substantially different distributions. The idea of using expert specialization to capture domain-specific patterns within a unified model is reasonable and potentially useful, especially when domain labels are not available at inference time.

  2. The proposed method reports consistent improvements over the selected baselines on both mixed-domain and single-domain segmentation tasks. The ablation study also suggests that the two gating components contribute to the performance gains.

Critical weaknesses
  1. The experimental baselines are relatively outdated, with the most recent methods in the main mixed-domain comparison coming from 2024. There have been several relevant MoE-based medical image segmentation methods published in 2025 and 2026, such as MambaHoME [1], SegMoTE [2] and TA-MoSC [3]. These approaches share important motivations with HG-MoE, particularly the use of specialized experts to handle data heterogeneity and improve segmentation performance. Although their specific architectures and experimental settings differ, I think a more detailed comparison with these recent methods is necessary to better understand the advantages of HG-MoE over existing MoE-based approaches.

  2. I am not fully convinced that the proposed hierarchical gating framework introduces sufficiently novel mechanisms. The micro-level feature gating appears to be a relatively straightforward combination of nonlinear feature transformation and sigmoid-based feature modulation, while the macro-level gating mainly relies on supervised domain recognition and Top-K expert selection. It is unclear how the two gating mechanisms interact to form the claimed hierarchical design, as Eq. (5) appears to use the original features HH rather than the enhanced features H^\hat{H} from the micro-level gating for expert routing. In addition, while mitigating expert competition and collapse is presented as a key contribution, the paper does not provide an explicit load-balancing mechanism or sufficient analysis of expert utilization. The ablation results suggest that the components improve segmentation accuracy, but do not fully establish what is methodologically new about their combination.

References:

[1] Mamba Goes HoME: Hierarchical Soft Mixture-of-Experts for 3D Medical Image Segmentation. NeurIPS 2025.

[2] SegMoTE: Token-Level Mixture of Experts for Medical Image Segmentation. CVPR 2026.

[3] Rethinking u-net: Task-adaptive mixture of skip connections for enhanced medical image segmentation. AAAI 2025.

Questions for authors
  1. Could the authors clarify the main methodological differences between HG-MoE and recent approaches such as MambaHoME and SegMoTE, particularly regarding adaptive expert specialization and routing?

  2. The paper emphasizes that macro-level gating reduces expert competition and collapse. However, it is unclear how the proposed Top-K routing ensures balanced expert utilization. Is there any explicit mechanism that prevents expert collapse? Could the authors provide expert utilization statistics or routing distributions to support this claim?

  3. The paper motivates the problem by highlighting the lack of explicit domain annotations, but Eq. (6) indicates that domain labels are used to supervise the domain recognition branch during training. Could the authors clarify whether domain labels are required during training and how this assumption differs from existing domain-aware MoE methods?

  4. Tables 3 and 4 report average Dice scores over all 14 and 16 anatomical structures, respectively, but only seven categories are shown. Could the authors provide the complete per-organ results and clarify how the overall averages are calculated?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.