acceptodds
Under review as a conference paper at ICLR 2027

Uncertainty-Guided Adaptive Expert Selection for Graph Neural Networks

Abstract

Graph Mixture-of-Experts (MoE) models use a router to assign node-specific probabilities to expert networks and combine selected expert outputs for node classification. The router typically varies which experts are selected while fixing their number across nodes. Our observations show that the benefit of additional experts varies with predictive uncertainty, motivating node-specific control of expert participation. We propose DMoE, which uses entropy from cached predictions to specify how much of the router's probability distribution over experts should be retained. At each layer, experts are selected in descending probability order until this coverage target is reached. Predictive uncertainty thus determines the requested coverage, while the router's preferences determine which experts participate and how many are needed. Our analysis identifies a weighted participation cost in load balancing. In a controlled multiclass comparison with fixed routing probabilities, expert responses, and costs, the preferred expert count and retained probability are nondecreasing with reference entropy when the reference prediction is correct. Across 13 homophilous and heterophilous benchmarks, DMoE variants collectively achieve the highest performance among the evaluated methods, with relative gains of up to 7.92% on heterophilous graphs. End-to-end comparisons on two large graphs further show reductions reaching 73.07% in peak GPU memory and 46.53% in training time to early stopping.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Graph Neural Networks, Mixture of Experts, Adaptive Routing, Predictive Uncertainty, Node Classification

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_u7bjOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes D2MoE, a Graph MoE framework that uses predictive uncertainty to adapt expert selection for each node. It maps cached predictive entropy to a node-specific probability coverage threshold and selects experts using dynamic top-p routing. The paper also provides a theoretical analysis and evaluates the method on 13 graph benchmarks, showing competitive performance against existing methods.

Critical strengths
  1. The method is simple and effective. It provides a practical way to adapt expert participation based on node-level predictive uncertainty.

  2. The experiments cover 13 homophilous and heterophilous datasets and compare against 20 baselines, including several Graph MoE methods.

  3. The paper provides theoretical analysis, ablation studies, and routing visualizations to support the proposed method.

Critical weaknesses
  1. Although the paper compares D2MoE with fixed top-p and random top-p, these methods may activate different numbers of experts. It is unclear whether the improvements come from uncertainty-guided routing or different expert participation budgets.

  2. Figure 1 shows that the benefits of increasing the expert count vary across predictive entropy levels. However, it does not directly verify whether higher predictive uncertainty requires greater routing probability coverage.

  3. The inference procedure is not clearly explained. Since the method relies on predictive entropy cached from previous epochs, it is unclear how the coverage thresholds are obtained during inference.

Questions for authors
  1. Could the authors compare different routing strategies under similar average numbers of activated experts to better demonstrate the benefit of uncertainty guidance?

  2. Could the authors provide more direct evidence that higher predictive uncertainty requires greater routing probability coverage?

  3. How are the predictive entropy cache and coverage thresholds handled during inference?

Reviewer_YKSwOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

D²MoE sets each node's expert count via dynamic top-p routing, with the coverage target given by a sigmoid of the node's cached predictive entropy from the previous epoch. A theorem shows preferred expert count is nondecreasing in entropy when the reference prediction is correct. On 13 graphs vs 20 baselines, one of three D²MoE variants reports the best mean on every dataset. Appendices were not attached.

Critical strengths

Simple, cheap rule with a fair independent-GCN decile evaluation protocol.

D²MoE(GCN) beats the matched baseline GMoE on 12 of 13 datasets

Critical weaknesses

Gains are within noise and not isolated to the routing. On 11 of 13 datasets the gap to the best baseline is below one standard deviation; the headline 7.92% is Chameleon-fix at 50.39 ± 4.71 vs 46.69 ± 3.55. The best of three variants is picked per dataset on test.

Inconsistencies: FSGNN has identical entries on Tolokers and Penn94; GraphSAGE is OOM on Penn94 while D²MoE with K SAGE experts runs in ~3 GB; efficiency gains are credited to a method that evaluates every expert; Theorem 1 prescribes the most experts for confidently wrong nodes while the rule gives the fewest.

Questions for authors

Baseline configurations; correct FSGNN entries; the SAGE OOM contradiction.

Parameter counts and per-epoch time for Figure 8 methods.

Reviewer_s78wOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes D²MoE, a Graph Mixture-of-Experts framework that uses cached predictive entropy to determine node-specific routing coverage through dynamic top-p selection. It provides a theoretical motivation under fixed-model assumptions and evaluates classification performance across 13 graph benchmarks.

Critical strengths
  1. Well-motivated adaptive routing. The separation between uncertainty-dependent coverage and router-dependent expert selection is intuitive. Theorem 1 provides a mathematically motivated direction under explicitly stated assumptions.

  2. Broad empirical evaluation. Table 1 covers 13 datasets and 20 baselines, while Figures 4 and 10 compare static top-k, fixed top-p, and random top-p routing. These ablations provide meaningful evidence for uncertainty-conditioned coverage.

Critical weaknesses
  1. Limited theoretical justification for the actual algorithm. Theorem 1 assumes fixed routing probabilities, expert responses, and participation costs, with uncertainty induced by varying shared true-class evidence. However, D²MoE uses evolving parameters, cached predictions, and a heuristic sigmoid mapping (Eq. 17). The theorem therefore establishes a restricted monotonicity property rather than the optimality or effectiveness of the implemented routing policy.

  2. Efficiency claims require stronger causal validation. Algorithm 1 explicitly computes all expert outputs before selection, and Section 6 acknowledges that reduced participation does not directly reduce expert computation. Consequently, the memory and training-time improvements in Figure 8 cannot be attributed directly to sparse expert execution. Matched-backbone, matched-capacity comparisons are needed to isolate routing-related efficiency gains.

Questions for authors
  1. Can the authors establish whether Theorem 1's monotonicity prediction holds empirically when routing distributions and expert parameters evolve during training?

  2. How sensitive is performance to confidently incorrect cached predictions, particularly under distribution shift or severe heterophily?

  3. Under identical expert architectures, counts, and parameter budgets, what fraction of the reported efficiency improvement remains attributable to adaptive routing?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.