acceptodds
Under review as a conference paper at ICLR 2027

RouterGFM: Inductive Routing and Local Expertise Transfer for Graph Learning

Abstract

Reusing pretrained *graph neural networks* (GNNs) requires deciding which experts suit a new application and how to combine their predictions across graph contexts. Application-level selection does not resolve instance-specific contributions, while learning a new local router can require more target supervision than is available. We propose GraphRouter, a graph mixture-of-experts framework that combines inductive expert selection with context-guided local integration. A shared application–expert scorer over a heterogeneous context graph selects experts from metadata and historical evaluations before executing candidate GNNs on target data. After fitting the selected task heads, expert-conditioned retrieval transfers historical local performance differences into instance-dependent prediction weights, while the pretrained GNNs and router parameters remain fixed. Our conditional analysis separates omitted local expertise, performance-estimation error, and mixture temperature, and characterizes the limits of metadata-only expert insertion. Experiments on nine datasets cover node classification, link prediction, graph classification, and graph regression. In the five-shot classification diagnosis with expert predictions fixed, local integration reduces average and worst-cell Brier loss by 11.7% and 17.3% relative to application-level weighting. Further analyses distinguish useful expert coverage from accurate integration and identify when unreliable historical evidence limits transfer.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Graph Foundation Models, Mixture of Experts, Expert Routing, Graph Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_QigVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RouterGFM to reuse pretrained GNN experts on new graph applications with limited target supervision. It first selects experts based on evaluations from previous applications, then weights the selected experts for each query according to how they performed in similar local contexts. The authors evaluate the method using a large collection of pretrained GNNs.

Critical strengths
  1. The paper tackles a practical problem: how to choose among pretrained GNNs when target labels are scarce. Using evaluations from previous applications can reduce the need to run every candidate model on a new application.
  2. RouterGFM considers both an expert's overall performance and its performance in specific graph contexts. This is a sensible choice, since the best expert may vary across query samples.
  3. The experiments cover 588 pre-trained GNNs with different architectures, pretraining objectives, and source datasets. This provides a useful setting for evaluating expert selection.
Critical weaknesses
  1. It is unclear how much historical data the context graph needs. Does RouterGFM still perform well when evaluations are available for only a few previous applications?
  2. Table 4 uses the same experts and predictions, but does not ensure that baselines such as Local-MLP have access to the same historical evaluation data. It is therefore unclear whether the gains come from the integration method or additional supervision. These baselines should be evaluated using the same historical data.
  3. The computational cost of the framework is unclear. Building the historical evaluation archive and retrieving records also incur costs. Please report the runtime and storage costs of each stage and compare the overall cost with existing methods.
  4. Figure 2 is unclear and should better illustrate the overall workflow.
Questions for authors

The major concerns are already discussed in the weaknesses above.

2 comments
AuthorsOpenReview ↗

Response to Reviewer QigV

Summary: We specify the historical source configuration, distinguish the information used by integration baselines, summarize costs, and present a revised Figure 2.

Thank you for recognizing the value of expert reuse and context-dependent integration.

1. How much history is needed, and what if only a few applications are available?

Our setting assumes access to public source datasets from which historical expert evaluations can be built once and reused. The released default configuration specifies:

Historical source configurationDataset-task pairsDistinct datasets
Full catalog25252121
After excluding the current target dataset24242020

The catalog combines nine evaluated dataset-task pairs with sixteen additional source pairs. All tasks, budgets, and splits of the target dataset are excluded from its history. An application also specifies a support budget and split seed; these repetitions do not count as new datasets. The archive retains available expert evaluations, so a complete application-by-expert matrix is unnecessary. Appendix B.3 now makes this configuration explicit.

The shared source archive can be reused for new targets. Our release plan covers code, data preparation, and checkpoints to support reuse and community additions.

The reported experiments do not vary the total number of source applications used to train the router, so they do not establish how well a router trained from only a few applications performs. These configuration counts specify the default pool; they are not a measured minimum data requirement.

2. What information do the integration baselines use?

Target-trained gates. The reported Local-MLP learns weights for the current expert team from target-support contexts, predictions, and labels. Training it across historical applications requires aligning expert identities or conditioning the gate on expert metadata. Table 4 does not include that adaptation.

Methods that learn from history. MetaGL and Model Spider use historical evaluations for selection, with the same eligible inventory and the target dataset excluded from history (Table 9). Their five-shot hit@5 values are both 72%, compared with RouterGFM's 84%.

For local integration, Table 4 fixes checkpoints, heads, query predictions, and evaluation cells. Its archive controls also retain the same historical records:

Historical integrationMean BrierWorst-cell Brier
No residual centering0.2280.2280.3680.368
Shuffled context-residual pairs0.2440.2440.4070.407
RouterGFM0.2110.2110.3210.321

These controls show that centering and correct context-residual correspondence improve integration with history fixed. RouterGFM's contribution couples adaptive historical learning through the heterogeneous context graph with transfer of local performance into instance-dependent weights. Section 4.4 and Appendix C distinguish selection comparisons from the fixed-team integration controls.

3. Computation and storage.

Preparation is reused across targets; prediction adds historical retrieval and local weighting to the selected experts' forward passes. Appendix B.4 distinguishes offline expert/history construction and router training from target descriptor extraction, selection, head fitting, and prediction. Selection scores metadata before executing candidate GNNs; the selected team has K=5K=5 in the main comparison. For NN queries, RR compatible records, and key dimension dkd_k, exhaustive retrieval adds O(NKRdk)O(NKRd_k) work and float32 cached keys occupy 4Rdk4Rd_k bytes.

Stored filesDisk usage (GiB)
Pretrained models18.118.1
Fine-tuned models676.7676.7
Whole projectabout 800

These are aggregate project disk totals. Relative to global weighting on the same team, RouterGFM shares expert predictions and adds retrieval/weighting. Measured stagewise runtimes are not reported; the comparison above concerns computation and storage accounting.

4. Revised Figure 2.

We have clarified Figure 2 while preserving its original layout. Matching step labels connect the overall pipeline on the left to expert selection and local integration on the right. The evaluation edges are now labeled as historical evaluations. The expanded caption explains the context graph, local archive, target-support head fitting, and how local loss estimates yield softmax weights for each query.

Does this version make the selection and integration steps clearer? We would welcome a specific suggestion if any connection remains difficult to follow.

AuthorsOpenReview ↗

Follow-up to Reviewer QigV

Thank you again for your helpful suggestions. In our response and revised manuscript, we have clarified the historical-data configuration, the information available to the baselines, and the cost breakdown. We have also improved Figure 2 and its caption to make the workflow easier to follow.

When you have a chance, we would appreciate hearing whether these explanations and presentation changes are helpful, and whether any point still needs clarification. We would be happy to address any remaining questions during the discussion.

Reviewer_wVo7OpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes ROUTER GFM, a graph mixture-of-experts framework combining inductive expert selection with context-dependent prediction integration. A heterogeneous graph predicts application-level expert suitability, while retrieval of historical performance residuals determines local mixture weights without training a target-specific router. The paper provides conditional risk bounds and evaluates the framework across nine datasets and four graph-learning tasks.

Critical strengths
  1. Well-motivated framework with meaningful experimental controls. Separating global selection from local integration is practically relevant. The fixed-prediction comparison (Figure 4, Table 4) reduces average Brier loss from 0.239 to 0.211 and worst-cell loss from 0.388 to 0.321.

  2. Carefully scoped theoretical analysis. Theorem 1 distinguishes expert omission, estimation error, and mixture temperature. The authors correctly restrict the guarantee to convex prediction losses and a context-wise single-expert comparator.

Critical weaknesses
  1. Insufficiently controlled comparison with learned local gating. Table 4 compares ROUTER GFM against Local-MLP (0.226 versus 0.211 Brier loss), but explicitly acknowledges unmatched historical supervision. Consequently, the experiment does not establish whether the advantage arises from residual retrieval or greater access to transferable historical information.

  2. Limited statistical evidence for the central integration mechanism. The fixed-team diagnostic reports point estimates without variability or confidence intervals (Table 4). Although Table 1 reports five-seed statistics, uncertainty for the critical controlled comparison remains unquantified, weakening claims about consistent local-integration improvements.

Questions for authors
  1. Can Local-MLP and alternative learned gates be trained with identical historical applications, supervision, and validation budgets to isolate the contribution of residual retrieval?

  2. What are the across-seed confidence intervals and paired significance results for the fixed-team improvements in Table 4?

  3. How closely do the estimated conditional risks satisfy the compatibility assumption in Theorem 3, especially under the missing-family and residual-reversal conditions in Figure 5?

4 comments
AuthorsOpenReview ↗

Response to Reviewer wVo7

Summary: With the archive fixed, centering and correct context-residual correspondence improve integration. We clarify this attribution, the statistical scope of Table 4, and what the reliability tests measure in Theorem 3.

Thank you for recognizing the experimental controls and the scope of the theory.

1. Does the gain come from retrieval or additional history? (Weakness 1; Q1)

Yes, a learned gate can use historical applications if its training protocol aligns expert identities across applications or conditions the gate on expert metadata. The reported Local-MLP instead maps query context to weights for the current team's KK slots using target-support supervision. Table 4 does not report a gate trained with identical historical applications and validation budgets.

To test how historical information is used, the archive controls keep the records available while removing centering or disrupting the match between context and performance. All rows share checkpoints, fitted heads, predictions, and evaluation cells:

Integration ruleMean BrierWorst-cell Brier
Global weighting0.2390.2390.3880.388
Local-MLP0.2260.2260.3540.354
No residual centering0.2280.2280.3680.368
Shuffled context-residual pairs0.2440.2440.4070.407
RouterGFM0.2110.2110.3210.321

Removing centering increases mean/worst-cell loss by 0.017/0.047; shuffling context-residual correspondence increases them by 0.033/0.086. These controls show that correctly matched, centered local performance improves integration with history fixed. The 0.015 mean-loss gap to Local-MLP compares procedures with different supervision and does not isolate retrieval from a gate trained on the same history.

The novel capability of RouterGFM is to learn expert suitability adaptively through its heterogeneous context graph and transfer retrieved historical local performance into instance-dependent weights without fitting a target gate. Section 4.4 and Appendix C state the inputs and scope of each comparison.

2. Cross-seed intervals and paired significance. (Weakness 2; Q2)

Paired uncertainty comes from comparing methods within the same runs. Table 4 reports point estimates without paired confidence intervals or p-values. The observed mean/worst-cell reductions are 0.028/0.067 against global weighting and 0.015/0.033 against Local-MLP. The requested intervals require per-seed paired differences for these fixed-team episodes; they cannot be recovered from the aggregate values. Table 1's five-seed standard deviations concern different downstream metrics and do not quantify uncertainty for this diagnostic.

3. Compatibility under missing-family and reversal conditions. (Q3)

Intuitively, the assumption asks whether nearby context-expert pairs have similar performance relative to their own application averages. Figure 5 tests the effect of incomplete or corrupted evidence; it does not measure how closely the true conditional risks satisfy this condition.

For δ=μ−r\delta=\mu-r, Theorem 3 gives

∣δ^−δ∣≤L∑iwi(di+si)+η+∑iwiεi.|\widehat\delta-\delta|\leq L\sum_i w_i(d_i+s_i)+\eta+\sum_iw_i\varepsilon_i.

Here did_i is retrieval distance, sis_i within-cell variation, η\eta cross-application mismatch of true residuals, and εi\varepsilon_i error in a stored residual.

Removing a context family reduces coverage and can increase did_i. Reversal flips cached residuals at unchanged keys, while target predictors and their true risks remain fixed; it therefore tests record error εi\varepsilon_i. With retrieval weights, global estimates, and ρ\rho fixed, δ^rev=−δ^\widehat\delta^{\mathrm{rev}}=-\widehat\delta and r^rev−r^=2ρδ^\widehat r^{\mathrm{rev}}-\widehat r=2\rho\widehat\delta.

Archive conditionGlobalFull correctionSource-validated ρ\rho
Matched0.2390.2390.2110.2110.2110.211
Missing family0.2390.2390.2810.2810.2420.242
Reversed residuals0.2390.2390.3340.3340.2510.251

Table 13 reports Brier losses, not estimates of LL, η\eta, or compatibility-violation rates. Section 4.5 and Appendix D.5 now make the tested archive perturbations and this distinction explicit.

AuthorsOpenReview ↗

Follow-up to Reviewer wVo7

We would appreciate your thoughts on our updated response, particularly the clarifications concerning historical supervision, the statistical interpretation of Table 4, and how the archive perturbations relate to the compatibility assumption in Theorem 3.

Could you let us know which concerns, if any, remain unresolved? Your feedback would help us focus the remaining discussion. We would also be grateful if you could take these clarifications and the revised manuscript into account in your final assessment. Thank you again for your careful review.

AuthorsOpenReview ↗

Closing follow-up to Reviewer wVo7

Thank you again for your careful assessment of the experimental controls and theory. Before the discussion closes, we would appreciate your consideration of our response on historical supervision, the statistical scope of the fixed-team results, and the relationship between the archive perturbations and Theorem 3.

In particular, the controls that retain the same archive examine the roles of residual centering and context-performance correspondence. Our response also clarifies which conclusions the reported statistics and theoretical assumptions support.

We would welcome a brief note on any point that still needs clarification, and would be grateful if you could consider the response and revisions in your final assessment.

Thank you for the response. It clarified some of my concerns.

Reviewer_xTchOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RouterGFM, a framework for selecting and combining pretrained GNNs on new graph tasks. It first uses a heterogeneous graph to select suitable experts based on metadata and historical evaluations. It then retrieves historical local performance information to assign instance-specific weights to the selected experts. Experiments on nine datasets covering four graph learning tasks show improvements over existing baselines.

Critical strengths

1.The paper addresses a practical problem in graph model reuse. Separating expert selection from instance-level integration is well motivated, especially in few-shot settings where learning a new router can be difficult.

2.The experiments cover a wide range of graph tasks and datasets, with generally strong results against existing methods. The improvements are particularly noticeable on several node classification datasets.

3.The fixed-team experiments in Section 4.4 are useful. By keeping expert predictions unchanged, the authors provide evidence that the gains come from better integration rather than simply selecting stronger experts.

Critical weaknesses

1.The comparison with other local integration methods is not entirely fair. RouterGFM uses historical evaluation data, while the competing methods do not necessarily have access to the same supervision. It would be useful to compare against baselines trained with the same historical information.

2.The method relies on a large collection of pretrained experts and their historical evaluations. Building and maintaining this information could be expensive, but the paper does not provide a detailed analysis of the end-to-end computational and storage costs.

3.The method depends heavily on the quality of historical information. As shown in Table 13, local integration can perform worse than global weighting when historical evidence is missing or unreliable. It is unclear how the method could detect such cases and avoid negative transfer on new applications.

Questions for authors

please see weaknesses

3 comments
AuthorsOpenReview ↗

Response to Reviewer xTch

Summary: RouterGFM learns expert suitability from historical evaluations and transfers local performance into query-specific weights. We clarify the comparisons, reusable costs, and fallback behavior.

Thank you for recognizing the motivation and the fixed-team evidence.

1. Comparisons using historical information.

Target-trained gates. Local-MLP, Link-MoE, and SAGMM-PE learn gates for a downstream task. Their evaluated formulations do not use cross-application evaluation records. Using such records requires an additional training protocol with aligned expert identities or expert conditioning. The reported Local-MLP uses target support; Table 4 fixes checkpoints, heads, predictions, and evaluation cells, but does not include this historical-gate adaptation.

Methods that learn from history. MetaGL and Model Spider are included in Table 9, with the same eligible expert inventory and the target dataset excluded from history. At five shots, their hit@5/regret@5 are 72%/0.032 and 72%/0.029, compared with 84%/0.019 for RouterGFM. These comparisons evaluate application-level selection.

For local integration, Table 4 retains the same archive in RouterGFM, the no-centering variant, and the shuffled-correspondence variant:

Integration ruleMean BrierWorst-cell Brier
Global weighting0.2390.2390.3880.388
Local-MLP0.2260.2260.3540.354
No residual centering0.2280.2280.3680.368
Shuffled context-residual pairs0.2440.2440.4070.407
RouterGFM0.2110.2110.3210.321

With history fixed, centering lowers mean/worst-cell loss by 0.017/0.047; correct context-residual matching lowers them by 0.033/0.086 relative to shuffling. Thus, how historical performance is transferred matters.

RouterGFM combines adaptive learning through its heterogeneous context graph with retrieval and transfer of historical local performance into instance-dependent weights. Among the compared graph MoEs, only RouterGFM provides this cross-application local transfer without fitting a target gate. Section 4.4 and Appendix C distinguish the information used by these comparisons.

2. End-to-end computation and storage.

Historical preparation is reused across targets; on a fixed team, the extra work over global weighting is retrieving local records and computing query-specific weights. Appendix B.4 separates this preparation from target processing. Preparation includes expert pretraining, historical head fitting/evaluation, historical descriptor construction, router training, and archive construction. Target processing includes descriptor extraction and metadata scoring before candidate GNN execution; only the selected heads are fitted. The main comparison uses K=5K=5 experts.

For NN queries, RR compatible records, and key dimension dkd_k, exhaustive retrieval costs O(NKRdk)O(NKRd_k) and cached keys occupy 4Rdk4Rd_k bytes in float32. Global weighting on the same team shares the expert predictions; RouterGFM adds retrieval and local weighting.

Stored filesDisk usage (GiB)
Pretrained models18.118.1
Fine-tuned models676.7676.7
Whole projectabout 800

These are project-wide disk totals, dominated by saved fine-tuned models. A target uses selected checkpoints and heads with the shared archive. We report storage and computational complexity; the submission contains no measured stagewise runtime comparison.

3. Detecting unreliable evidence and avoiding negative transfer.

RouterGFM filters records by task/loss family, support budget, and normalization. Without a compatible record, an expert falls back to its global estimate. Source-validated shrinkage limits the influence of retained records; ρ=0\rho=0 gives global weighting.

Table 13 measures how well this protection works:

Historical evidenceGlobalFull correctionSource-validated ρ\rho
Matched coverage0.2390.2390.2110.2110.2110.211
Half of context cells missing0.2390.2390.2350.2350.2270.227
Entire context family missing0.2390.2390.2810.2810.2420.242
Archived residuals reversed0.2390.2390.3340.3340.2510.251

With half the cells missing, shrinkage preserves a 0.012 improvement over global weighting. For a missing family and reversed archive residuals, it reduces loss by 0.039/0.083 relative to full correction, leaving 0.003/0.012 excess over global weighting. The reversal test corrupts cached residuals while keeping retrieval keys fixed, so metadata checks cannot identify that corruption.

Therefore, missing compatible evidence has an explicit fallback, while source-validated shrinkage limits the observed harm from unreliable evidence; it does not reliably detect or eliminate misleading residuals that pass the compatibility checks. We have clarified this behavior and the exact archive perturbation in Section 4.5 and Appendix D.5.

AuthorsOpenReview ↗

Follow-up to Reviewer xTch

Thank you again for your thoughtful review. We would like to briefly follow up on our response, which clarifies the historical information used by the compared methods, the computational and storage costs, and the behavior when local evidence is unreliable.

We would particularly appreciate your feedback on the historical-supervision comparison, as this was a central concern in your review. Could you let us know whether the explanation addresses your concern, or which aspect needs further clarification? Thank you for your time and consideration.

AuthorsOpenReview ↗

Closing follow-up to Reviewer xTch

As the discussion period comes to a close, we would like to thank you again for your thoughtful review.

Our response clarifies the historical supervision used in the comparisons, separates reusable preparation from target processing costs, and explains both the fallback behavior and the limits of shrinkage under unreliable evidence. The controls using the same archive provide evidence for the role of centered local performance transfer.

If time permits, we would appreciate a brief update on whether these clarifications address your concerns. Thank you for considering our response and revised manuscript in your final assessment.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Follow-up on Historical Data, Baseline Fairness, and Runtime

Thank you for revisiting our response and for recognizing the value of the work. We appreciate your clear explanation of the remaining concerns.

Historical applications. The current evaluation assumes an existing archive of source applications. The missing-context tests assess incomplete local evidence, while performance with only a few source applications remains unquantified. We agree that these are different questions.

Local-MLP comparison. Fixing the expert predictions does not equalize historical supervision. The evidence for how RouterGFM uses history comes from the controls that retain the same archive: removing residual centering or shuffling context-performance correspondence increases mean Brier loss from 0.211 to 0.228 or 0.244, respectively (Table 4). These results support the local transfer mechanism, while comparison with a learned gate trained on identical history remains open.

Runtime. The storage figures and complexity analysis describe the resource requirements; measured runtime is still missing. They therefore do not establish an end-to-end speed advantage.

Within this scope, our contribution is the ability to learn expert suitability adaptively through the heterogeneous context graph and transfer historical local performance into instance-dependent prediction weights. We hope this capability and the controlled evidence can be considered alongside the remaining evaluation gaps. Thank you again for your time and constructive feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.