- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes RouterGFM to reuse pretrained GNN experts on new graph applications with limited target supervision. It first selects experts based on evaluations from previous applications, then weights the selected experts for each query according to how they performed in similar local contexts. The authors evaluate the method using a large collection of pretrained GNNs.
Critical strengths
- The paper tackles a practical problem: how to choose among pretrained GNNs when target labels are scarce. Using evaluations from previous applications can reduce the need to run every candidate model on a new application.
- RouterGFM considers both an expert's overall performance and its performance in specific graph contexts. This is a sensible choice, since the best expert may vary across query samples.
- The experiments cover 588 pre-trained GNNs with different architectures, pretraining objectives, and source datasets. This provides a useful setting for evaluating expert selection.
Critical weaknesses
- It is unclear how much historical data the context graph needs. Does RouterGFM still perform well when evaluations are available for only a few previous applications?
- Table 4 uses the same experts and predictions, but does not ensure that baselines such as Local-MLP have access to the same historical evaluation data. It is therefore unclear whether the gains come from the integration method or additional supervision. These baselines should be evaluated using the same historical data.
- The computational cost of the framework is unclear. Building the historical evaluation archive and retrieving records also incur costs. Please report the runtime and storage costs of each stage and compare the overall cost with existing methods.
- Figure 2 is unclear and should better illustrate the overall workflow.
Questions for authors
The major concerns are already discussed in the weaknesses above.
2 comments
Follow-up to Reviewer QigV
Thank you again for your helpful suggestions. In our response and revised manuscript, we have clarified the historical-data configuration, the information available to the baselines, and the cost breakdown. We have also improved Figure 2 and its caption to make the workflow easier to follow.
When you have a chance, we would appreciate hearing whether these explanations and presentation changes are helpful, and whether any point still needs clarification. We would be happy to address any remaining questions during the discussion.
Response to Reviewer QigV
Summary: We specify the historical source configuration, distinguish the information used by integration baselines, summarize costs, and present a revised Figure 2.
Thank you for recognizing the value of expert reuse and context-dependent integration.
1. How much history is needed, and what if only a few applications are available?
Our setting assumes access to public source datasets from which historical expert evaluations can be built once and reused. The released default configuration specifies:
The catalog combines nine evaluated dataset-task pairs with sixteen additional source pairs. All tasks, budgets, and splits of the target dataset are excluded from its history. An application also specifies a support budget and split seed; these repetitions do not count as new datasets. The archive retains available expert evaluations, so a complete application-by-expert matrix is unnecessary. Appendix B.3 now makes this configuration explicit.
The shared source archive can be reused for new targets. Our release plan covers code, data preparation, and checkpoints to support reuse and community additions.
The reported experiments do not vary the total number of source applications used to train the router, so they do not establish how well a router trained from only a few applications performs. These configuration counts specify the default pool; they are not a measured minimum data requirement.
2. What information do the integration baselines use?
Target-trained gates. The reported Local-MLP learns weights for the current expert team from target-support contexts, predictions, and labels. Training it across historical applications requires aligning expert identities or conditioning the gate on expert metadata. Table 4 does not include that adaptation.
Methods that learn from history. MetaGL and Model Spider use historical evaluations for selection, with the same eligible inventory and the target dataset excluded from history (Table 9). Their five-shot hit@5 values are both 72%, compared with RouterGFM's 84%.
For local integration, Table 4 fixes checkpoints, heads, query predictions, and evaluation cells. Its archive controls also retain the same historical records:
These controls show that centering and correct context-residual correspondence improve integration with history fixed. RouterGFM's contribution couples adaptive historical learning through the heterogeneous context graph with transfer of local performance into instance-dependent weights. Section 4.4 and Appendix C distinguish selection comparisons from the fixed-team integration controls.
3. Computation and storage.
Preparation is reused across targets; prediction adds historical retrieval and local weighting to the selected experts' forward passes. Appendix B.4 distinguishes offline expert/history construction and router training from target descriptor extraction, selection, head fitting, and prediction. Selection scores metadata before executing candidate GNNs; the selected team has K=5 in the main comparison. For N queries, R compatible records, and key dimension dk, exhaustive retrieval adds O(NKRdk) work and float32 cached keys occupy 4Rdk bytes.
These are aggregate project disk totals. Relative to global weighting on the same team, RouterGFM shares expert predictions and adds retrieval/weighting. Measured stagewise runtimes are not reported; the comparison above concerns computation and storage accounting.
4. Revised Figure 2.
We have clarified Figure 2 while preserving its original layout. Matching step labels connect the overall pipeline on the left to expert selection and local integration on the right. The evaluation edges are now labeled as historical evaluations. The expanded caption explains the context graph, local archive, target-support head fitting, and how local loss estimates yield softmax weights for each query.
Does this version make the selection and integration steps clearer? We would welcome a specific suggestion if any connection remains difficult to follow.