acceptodds
Under review as a conference paper at ICLR 2027

Whether to Trust GNNs or LLMs? FairRouter for Few-Shot Node Classification

Abstract

Text-attributed graphs (TAGs) couple textual semantics with graph topology, and recent GNN–LLM collaborative approaches exploit these complementary modalities to generate pseudo-labels for few-shot learning. However, existing approaches typically fix one model as the annotator or judge, regardless of which modality is more reliable for a given node, leading to biased pseudo-labels and underutilized complementary information. To this end, we propose FairRouter, which generates pseudo-labels through fair modality routing. Specifically, the router characterizes each node with multi-level evidence from both models and learns two signals: a trust score that decides whether the node yields a reliable pseudo-label, and a preference score that decides which modality to follow when the two disagree. % To generalize from the few labeled nodes, the router is trained on augmented views with semantic, topological, and modality-preference perturbations. To generalize from scarce labels, the router learns from augmented views, where semantic and topological perturbations are applied to simulate diverse modality trust and preference patterns. The routed pseudo-labels then supervise a multimodal residual classifier, which recovers nodes misclassified by both modalities while preserving their consensus. Experiments on five TAG benchmarks show that FairRouter achieves the best overall accuracy under all label budgets and the highest pseudo-label precision on all datasets, with gains of up to nine percentage points in the most label-scarce settings. The codebase is publicly availableThe code of our project is available at https://anonymous.4open.science/r/FairRouter-C6F1https://anonymous.4open.science/r/FairRouter-C6F1..

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Large Language Models, Graph Neural Networks, Graph Few-shot Learning, Semi-Supervised Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_6GXEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper targets few-shot node classification on Text-Attributed Graphs (TAGs) using GNNs and LLMs. The authors critique existing methods for relying on a fixed teacher (e.g., GNN-as-Judge) and propose \textbf{FairRouter}. FairRouter extracts multi-modal evidence across probability, topology, and gold prototypes, decomposes routing into a trust score (tit_i) and a preference score (rir_i), augments few-shot support views, selects pseudo-labels, and trains a multi-modal residual MLP to fuse expert distributions.

Critical strengths

Symmetrical, node-adaptive routing between GNNs and LLMs is inherently more principled than hardcoding one modality as an infallible judge.

The paper provides extensive breakdowns of pseudo-label precision, agreement/disagreement partitions, and hyperparameter sensitivity.

Critical weaknesses

The main paper presents FairRouter as a unified framework, but the appendix reveals an extreme patchwork of hand-picked heuristics. As shown in Tables 4 and 5: Agreement scoring uses unlearned geometric mean on Cora, raw LLM confidence on Citeseer, logistic regression on PubMed, and Extra Trees ensembles on Arxiv/Products. Disagreement scoring pairs an RBF-SVM on Cora, a transfer neural network on Citeseer, an Extra Trees model on PubMed, and a joint neural net on Arxiv. This is not a generalizable representation learning method; it is a collection of dataset-specific Kaggle-style recipes overfitted to individual benchmarks.

Lines 1085--1088 and 1124--1126 admit that for PubMed and Products, the router classifiers are trained on the other four datasets (Source training), pooling thousands of labeled nodes from large graphs like Arxiv. Calling this a ''few-shot setting'' (k∈{3,5,10}k \in \{3, 5, 10\}) while secretly injecting supervised cross-dataset pretraining into the routing classifiers is an unfair evaluation against strictly few-shot baselines.

The router relies on a massive stack of 43 to 70 hand-engineered tabular statistics (Appendix A: Bhattacharyya overlap, Jensen-Shannon divergence, sentinel padding distances of 2.0, empirical rank transforms, and arbitrary feature clipping intervals [−8,8][-8, 8] or [−20,20][-20, 20]). Such extreme heuristics obscure whether empirical gains stem from principled multi-modal synergy or brittle feature-level hand-crafting.

In Table 7, the validation-selected acceptance ratios ρA/ρD\rho_A / \rho_D exhibit catastrophic instability across random seeds. For instance, on Arxiv 3-shot, ρA\rho_A swings wildly from 0.25% (Seed 43) to 1.0% (Seed 42) to 100.0% (Seed 44). A selector whose decision swings by two orders of magnitude between different 3-shot splits of the same dataset indicates extreme sensitivity to the few-shot support drawing.

Questions for authors

On ogbn-products, FairRouter consistently loses to GNN-as-Judge by 2.67--3.57 pp across all budgets (Table 1). The authors attribute this to GNN-as-Judge fine-tuning the LLM while FairRouter leaves it frozen. Furthermore, Section D.2 admits that the authors modified GNN-as-Judge's official implementation by disabling its 'retrain_gnn' component before evaluation. Modifying a competing baseline while underperforming against it on complex graphs weakens the core empirical claims.

5 comments

Response for reviewer 6GXE (1/4)

We sincerely thank the reviewer for recognizing our principled symmetric, node-adaptive routing and extensive analyses of pseudo-label precision and hyperparameter sensitivity. We carefully address each concern below and have incorporated the suggested improvements into our revised manuscript.

Weakness 1: not a generalizable representation learning method.

We thank the reviewer for highlighting the issue with dataset-specific implementations. We respectfully clarify that FairRouter unifies Trust–Preference factorization and node-adaptive routing mechanism, with different scorers serving as alternative implementations within this shared framework.

Following the reviewer's suggestion in Ethics Concern 2, we evaluated a single, unified router universally across all five datasets: logistic regression on the same probability-derived compact features, with both acceptance ratios fixed. As shown in Table W1-1, FairRouter still outperforms competing baselines when scorer families, features, and acceptance ratios are fixed across datasets.

Table W1-1. Uniform configuration results and comparisons.

Average accuracy (%)3-shot5-shot10-shot
LLM-GNN63.1764.0165.61
Locle66.4167.6369.03
GNN-as-Judge71.2373.5874.79
FairRouter (original)74.7975.5276.15
FairRouter (Uniform router)73.1074.0975.84

Response for reviewer 6GXE (2/4)

Weakness 2: secretly injecting supervised cross-dataset pretraining.

We thank the reviewer for pointing out this important question. We respectfully clarify that cross-dataset pretraining is confined to the router, not the classifier, transferring expert-reliability patterns rather than class information (Appendix B), and cross-dataset pretraining has also been deployed in prior work on few-shot graph learning [1,2]. We agree that the main text should have described the cross-dataset pretraining more prominently, and we have added it to Section 3.2 of the revised paper (paragraph “Unified Pretraining”, marked in red).

Following the reviewer’s suggestion in Ethics Concern 1, we evaluate FairRouter without cross-dataset pretraining and fitted only on the target k-shot support. As shown in Table W2-1, removing pretraining slightly lowers average accuracy. However, target-only FairRouter still outperforms strongest baseline GNN-as-Judge on average and beats LLM-GNN and Locle in all 15 settings under this strict protocol.

Table W2-1. Removing cross-dataset pretraining results and comparisons.

ShotMethodCoraCiteseerPubMedArxivProductsAvg.
3LLM-GNN73.37±1.7666.80±0.6573.37±0.6847.40±1.4854.93±0.4563.17
3Locle74.30±1.9069.30±0.3772.97±1.9251.70±0.9863.80±1.6366.41
3GNN-as-Judge74.37±1.1964.07±1.8288.93±1.3053.80±3.8275.00±1.7171.23
3FairRouter (paper)79.37±0.1773.37±0.1290.80±1.0458.13±0.1272.30±0.8574.79
3FairRouter (Target-only)79.17±0.5472.63±0.9287.47±2.1356.70±0.7571.00±1.3073.39
5LLM-GNN74.30±2.0667.30±0.5774.90±1.4748.83±1.0554.73±0.8364.01
5Locle75.77±1.4469.07±0.5676.87±1.7952.37±0.8464.07±0.6367.63
5GNN-as-Judge76.20±2.3365.90±4.1990.80±1.5559.90±1.2275.10±0.9573.58
5FairRouter (paper)80.70±1.9172.60±0.7991.47±1.3060.40±1.5672.43±0.4175.52
5FairRouter (Target-only)79.27±0.9973.17±1.1190.40±0.3659.93±2.1071.60±0.6574.87
10LLM-GNN76.47±1.5068.33±0.9575.90±1.2852.23±1.8255.13±0.3965.61
10Locle78.30±1.3068.87±0.5276.40±1.6956.23±1.1365.37±1.1469.03
10GNN-as-Judge79.17±1.2368.17±1.0191.43±1.1058.80±5.4076.37±0.6574.79
10FairRouter (paper)81.27±1.3673.47±1.2591.63±0.8361.60±0.8672.80±0.8276.15
10FairRouter (Target-only)80.67±1.6272.97±0.8590.53±2.0260.57±0.6972.13±1.6575.37

[1] Zhao, Haihong, et al. "All in one and one for all: A simple yet effective method towards cross-domain graph pretraining." Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2024.

[2] Liu, Hao, et al. "One for all: Towards training one graph model for all classification tasks." International conference on learning representations. Vol. 2024. 2024.

Response for reviewer 6GXE (3/4)

Weakness 3: whether empirical gains stem from principled multi-modal synergy or brittle feature-level hand-crafting.

We thank the reviewer for raising this feature-level concern. We would like to clarify that we use a fixed set of interpretable statistics to provide multi-level evidence, allowing the router to learn their relative importance rather than prescribing hand-crafted rules.

To clarify this, we added an ablation study to assess the contribution of each feature group. The ablation shows that more information comes from cross-modal probability evidence (Bhattacharyya overlap, JS divergence), and sentinel padding distances of 2.0 fall into the prototype group, which is not the main source of gains (Table W3-1). This indicates that performance does not come from accumulating features, nor from manually selecting a few effective ones; instead, the router itself learns to weigh which evidence is useful.

Table W3-1. Performance change removing each feature group.

Removed feature group: accuracy change (pp)3-shot5-shot10-shot
Probability features−1.71−0.77−1.04
Prototype features−0.07−0.17−0.23
Neighborhood features−0.05−0.37−0.35
Representation projections−0.19−0.44−0.10

The specific constants reviewer mention have simple roles:

  • The sentinel value 2.0 is the maximum possible cosine distance (1 − cos ∈ [0, 2]). It encodes "no reference available" as "as far as possible".
  • The clipping ranges are outlier guards applied after standardization.

Weakness 4: the validation-selected acceptance ratios exhibit catastrophic instability.

1. Can the authors explain why Arxiv 3-shot acceptance jumps from 0.25% to 100% between Seed 43 and Seed 44?

We thank the reviewer for highlighting this variation. Actually, the jump is in agreement branch and follows the validation-based Wilson criterion. For Seed 43, the top 0.25% is 27/27 correct, giving a Wilson lower bound of 0.9089 versus 0.8179 at full coverage. For Seed 44, the top 0.25% is only 18/24 correct, while full coverage reaches 7,323/8,798 correct and the highest Wilson bound (0.8257), so 100% is chosen. The ratio jump reflects the current selection rule’s sensitivity to validation noise, but this does not imply that the router ranks unreliably.

2. How can users trust this router in deployment?

We thank the reviewer for this practical question. To show that the router is reliable enough for deployment, we removed the validation-based ratio search entirely and ran FairRouter with one fixed ratio ρ_A = ρ_D = 15% across every dataset and settings. The results show that a single fixed ratio already matches or exceeds validation-selected ratios, and the final accuracy is relative stable across seeds. Users can therefore deploy FairRouter without per-dataset or per-split tuning.

Table W4-1. Performance on Arxiv change fixing acceptance ratio (15%).

Result3-shot5-shot10-shot
Arxiv: original accuracy (%)58.13±0.1260.40±1.5661.60±0.86
Arxiv: fixed-ratio accuracy (%)58.47±1.0860.90±1.4461.60±0.83
Average accuracy change (pp)+0.34+0.50+0.00
Reviewer_MTfUOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies pseudo-label generation for few-shot node classification on text-attributed graphs. FairRouter separates the decision to trust either expert from the decision to prefer the GNN or LLM when their predictions disagree. It forms scores qG=trq_G=tr, qL=t(1−r)q_L=t(1-r), and qN=1−tq_N=1-t, then selects pseudo-labels using separate agreement/disagreement budgets. A residual classifier combines frozen graph and language representations with the expert distributions to predict classes beyond either expert's top prediction. On five benchmarks with 33, 55, and 1010 labels per class, the reported average accuracies improve over GNN-as-Judge by 3.563.56, 1.941.94, and 1.361.36 percentage points, respectively. Additional experiments examine alternative LLMs and two heterophilous graphs.

Critical strengths
  1. A useful decomposition of reliability and modality preference. Separating whether a correct candidate exists from which expert supplies it directly addresses the failure of choosing between two incorrect predictions. The conditional construction in Eqs. (1)–(3) is coherent and does not require independent expert errors. The matched-coverage confidence-router comparison provides evidence that the selected nodes and labels matter, beyond simply increasing pseudo-label volume (Fig. 10 and Tables 10–12).

  2. Meaningful empirical gains with informative limitations. Table 1 reports improvements in 12 of 15 dataset–budget settings, including a substantial Citeseer gain at three shots. The paper also reports its Products deficit and the cases where confidence is more precise than learned routing. Frozen experts and the residual branch make the framework a potentially practical alternative to repeated LLM fine-tuning.

Critical weaknesses
  1. The evidence does not isolate the proposed few-shot router from additional supervision and configuration selection. The implementation is substantially more specialized than the main-text description suggests. Appendix B, Tables 4–5, uses different scorer families, feature modes, source/target training populations, and transfer procedures for different datasets. Source training uses labeled observations from the other four benchmark datasets. Appendix D.1 also supplies 500 validation labels on each Planetoid dataset, and Appendix B.3 selects routing ratios separately per dataset, budget, and seed using validation pseudo-label correctness; further validation selection affects transformations and the residual classifier. These choices are disclosed and are not, by themselves, evidence of test-label leakage. However, the comparisons do not establish how much of the gain comes from node-wise routing rather than cross-dataset supervision and the larger selection procedure. This matters particularly for the claim that the router generalizes from only a few labels. A target-only, fixed-family router and a matched label/selection-budget comparison are needed to resolve this attribution.

  2. Prototype-based router training and calibration have a consequential support/query mismatch. Appendix A.4 explicitly retains each support node in its own class prototype and nearest-support search, and the prototypes are not rebuilt within calibration folds (p. 18, lines 930–935; Appendix B.2, line 1232). On clean views, a support node has zero nearest-support distance to itself, whereas an unseen query generally does not. Moreover, an out-of-fold node's label still affects the prototype features used by its scorer. This affects configurations that actually use prototype features, including Citeseer trust and Arxiv agreement; it does not apply to probability-only scorers. Grouping augmented views by node therefore does not by itself make the full feature/calibration pipeline out of fold. The paper does not establish whether the gains survive support-node exclusion and fold-specific reference construction.

Questions for authors
  1. Clarification: supervision budget. What is the exact number of unique labeled nodes used from each source and target dataset by the source-only, pooled, and transfer scorers in Tables 4–5? Please specify whether the source-label budgets change with the target budget k∈{3,5,10}k\in\{3,5,10\}. This is necessary to interpret the effective supervision budget behind the few-shot results.

  2. Evidence request: attribution to routing versus transfer and specialization. How much of the reported improvement remains when the router is trained only on the target support set? Separately, what happens when the scorer family and feature configuration are fixed across datasets? These controls would distinguish the contribution of the routing formulation from cross-dataset supervision and dataset-specific model selection.

  3. Clarification: validation-based selection. Appendix D.1 provides 500 validation labels on each Planetoid dataset, while the router uses validation-based choices for scorer configurations, score transformations, acceptance ratios, and downstream training. What are the effective search budgets for these choices, and do the competing methods receive comparable validation-label access and model-selection effort? Please explain how the comparison controls for this additional source of supervision.

  4. Evidence request: comparison with independently tuned confidence routing. Tables 10–12 state that the retrained variants reuse the full model's per-split hyperparameters. Does FairRouter retain its advantage when the confidence-router baseline receives its own downstream hyperparameter selection under the same validation-search budget? This would help determine whether the ablation isolates routing quality or also reflects a downstream configuration selected for the full method.

3 comments

Response for reviewer MTfU (1/2)

We sincerely thank the reviewer for the thoughtful evaluation and for recognizing our Trust–Preference design, empirical gains, and practical potential of FairRouter. We address each concern below and have incorporated the suggested improvements into the revised manuscript.

Q1: Clarification: supervision budget.

We thank the reviewer for requesting this clarification. Source and target datasets use the same k=3/5/10k=3/5/10 support nodes per class, capped by availability. Table Q1-1 reports unique labeled-node counts for all scorers in Tables 4–5. Transfer/unified pretraining uses only support labels from the other four datasets, excluding validation/test labels.

Table Q1-1. Number of unique labeled nodes used by each scorer in Tables 4–5.

TargetTarget support (k = 3/5/10)Unified-pretraining support, all four source datasets (k = 3/5/10)
Cora21 / 35 / 70239 / 392 / 760
Citeseer18 / 30 / 60242 / 397 / 770
PubMed9 / 15 / 30251 / 412 / 800
Arxiv120 / 200 / 400140 / 227 / 430
Products92 / 147 / 270168 / 280 / 560

Q2: Evidence request: attribution to routing versus transfer and specialization.

Sub-question 1: How much of the reported improvement remains when the router is trained only on the target support set?

Following the reviewer’s suggestion, we retrain the router using only the target support set, fixing both acceptance ratios at 15% and keeping all other settings unchanged. Removing cross-dataset pretraining yields only minor performance drop while retaining gains over GNN-as-Judge (Table Q2-1), indicating that FairRouter’s overall advantage does not depend on this pretraining.

Table Q2-1. Cross-dataset pre-training ablation results (averaged over five datasets and three seeds).

Configuration3-shot5-shot10-shot
FairRouter (original configuration)74.7975.5276.15
Target-only router73.3974.8775.37
GNN-as-Judge71.2373.5874.79

Besides, we respectfully note that prior work also deploys cross-dataset pretraining for few-shot graph learning (Zhao et al., 2024; Liu et al., 2024). In FairRouter, it is confined to the router, transferring expert-reliability patterns rather than label semantics. We thank the reviewer and clarify this protocol in Section 3.2 of the revised paper (paragraph “Unified Pretraining”, marked in red).

[1] Zhao, Haihong, et al. "All in one and one for all: A simple yet effective method towards cross-domain graph pretraining." Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2024.

[2] Liu, Hao, et al. "One for all: Towards training one graph model for all classification tasks." International conference on learning representations. Vol. 2024. 2024.

Sub-question 2: what happens when the scorer family and feature configuration are fixed across datasets?

Following the reviewer’s suggestion, we evaluate a uniform router with the same scorer family—logistic regression (L2-regularized, C = 0.1) and the same probability-derived compact features (15 dimensions for agreement and 24 for disagreement) across all datasets. FairRouter retains gains over strongest baseline GNN-as-Judge (Table Q2-2), suggesting that FairRouter’s advantages do not rely on dataset-specific choices.

Table Q2-2. Uniform configuration results and comparisons.

Configuration3-shot5-shot10-shot
FairRouter (original configuration)74.7975.5276.15
Uniform router, cross-dataset pre-training73.1074.0975.84
Uniform router, target-only training72.7774.3375.39
GNN-as-Judge71.2373.5874.79

Response for reviewer MTfU (2/2)

Q3: Clarification: validation-based selection.

We thank the reviewer for requesting clarification on the effective search budgets and comparable validation access & selection effort. We provide additional experiments to show how we control for the validation-based selection.

(a) Effective search budgets. The search budgets and procedures in Table Q3-1.

Table Q3-1. Effective search budgets.

Component / stageSearch budgetChoices explored
Router scorers≤30 initial + ≤20 refined candidates per headScorer configurations for agreement, trust, and preference
Residual: core search27 combinationsEmbedding dimension, learning rate, and pseudo-label weight
Residual: training strategies12 strategiesGate learning rate, warm-up, gold-only training, and auxiliary CE/KL terms
Residual: local refinementCounted in the 39–51 total configurationsRefinement of selected configurations
Residual: loss ablationsCounted in the 39–51 total configurationsRemoval of individual loss terms and combinations of auxiliary regularizers

(b) Comparable validation access and selection effort. FairRouter and GNN-as-Judge share support/validation sets across all 45 splits; GNN-as-Judge also selects checkpoints by validation accuracy. Other baselines use official implementations and recommended hyperparameters on the same splits.

(c) Controlling validation-based selection. We further restrict additional FairRouter selection to k validation labels per class: as shown in Table Q3-2, FairRouter still exceeds baselines with 7–56× fewer validation labels on Planetoid (e.g., 21 instead of 500 on Cora at 3 shots). FairRouter’s gains persist with reduced search under matched validation-label budgets.

Table Q3-2. Accuracy(%) under k/class validation budgets.

Avg. accuracy (%)3-shot5-shot10-shot
GNN-as-Judge (full validation)71.2373.5874.79
FairRouter (full validation)74.7975.5276.15
FairRouter (k/class validation)73.7373.7975.44

Q4: Evidence request: comparison with independently tuned confidence routing.

Following the reviewer’s suggestion, we compare FairRouter with raw confidence while keeping downstream classification unchanged. FairRouter gains 1.69–2.15 pp across all settings (Table Q4-1), including 5.33 pp on Cora (2,708 nodes) and 3.13 pp on ogbn-products (54,025 nodes) at 5-shot, supporting the effectiveness of its expert selection and reliability estimation.

Table Q4-1. Accuracy(%) comparisons with confidence router.

Average accuracy (%)3-shot5-shot10-shot
FairRouter74.7975.5276.15
Independently tuned confidence router72.6773.3774.46
FairRouter gain (pp)+2.12+2.15+1.69

W1: The evidence does not isolate the proposed few-shot router from additional supervision and configuration selection.

The reviewer raises two main aspects in weakness 1: (1) dataset-specific scorer families, feature modes, cross-dataset supervision, and transfer procedures; (2) additional validation-based selection of configurations.

To address these concerns, (1) we implement a target-only, fixed-family router, demonstrating its effectiveness without cross-dataset supervision or dataset-specific selections (see Q2); (2) we also conduct a comparison with matched label and selection budgets, showing that our gains do not depend on additional validation-based selection (see Q3).

W2: Prototype-based router training and calibration have a consequential support/query mismatch.

We thank the reviewer for this precise observation. We therefore evaluate a stricter protocol that excludes each training node from its own references and rebuilds feature transformations, keeping all other configurations unchanged. Under this protocol, the performance slightly improves (mean change is +0.11 pp, Table W2-1). These results show that FairRouter’s gains persist under the stricter protocol.

Table W2-1. Prototype feature repair results at 3/5/10 shots × 3 seeds

Accuracy (%)ShotMatched controlLOO-correctedChange (pp)
Citeseer372.83±0.8672.97±0.87+0.14
Citeseer572.63±1.0973.43±1.04+0.80
Citeseer1073.47±1.2073.10±0.94−0.37
Arxiv358.67±0.9058.67±0.900.00
Arxiv559.90±0.9659.87±0.82−0.03
Arxiv1060.97±0.1761.10±0.36+0.13

Dear Reviewer MTfU,

We sincerely appreciate your time and effort in reviewing our manuscript.

We have submitted our responses to your feedback and revised manuscript. With three days left in the discussion period, we would like to confirm whether these revisions and clarifications resolve your concerns.

If you have any remaining questions, please let us know and we're willing to address them promptly.

Thank you again for your valuable feedback.

Best regards,

Authors

Reviewer_kAvcOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FairRouter, a framework for adaptive modality selection and pseudo-label generation in few-shot node classification on text-attributed graphs (TAGs). The method considers the varying reliability of textual semantics and graph structural information across different nodes, and introduces a node-wise adaptive routing mechanism to dynamically select the more reliable modality for pseudo-label generation. Furthermore, the authors introduce a multimodal residual classifier that integrates complementary representations from the GNN and LLM to refine the original predictions and potentially correct cases where both modalities fail to predict the ground-truth class.

Critical strengths
  1. The paper identifies an important limitation of existing GNN–LLM collaborative approaches, which typically rely on a fixed modality as the annotator or judge. The proposed node-wise adaptive routing mechanism accounts for the varying reliability of textual semantics and graph structural information across different nodes, making the pseudo-label generation process more flexible and adaptive.
  2. The proposed Trust–Preference factorization explicitly separates two different decisions: whether a node is reliable enough to be pseudo-labeled and which modality should be followed when the two experts disagree. This distinction provides a clear and interpretable formulation of modality selection.
  3. The authors evaluate FairRouter on five TAG benchmarks under different few-shot settings (3-, 5-, and 10-shot) and compare it with various GNN, LLM, and collaborative pseudo-labeling baselines. The reported results demonstrate consistent improvements in overall classification accuracy and pseudo-label precision.
Critical weaknesses
  1. The authors first rank the unlabeled nodes according to their estimated reliability scores and then introduce separate acceptance ratios for the agreement and disagreement branches to control the number of selected pseudo-labels. However, according to the experimental settings (Table 7), these acceptance ratios are independently optimized on the validation set for each dataset, label budget, and random seed. This indicates that the proposed pseudo-label selection strategy relies heavily on split-specific hyperparameter tuning, rather than providing a stable and transferable selection mechanism. Moreover, the substantial variations in the selected acceptance ratios across different settings raise concerns about the sensitivity and generalizability of the proposed strategy.
  2. The residual classifier is designed to predict classes beyond the top-one predictions of both modalities. However, there appears to be a potential conflict between this objective and the supervision strategy in Eq. (6). Specifically, the disagreement loss (L_{\mathrm{dis}}(p^R)) directly supervises the residual classifier using pseudo-labels selected from either the GNN or the LLM. Consequently, this loss encourages the residual classifier to reproduce one of the experts' top-one predictions, rather than explicitly learning to recover alternative classes missed by both modalities. Although the gold-label supervision term (L_{\mathrm{sup}}(p^R)) may help the residual classifier learn complementary class-discriminative patterns, its effectiveness could be limited under the few-shot setting, where only a small number of labeled examples are available. .
  3. The proposed method selects reliable pseudo-labels based on the relative ranking of router scores, with separate acceptance ratios for the agreement and disagreement branches. However, relying solely on relative rankings does not necessarily guarantee the absolute reliability of the selected samples. For example, during the early stages of router training, when the router may not yet be able to accurately distinguish reliable samples from unreliable ones, the Top-K strategy may still accept the highest-ranked samples even if their reliability scores are relatively low in absolute terms. This could introduce noisy pseudo-labels into the subsequent training of the residual classifier, potentially compromising its optimization and final prediction performance.
Questions for authors
  1. Regarding Weakness 1, it would be helpful to evaluate the robustness of FairRouter under fixed acceptance ratios or a more generalizable, data-driven ratio selection strategy.
  2. Regarding Weakness 2, could the authors provide further theoretical insights or empirical evidence to explain how the residual classifier learns complementary class-discriminative information and corrects errors shared by both modalities?
  3. Regarding Weakness 3, could the authors clarify how the proposed Top-K selection mechanism prevents unreliable pseudo-labels from being accepted when the overall reliability scores within a branch are low?
  4. For disagreement nodes, the authors decompose modality selection into two stages: a trust scorer first determines whether at least one modality provides a correct prediction, and a preference classifier then determines which modality to trust, conditioned on the existence of a correct candidate. However, it is unclear whether this factorization is necessary, as the same problem could be formulated as a straightforward three-class classification task, with the three categories corresponding to GNN-trust, LLM-trust, and neither-correct. It would be helpful if the authors could further clarify the advantages of combining trust and preference in this factorized formulation.
4 comments

Response for reviewer kAvc (1/3)

We sincerely thank the reviewer for recognizing the value of node-wise adaptive routing, our clear and interpretable Trust–Preference factorization, and consistent gains across five TAG benchmarks under multiple few-shot settings. We address each concern below.

Q1 / W1: It would be helpful to evaluate the robustness of FairRouter under fixed acceptance ratios or a more generalizable, data-driven ratio selection strategy.

We thank the reviewer for this suggestion. Following reviewer's suggestion, we evaluate FairRouter under fixed acceptance ratios (top 15% in each branch) with all other settings unchanged. Across 3/5/10-shot settings, performance drops by only 0.09–0.23 pp on evaluation nodes and the full query set (Tables Q1-1 and Q1-2), while FairRouter still outperforms the strongest baseline by 1.15–3.40 pp (Table Q1-1), supporting FairRouter's robustness under fixed acceptance ratios.

Table Q1-1. Accuracy (%, mean ± population std over three seeds). Δ = Fixed 15% − Paper (pp).

ShotMethodCoraCiteseerPubMedArxivProductsAvg.
3GNN-as-Judge74.37±1.1964.07±1.8288.93±1.3053.80±3.8275.00±1.7171.23
3FairRouter(paper)79.37±0.1773.37±0.1290.80±1.0458.13±0.1272.30±0.8574.79
3FairRouter(Fixed 15%)79.13±0.2172.90±0.7990.93±0.6158.47±1.0871.73±0.6674.63
3Δ−0.24−0.47+0.13+0.34−0.57−0.16
5GNN-as-Judge76.20±2.3365.90±4.1990.80±1.5559.90±1.2275.10±0.9573.58
5FairRouter(paper)80.70±1.9172.60±0.7991.47±1.3060.40±1.5672.43±0.4175.52
5FairRouter(Fixed 15%)79.67±1.7673.23±0.6191.60±1.4160.90±1.4471.77±0.3775.43
5Δ−1.03+0.63+0.13+0.50−0.66−0.09
10GNN-as-Judge79.17±1.2368.17±1.0191.43±1.1058.80±5.4076.37±0.6574.79
10FairRouter(paper)81.27±1.3673.47±1.2591.63±0.8361.60±0.8672.80±0.8276.15
10FairRouter(Fixed 15%)80.83±1.2073.73±1.3890.90±1.3661.60±0.8372.63±0.5375.94
10Δ−0.44+0.26−0.730.00−0.17−0.21

Table Q1-2. Accuracy (%) on the full query set (all nodes outside the support and validation sets), 3/5/10 shots.

MethodCoraCiteseerPubMedArxivProducts
Paper79.36 / 80.35 / 81.2973.48 / 72.60 / 73.6690.75 / 91.19 / 91.1458.78 / 60.57 / 62.1872.46 / 73.08 / 73.39
Fixed 15%79.23 / 79.34 / 80.7572.98 / 73.23 / 73.6790.81 / 91.22 / 90.5459.12 / 60.98 / 61.9872.08 / 72.32 / 73.59

Response for reviewer kAvc (2/3)

Q2 / W2: Could the authors provide further theoretical insights or empirical evidence to explain how the residual classifier learns complementary class-discriminative information and corrects errors shared by both modalities?

We thank the reviewer for raising this concern and respectfully clarify our design with additional empirical validation.

(a) Explanation of how the residual classifier works.

The residual classifier receives multi-modal embeddings to learn joint decision boundaries unavailable to either expert individually. With sufficient pseudo-label supervision covering relevant discriminative patterns, residual classifier can generalize to nodes misclassified by both experts. (Cheng et al., 2020, Theorem 1). Thus, expert-derived supervision and complementary correction are compatible.

The fusion in Eq. (5) also permits this correction to affect the final prediction. For a true class yy to beat a competing class cc, it is sufficient that piR(y)−piR(c)>α[piG(c)−piG(y)]+β[piL(c)−piL(y)].p_i^R(y)-p_i^R(c) > \alpha\bigl[p_i^G(c)-p_i^G(y)\bigr] + \beta\bigl[p_i^L(c)-p_i^L(y)\bigr]. This permits recovery even when neither expert ranks yy first.

[1] Cheng, Jiacheng, Tongliang Liu, Kotagiri Ramamohanarao, and Dacheng Tao. "Learning with bounded instance and label-dependent label noise." In International conference on machine learning, pp. 1789-1799. PMLR, 2020.

(b) Empirical evidence for learning complementary class-discriminative information.

We supplement an ablation study, removing pRp^R from the final prediction at inference. Retaining pRp^R improves performance by 2.17–2.94 pp across all few-shot settings (Table Q2-1), suggesting that the residual classifier learns complementary class-discriminative information.

Table Q2-1. Average accuracy (%) over five datasets, original three-seed setting.

Prediction3-shot5-shot10-shot
Full prediction74.7975.5276.15
w/o pRp^R71.8573.3573.46
Difference attributable to retaining pRp^R at inference (pp)+2.94+2.17+2.69

(c) Empirical evidence for correcting errors shared by both modalities.

Following the reviewer’s suggestion, we evaluate nodes misclassified by both experts. Across 3/5/10-shot settings, FairRouter corrects 215–221 disagreement nodes, versus 20–36 for a convex mixture of the experts, and 42–45 erroneous agreement nodes (Table Q2-3), demonstrating its ability to correct errors shared by both modalities.

Table Q2-3. Corrections on shared-error nodes; counts pooled over five datasets × three seeds.

ShotStratum#NodesBaseline correctFairRouter correctRepairHarmNet
3Agree & wrong1,045042420+42
3Disagree, both wrong1,8762922120412+192
5Agree & wrong1,051043430+43
5Disagree, both wrong1,764202202077+200
10Agree & wrong1,064045450+45
10Disagree, both wrong1,5823621519415+179

Response for reviewer kAvc (3/3)

Q3 / W3: Regarding Weakness 3, could the authors clarify how the proposed Top-K selection mechanism prevents unreliable pseudo-labels from being accepted when the overall reliability scores within a branch are low?

We sincerely thank the reviewer for raising this critical concern about the reliability in FairRouter. We respectfully clarify that FairRouter uses a two-stage filtering mechanism to limit unreliable pseudo-label acceptance. It first selects the more reliable expert and discards the other unreliable one, then ranks nodes by learned reliability and accepts only the top-ranked nodes.

We further provide a hyperparameter study by varying the accepted coverage and observe that precision decreases monotonically with increasing coverage across all label budgets (Table Q3-1). This indicates that the router concentrates correct pseudo-labels near the top, allowing Top-K selection to filter out less reliable nodes.

Table Q3-1. Pseudo-label precision under varying accepted coverage. Results are averaged over five datasets and three seeds.

Coverage of disagreement branch5%15%30%50%100%
3-shot precision (%)89.085.379.474.658.4
5-shot precision (%)87.584.277.871.558.6
10-shot precision (%)86.181.476.170.457.1

Q4: Why decompose trust and preference instead of using a three-way classifier?

We thank the reviewer for this insightful question and we are very willing to explain our motivation for this design. The decomposition improves interpretability by separating the three disagreement states into two questions: Trust asks whether either expert provides a correct label, indicating whether single modality is reliable for the node; Preference asks which expert to follow, reflecting the relative strength of the two modalities. This yields two meaningful, separately testable signals.

Using preference as an example, we empirically compare it with raw expert confidence and find it substantially more reliable for modality selection (Table Q4-2). This demonstrates its independent value in deciding which expert to follow.

Table Q4-2. Results below are averaged over 15 runs per dataset.

Disagreement precision at 5% / 20% coverage (%)Learned preferenceConfidence-based preference
Cora94.40 / 88.9644.49 / 44.86
Citeseer77.34 / 68.4749.12 / 49.73
PubMed92.46 / 90.5074.01 / 72.48
Arxiv77.49 / 69.3658.30 / 55.87
Products86.94 / 81.5877.84 / 73.74
Average over all 75 runs85.73 / 79.7860.75 / 59.34

Following the reviewer’s suggestion, we compare our decomposed router with a three-way classifier and observe comparable performance (−0.07/+0.14/+0.03 pp relative to decomposition at 3/5/10 shots). This is expected, as both formulations model the same three states through different parameterizations.

Dear Reviewer kAvc,

We sincerely appreciate your time and effort in reviewing our manuscript.

We have submitted our responses to your feedback and a revised manuscript. With three days remaining in the discussion period, we would like to confirm whether our responses have effectively addressed your concerns.

If you have any remaining questions, please let us know and we're very willing to address them promptly.

Thank you again for your valuable feedback.

Best regards,

Authors

Comments on the paper

7 comments

Response for reviewer RLTp (1/4)

We sincerely thank the reviewer for recognizing our clear problem formulation, modular Trust–Preference design, broad empirical evaluation, and detailed analyses of pseudo-label quality. We carefully address each concern below.

Weakness 1: The novelty and definition of “fairness” are not fully established

We sincerely thank the reviewer for this insightful question. The word "fair" in FairRouter refers to fair treatment of the two modalities in TAG few-shot learning. Concretely, it means three properties:

  1. No prior role assignment. Neither model is designated as annotator or judge. Which modality supplies a node's pseudo-label is learned from labeled observations, in contrast to fixed-teacher methods such as GNN-as-Judge.
  2. A symmetric decision rule. Given trust t_i and preference r_i, the rule q_G = t_i·r_i, q_L = t_i(1 − r_i) treats the two modalities symmetrically under r ↔ 1 − r, with the boundary at r = 0.5.
  3. Balanced supervision. The disagreement loss averages the GNN-selected and LLM-selected pools separately (Eq. 58), acting as modality-balanced supervision who treats the two expert-selected pseudo-label pools equally.

We agree that "modality-fair" or "node-adaptive" is a precise description. We have added this definition to Section 3.2 of the revised paper (marked in red) and clarified the terminology.

Weakness 2: The comparison with GNN-as-Judge is not fully symmetric

We thank the reviewer for this insightful observation. Following reviewer's suggestion, we provide additional experiments to isolate the routing and fusion strategy. As shown in Table W2-1, FairRouter pseudo-labels outperform GNN-as-Judge's in 6 of 9 settings with residual classifier; with the same fine-tuned LLM, FairRouter remain competitive, improving Citeseer by up to 6.26 pp and Products at 3 and 5 shots, while trailing on Cora. We observe that the advantage of routing holds under different fine-tuning and computational budgets.

Table W2-1. Accuracy (%) under matched supervision counts and learner settings. PL: pseudo-labels; GaJ: GNN-as-Judge.

DatasetShotFairRouter PL → Frozen experts + residualGaJ PL → Frozen experts + residualFairRouter PL → LLM finetunedGaJ PL → LLM finetuned
Cora378.5776.5770.8074.07
Cora579.6078.1369.3771.03
Cora1081.1080.2074.4777.40
Citeseer373.4073.2369.6363.37
Citeseer571.4771.5767.4066.80
Citeseer1072.4772.4068.2368.67
Products369.3069.3772.7372.03
Products570.2069.7773.9772.93
Products1071.8072.0072.2775.17

Response for reviewer RLTp (2/4)

Weakness 3: Dataset-specific configuration may limit the generality of the conclusions

We thank the reviewer for this important concern. We address each of the four points below.

(1) Which labels are used to fit each component. The table below lists, for each component, which labels it uses. Query/test ground-truth labels never enter any fitted parameter, feature transformation, calibration, pseudo-label pool, or residual checkpoint selection.

Table W3-1. Label usage of each component.

ComponentTarget support labelsOther datasets' support labelsTarget validation labelsQuery (test) labels
Frozen GNN classification head✓ (fit)✗checkpoint selection✗
Router heads (agreement / trust / preference)✓ (fit)only unified-pretraining/pooled/transfer heads✗✗
Feature transformations (moments, column filtering, rank transform)✓as for the corresponding head✗✗
Out-of-fold calibration / training-score normalization✓as for the corresponding head✗✗
Residual classifier and fusion parameters✓ (fit)✗checkpoint selection✗ (query features are used transductively; labels are not)
Pseudo-label precision / accuracy reporting———✓ (evaluation only)

(2) Whether target-dataset validation labels are used. Yes; validation labels are used only for selection: acceptance ratios, router hyperparameter selection, residual-classifier configuration, and checkpoints. All baselines use the same validation split.

(3) Whether configuration selection is performed independently for every seed. Yes, selection is performed independently for different few-shot setting (k-shot split changing with seeds).

(4) How a single shared router performs across all datasets. Following reviewer's suggestion, we evaluated a uniform router shared across all five datasets, i.e., logistic regression on the same compact probability features, with both acceptance ratios fixed at 15%. As shown in Table W3-2, FairRouter is still superior over strongest baselines based on a single shared router.

Table W3-2. Uniform configuration results and comparisons.

ShotMethodCoraCiteseerPubMedArxivProductsAvg.
3Locle74.30±1.9069.30±0.3772.97±1.9251.70±0.9863.80±1.6366.41
3GNN-as-Judge74.37±1.1964.07±1.8288.93±1.3053.80±3.8275.00±1.7171.23
3FairRouter (paper)79.37±0.1773.37±0.1290.80±1.0458.13±0.1272.30±0.8574.79
3FairRouter (uniform)78.63±0.6973.33±0.4687.47±2.1354.87±0.2969.53±0.5872.77
5Locle75.77±1.4469.07±0.5676.87±1.7952.37±0.8464.07±0.6367.63
5GNN-as-Judge76.20±2.3365.90±4.1990.80±1.5559.90±1.2275.10±0.9573.58
5FairRouter (paper)80.70±1.9172.60±0.7991.47±1.3060.40±1.5672.43±0.4175.52
5FairRouter (uniform)79.30±2.0574.30±0.5089.67±2.1757.67±1.4670.70±0.9974.33
10Locle78.30±1.3068.87±0.5276.40±1.6956.23±1.1365.37±1.1469.03
10GNN-as-Judge79.17±1.2368.17±1.0191.43±1.1058.80±5.4076.37±0.6574.79
10FairRouter (paper)81.27±1.3673.47±1.2591.63±0.8361.60±0.8672.80±0.8276.15
10FairRouter (uniform)81.03±1.7674.70±0.7390.33±2.1460.57±0.9670.33±1.3575.39

Response for reviewer RLTp (3/4)

Question 1: What are the exact data boundaries?

We kindly refer the reviewer to our response to Weakness 3:Table W3-1, where this question is clarified in detail.

Question 2: What is the precise cross-dataset training protocol?

We thank the reviewer for this question and summarize the cross-dataset training protocol of each scorer below.

  • Unified pretraining only: router pre-trained on the other four datasets' support observations.
  • Target only: router fit on the target support observations.
  • Pooled: router pre-trained on all support observations, with target mass 0.25 and each other dataset 0.1875.
  • Transfer: router unified pretraining, then target adaptation.

All four protocols use only support labels. The protocol differs by dataset and head (Appendix B, Tables 4–5).

We also provide a version in which the protocol is identical across datasets: both target-only for all and pooled for all, under the uniform router. The two differ by only −0.33/+0.24/−0.45 pp, showing that the choice of protocol is not critical.

Question 3: What exactly does “fair” mean?

We kindly refer the reviewer to our response to Weakness 1, where this concern is addressed in detail.

Question 4: How sensitive is the LLM scoring to class-number tokens and prompt design?

1. For datasets with more than nine classes, are all category numbers represented as single tokens?

We would like to clarify that every class code is a single, unique token for all three LLMs:

ModelClass label tokens
LLaMA-3-8B-InstructNumber tokens, e.g., "1", "40", "47"
Qwen3-8BCase-sensitive letter tokens: A…Z, a…u (up to 47 classes)
Mistral-7B-Instruct-v0.3Case-sensitive letter tokens A…Z, a…u, using the space-prefixed variants, e.g., " A", " a"

2. Are results stable under class-number permutations and alternative prompt templates?

To assess this, we re-ran FairRouter with LLaMA-3-8B under three random permutations of the class numbers and one alternative prompt template, keeping all other settings unchanged. From Table Q4-1, we can observe that:

  • Permutations. Across the three permutations, the mean accuracy varies by 0.77 pp (standard deviation across permutations) for the LLM and only 0.41 pp for FairRouter.
  • Template change. The alternative template lowers the LLM's accuracy by 6.97 pp, but FairRouter's by only 0.44 pp despite larger variation. Overall, FairRouter is insensitive to the class-token assignment and prompt template owing to the complementarity between the two experts.

Table Q4-1. Sensitivity of FairRouter and Direct LLM to class-number ordering and prompt template.

Accuracy (%), Cora 5-shotDirect LLMFairRouter
Original prompt (re-run)66.67±1.6679.97±0.74
Class-number permutation 167.97±1.3979.93±0.69
Class-number permutation 266.57±1.5379.93±1.43
Class-number permutation 366.17±1.6180.80±1.63
Alternative template59.70±1.3079.53±2.22

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.