acceptodds
Withdrawn from ICLR 2027

Beyond Global Confidence: Region-Aware Robust Learning for Graph–LLM Feedback

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Feedback between graph neural networks (GNNs) and large language models (LLMs) can improve average accuracy while leaving graph regions unreliable. We address the gap between selecting confident pairs and controlling their influence on learning through regional feedback allocation: regional contribution, structural pair admission, and residual risk within admitted regions. Stabilized importance weights balance regional influence; frozen-predictor counterfactual trials admit structure-dependent disagreements; and calibrated harmful-feedback estimates set regional KL-DRO budgets. Under matched train-plus-calibration access, our framework outperforms Co-teaching, the principal paired comparator, by 1.20–2.72 percentage points across five primary datasets. Published GNN-as-Judge results descriptively trail ours by 2.6–6.5 accuracy points and 13.8–20.4 support-filtered worst-region accuracy points. Co-teaching is rerun; GNN-Judge uses published results here. Component removals cost 3.3–8.6 points, while held-out admission quality and fixed-radius ablations link these gains to pair selection and regional risk allocation. Benefits extend to heterophilic graphs, another LLM family, and cross-dataset transfer. Few-shot denotes direct training supervision; shared validation labels remain selection-only, and a separate strict- adaptation test removes additional calibration labels. The results show how regional feedback diagnostics can guide training to improve both average accuracy and local reliability.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Graph Learning, Large Language Model, Graph–LLM Feedback

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_eCX2OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies regional imbalance in GNN–LLM feedback for text-attributed graphs: even when average accuracy improves, some graph regions may still receive unreliable preference feedback. The authors construct regions using spectral clustering of frozen GNN representations, the strength of text priors, and local structural consistency. They distinguish three controls: stabilized importance weights regulate regional contributions; prediction sensitivity under degree-preserving rewiring filters preference pairs arising from model disagreements; and harmful-feedback rates estimated on a calibration set, together with structural instability, determine region-specific KL-DRO radii. The final objective is a weighted robust DPO loss over admitted preference pairs, with a small chosen-label NLL term.

Critical strengths
  1. The problem formulation and the distinct levels of intervention are valuable. Separating which feedback enters training from how each region influences training addresses local failures that global confidence filtering may overlook. C1, C2, and C3 act on regional contributions, the admission support set, and the within-region loss distribution, respectively, making their effects individually testable. The authors also explicitly acknowledge that structural sensitivity does not imply label correctness and that TV diagnostics do not establish KL coverage, avoiding overly strong theoretical claims about empirical mechanisms.
Critical weaknesses
  1. The worst-region gains rely on published GNN-as-Judge results without guaranteed common splits or test nodes (Appendix H.1), and their mapping to the frozen (R,T,H) partition is unclear. Rerun Co-teaching lacks worst-region, P10, and CVaR results. Thus, average-accuracy gains are supported, but the central regional-reliability claim remains insufficiently established.

  2. The minibatch implementation’s correspondence to the KL-DRO objective is unclear. With batch size 8 and 828 regions on ogbn-arxiv, a region represented by one loss atom has radius-independent risk. Sampling may also alter regional contributions. Appendix K lacks sufficient sampling, cross-batch aggregation, and correction details to connect minibatch risks to Equations (6)/(7).

  3. Sparse calibration weakens the interpretation of regional adaptation. On ogbn-arxiv, 590/828 regions use global harmful-rate estimates, while 82% of admitted training pairs under strict kC rely on CF-proxy/global fallback. Aggregate results do not separate gains from regional estimates, global fallback, and structural proxies. Results by support condition and a precise CF-proxy mapping are needed to substantiate the adaptive-radius mechanism.

Questions for authors
  1. Could you report worst-region accuracy, P10, CVaR, and intervals for same-seed differences for the rerun Co-teaching baseline, using common test nodes and the same frozen (R,T,H) partition? What node-level or group-level records underlie the “Published” GNN-as-Judge regional results in Table 10, and how are they mapped to your partition?

  2. With a batch size of 8, which loss atoms actually enter each regional risk in Equation (7)? Please specify the per-batch objective, sampling distribution, information carried across batches, and treatment of absent regions, and explain how these correspond to Equation (6). What fraction of region occurrences within batches contain only one loss atom, and how does the radius affect those cases?

  3. Please distinguish the number of admitted calibration pairs from the number of admitted optimization pairs, and report estimation quality and the gains from adaptive versus fixed radii separately for empirical estimates, shrinkage estimates, and global fallback. How is CF-proxy numerically mapped to a harmful-feedback rate under strict kC, and what is the basis for its parameters? Can this proxy be independently validated in the setting with a held-out label pool?

  4. At matched admission coverage and optimization budgets, how does CFdep gating compare with random filtering and filtering based only on calibrated confidence or confidence gaps, in terms of preference quality and final accuracy? Could you also clarify whether the different filters change regional support, so that the additional value of structural information can be assessed?

  5. How many full training runs, or how much selection compute, do the 48 candidate configurations actually require, and what are the corresponding budgets for Co-teaching and the other rerun baselines? Does the average-accuracy gap remain stable with comparable selection compute or an increased search budget for Co-teaching?

Reviewer_Eg85OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper organizes Graph–LLM feedback around three controls: regional contribution, structural pair admission, and residual risk within admitted regions. Stabilized importance weights balance exposure, counterfactual trials filter disagreements, and harmful-feedback estimates set regional KL-DRO budgets for capped DPO losses, with chosen-label NLL outside the robust risk.

Critical strengths
  1. The component and factorial ablations help explain their roles. I also like that the paper checks admitted-pair quality on held-out labels instead of treating structural sensitivity as a correctness guarantee.
  2. The Co-teaching comparison is informative. Calibration-budget and selection-pool checks address alternatives to the proposed explanation.
Critical weaknesses
  1. The main regional claim needs a matched comparison. The worst-region gains over GNN-as-Judge use unaligned splits/seeds.
  2. The adaptive-risk mechanism needs a stronger check. Table 2's fixed-radius ablation does not establish an advantage over a comparably tuned common radius.
Questions for authors
  1. What are the paired worst-region, P10 and CVaR results for Co-teaching and the rerun GNN-as-Judge on the same splits and frozen regions?
  2. Does the adaptive radius still help against a common radius selected from a comparable validation grid, with the same admitted pairs and outer weights?
Reviewer_BkG7OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies few-shot node classification on text-attributed graphs. A GNN and an LLM each label every node. Where they disagree, the more confident model's label becomes the preferred answer, the other becomes the rejected answer, and the LLM is fine-tuned on these pairs with DPO. The paper adds three controls. C1 partitions the graph into regions (GNN cluster × text-prior bin × neighbor-agreement bin) and reweights training toward equal mass per region. C2 rewires the graph 20 times, preserving degrees, and admits a pair only if the frozen GNN's probability of the preferred label moves noticeably. C3 applies a per-region KL-DRO objective whose adversary can shift mass toward the highest-loss pairs, with a radius that grows with the region's estimated rate of wrong pairs and its rewiring instability. Reported results: +1.2 to +2.7 pp over a rerun Co-teaching baseline on five datasets, 13.8–20.4 pp worst-region gains over published GNN-as-Judge numbers, and ablations in which each control matters.

Critical strengths

The studied problem is important. Which pairs to trust and how much influence each region receives are different decisions, and worst-region accuracy is the right lens for the second. The evaluation is broad: five datasets, five seeds, a paired and multiplicity-corrected comparison against a rerun baseline, a full 2×2×2 factorial, a calibration-budget sweep down to zero additional calibration labels, while retaining shared labeled validation pools. The scope of every formal result is stated honestly.

Critical weaknesses
  1. The reweighting (proposed C1) applies smoothed inverse-frequency weights (Eq. 1) over the (R, T, H) partition. The weighting does not establish whether stratum frequency is an appropriate basis for allocating training influence. Specifically, a small region may represent an underserved population, a genuinely difficult topic, a noisy cluster, or an artifact of the partition. But C1 assigns equally prevalent strata the same weight regardless of their reliability. Removing C1 is the largest ablation effect (6.4–8.6 pp, Table 2), yet the only partition check is the boundary sweep in Table 4B, and there is no comparison against class balancing, balancing on H alone, or a random partition of matched granularity.

  2. Regarding the counterfactual structural calibration (C2)

  • C2 measures structural sensitivity, not correctness. In fact, a structure-sensitive prediction can be wrong, and a stable one can be right. . When the LLM supplies the preferred label, the gate probes the GNN’s probability of that label, which does not directly establish its correctness.
  • Experiments show that 16–26% of admitted pairs remain harmful under label-held-out evaluation (Table 3). Moreover, the lift is modest: from Table 12's precision, recall, and coverage, 51–62% of pre-gate pairs are already safe, so the gate raises precision to 0.74–0.84 while discarding about half the pairs.
  • The comparison is missing against simpler selection rules at the same retained-pair count (calibrated confidence, confidence margin, the neighbor-agreement factor H, degree etc.), and precision is not split by which model won the pair.
  1. The radius (C3) has critical issues. C3 relies on the premise that greater contamination warrants greater emphasis on the largest losses, i.e., noisier regions should lean harder on their highest-loss pairs. However, this is questionable and unsupported. Eq. 5 enlarges a region's KL radius with its estimated harmful rate, and the adversary (Eq. 7) spends that radius by up-weighting pairs in proportion to exp(loss/λ). So the more wrong pairs a region is estimated to hold, the more training concentrates on its largest losses. But a large DPO loss can mean a hard-but-correct pair or a mis-oriented one, and loss alone cannot tell them apart. Note Dr. DPO adopts the opposite weighting direction and down-weights high-loss pairs with a soft-minimum. The adversary cannot correct preference orientations or introduce abstention; it only reweights admitted loss atoms (Sec. 3.4). And Lemma 3.3 concerns the per-node KEEP/FLIP/ABSTAIN choice, a different object, so it does not justify the tilt. The experiments do not rescue the premise. With the same gate and no reweighting, fixed-radius KL-DRO is no better than plain DPO (Table 14 row 010 vs Table 2 "−Reweighting & DRO": 73.92 vs 74.23 on Cora, 56.84 vs 58.91 on ogbn-arxiv). The tested fixed radius provides no improvement in reported means without C1. A C1+C2+standard-DPO control is needed to isolate DRO’s contribution within the full pipeline. And the ablation's fixed radius, 0.5, exceeds every adaptive median (0.30–0.41, Table 11B), so the reported 3.9–4.9 pp drop may reflect a smaller radius rather than an adaptive one.

  2. Estimation and implementation limitations. A small empirical region cannot reveal an unobserved population tail, so the adversary can only re-emphasize pairs already present. The implementation is also ambiguous: Appendix K computes KL risks per minibatch (batch size 8) without within-batch renormalization, while Sec. 3.4 fits one λ_s per stratum over the full admitted set. If the normalizer is computed within the batch, a region holding a single pair in a batch has no reweighting freedom regardless of its radius. Please clarify how per-region normalizers are carried across batches, and report how often a region holds more than one pair per update.

  3. The authors did not adequately distinguish the proposed mechanisms from simpler, comparably tuned alternatives: Ours searches 48 configurations, while all baselines except Co-teaching use fixed or official configurations (Appendix K.1). Co-teaching is absent from the principal worst-region comparison (Fig. 2c/Table 10) and regional-noise experiment (Table 9), so the local-reliability claim is never tested against the strongest baseline.

Questions for authors

See weakness section. Also,

  1. How is the class-transition matrix for H-ELSD estimated with three labels per class?
  2. Which calibration-dependent components account for the accuracy increase from 64.82 to 71.35 in Table 6? Can the authors isolate the contribution of improved harmfulness estimation from other changes induced by additional calibration labels?
  3. Define "Full GCN" and "Calib.-rel."; do rDPO and Dr. DPO receive the gated, reweighted pairs or the raw disagreement set?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.