- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper studies misclassification detection for semi-supervised node classification. It argues that prior methods primarily parameterize uncertainty over model weights and posterior predictive distributions, which is insufficient when errors arise in heterophilic neighborhoods. The proposed DA-BGNN augments a variational Bayesian GCN with a composite score that combines Monte Carlo predictive entropy, epistemic variance, a node-level heterophily ratio , and a representation-shift term , implemented as a normalized sum. Experiments on Cora, CiteSeer, PubMed, and Amazon Computers show the best AUROC/AUPRC for misclassification detection (Table 2) while remaining competitive in accuracy and ECE.
Critical strengths
-
Confident misclassification of GNNs is a real risk for downstream applications such as fraud detection and medical diagnosis. Improving misclassification detection on node classification tasks is a well-motivated direction.
-
The empirical motivation is clear: neighborhood entropy is 3–11× higher in high-heterophily regions (Table 8, Fig. 1). The ablation (Table 3) isolates heterophily as the dominant contributor (AUPRC 0.445 → 0.522).
Critical weaknesses
W1. The paper reports the AUPRC gain as about 16% in the abstract, 21.0% / 18.8% in Table 4, and about 34% in Section 5 (vs. BGNN in Table 2). The ECE reduction is reported as 25% in the abstract and 19.6% in Table 4, while Table 2 implies roughly 62% relative to GCN (0.099 → 0.038). Appendix K.4 acknowledges that the percentages "depend on the chosen baseline." The authors should state one clearly defined headline comparison (fixed baseline, fixed setting) and use it consistently across the abstract, main text, and tables.
W2. Presentation needs substantial improvement. (i) Broken cross-references ("Equation ??") appear twice on page 4. (ii) There are numerous typos, e.g., "appraoch", "seperates", "EXPERIEMNTAL", "Laslty", "catach", "SOT results", "replied on". (iii) The introduction is written informally ("Why?", "There is one more question to ask ourself"). (iv) The text states that "All proofs are supplied in Appendix," but the paper contains no theorem or proof. (v) Appendix D contains a textbook-style numeric forward-pass example that does not relate to the contributions. (vi) The Ethics and Reproducibility statements are empty.
Questions for authors
Q1. Discrepancy between Table 2 and Table 3. The full DA-BGNN reports 0.880 / 0.571 (AUROC / AUPRC) in Table 2 but 0.854 / 0.510 in Table 3. Both captions state an average over the same four datasets. Averaging the per-dataset results in Table 5 reproduces Table 3, but not Table 2; the same mismatch appears for the GCN and BGNN rows. Could the authors clarify which setting (splits, perturbation, score variant) Table 2 reports? Relatedly, Table 3 suggests that adding the representation-shift term slightly reduces performance compared with "+ heterophily " (0.861 / 0.522). What is the justification for including in the final score?
Q2. Computation of may cause possible label leakage. Eq. 50 defines with the node's own label , which is unavailable for test nodes. Section 3.1 states that heterophily statistics use only training and validation labels. Please confirm that no test labels are used at any stage. This matters because is the dominant source of improvement in Table 3.
2 comments
Rebuttal (p.2)
Q1 and Q2
Ablation: mapping 0.880 / 0.571 vs 0.854 / 0.510
Mean over the four datasets:
| Variant | AUROC | AUPRC |
|---|---|---|
| Entropy only () | 0.80 | 0.45 |
| BGNN () | 0.80 | 0.42 |
| heterophily | 0.86 | 0.52 |
| Equal-weight full () | 0.85 | 0.51 |
| Validation-tuned full | 0.88 | 0.57 |
Your concern is exactly what this table shows. The per-dataset equal-weight appendix averages to the equal-weight family (≈ 0.85 / 0.51), not to the tuned row. Heterophily is the main clean equal-weight lift, tuning then moves the full feature set further.
Why keep after the equal-weight dip
Under equal weights you are right: (0.86 / 0.52) beats equal-weight full with (0.85 / 0.51). We do not claim that putting weight one on every coordinate is optimal.
We still keep for two reasons.
- The score we actually use is not equal-weight. Validation-tuned fusion over reaches 0.88 / 0.57, which is the decimal counterpart of the story behind the main 97.3 / 83.7 calibrated mean. Reweighting can keep a small weight on when it helps a subset of nodes without forcing weight one everywhere.
- Under shift, helps. With FGSM , adding to improves AUPRC on every dataset we report: Cora 0.44→0.46, CiteSeer 0.47→0.49, PubMed 0.37→0.39, Amazon 0.43→0.45. The residual is built for representation shift, that is where it earns its keep.
In other words, heterophily carries the result: is justified by tuning and by the perturbation table. Leaving the equal-weight regression in the paper is more honest than deleting and pretending the full model is only .
GCN / BGNN mismatches
Score variant cannot move GCN. If GCN or BGNN also differed across tables under captions that both claimed the same four-dataset average, that was a caption / aggregation error in the submitted draft—not a second experimental setting. The revision uses one protocol for every baseline row and separates tuned vs equal-weight DA-BGNN in the wording. The DA-BGNN gap you highlighted is explained by the score head, the baseline mismatch is acknowledged and corrected.
Q2. Does computing use test labels?
No. This is important precisely because is the dominant equal-weight gain in Table 3 (about 0.80 / 0.45 → 0.86 / 0.52). If that had used test labels, the ablation would be invalid.
The equation that writes with the node’s own label is the population definition:
We do not evaluate that formula on . At inference we use a label-safe estimator from training and validation labels only, . For a test node we look at neighbors that already have observed labels and compare them to the mode of those labels—never to :
\hat{\alpha}_i = 1 - \frac{\left|\left\\{j \in \mathcal{N}_{\mathrm{obs}} : \tilde{y}_j = \mathrm{mode}(\tilde{y}_{\mathcal{N}_{\mathrm{obs}}})\right\\}\right|}{\left|\mathcal{N}_{\mathrm{obs}}\right|},
with a median fallback if there are no observed neighbors. Example: observed neighbor labels give . The test label is never read.
The same observed-only discipline applies downstream. The heterophily law and the percentile are formed from observed nodes; the residual center for uses correctly classified observed embeddings. None of , , , or the final score reads labels from at any stage of training, validation tuning, or test scoring.
Section 3.1 already stated that heterophily statistics use training and validation labels; the submitted draft still made it too easy to confuse with . The revised PDF places the estimator next to the population equation and spells the procedure in Algorithm 1.
We would appreciate if you could raise your scores accordingly. Thank you.
Best regards,
Authors
Rebuttal (p.1)
Dear Reviewer BeZ8,
We sincerely thank you for your careful reading and for recognizing that confident GNN misclassification is a real risk, as well as for noting that our ablation isolates heterophily as the dominant contributor. Your rating is clear and your concerns are concrete, we treat them seriously. We have carefully revised the manuscript accordingly. Please refer to the updated PDF.
Below we provide point-by-point responses to all weaknesses and questions.
W1: (abstract / Table 4 / Section 5 / ECE claims)
We agree that the submitted draft mixed baselines and settings with gains. The revision has adopted one headline comparison and uses it consistently:
Under that definition, AUPRC goes 54.8 → 83.7 (≈ 53% relative), AUROC 85.6 → 97.3, while Acc/ECE stay at the frozen BGNN values (81.1 / 0.029). We no longer promote ECE “reduction by DA-BGNN”, because the method is post-hoc and does not change predictions/calibration of the backbone.
Appendix shows percentages that“depend on the chosen baseline”. Please read the revised PDF:
Main table:
W2 Presentation
We agree presentation was below standard in the submitted draft. The revision (i) fixes broken equation references, (ii) corrects the listed typos, (iii) tones down informal intro phrasing, (iv) makes Theorems 1–3 / Lemma 1 and their proofs explicit in the appendix (so “All proofs are in the Appendix” is no longer vacuous), (v) shortens / relocates the textbook forward-pass example so it no longer distracts from the contribution, and (vi) fills Reproducibility (and Ethics, as required) with concrete pointers to splits, seeds, Algorithm 1, and anonymized code.
(i) Broken cross-references (“Equation ??”).
Caused by duplicate / missing equation labels in the draft. The revision keeps a single
\label{eq:bayesian}(and related labels) and verified references on page 4 resolve.(ii) Typos (
appraoch,seperates,EXPERIEMNTAL,Laslty,catach,SOT results,replied on, etc.).All listed instances and nearby paste artifacts are corrected in the revised PDF. We additionally proofread captions and appendix tables for similar errors.
(iii) Informal introduction.
We rewrite rhetorical asides (“Why?”, “ask ourself”) into standard academic prose stating the research questions and contributions directly.
(iv) “All proofs are supplied in Appendix” with no theorem.
Fair criticism of the submitted draft. The revised appendix now contains:
with proofs under the corresponding appendix subsections. The main-text pointer now matches actual content.
(v) Appendix D textbook forward-pass.
We agree it did not support the contribution. In the revision it is removed or reduced to a short pointer so the appendix focuses on Algorithm 1, label-safe α^, ablations, and theory.
(vi) Empty Ethics / Reproducibility statements.
Reproducibility now points to appendix details (datasets/splits, models, Algorithm 1) and the anonymized repository: https://anonymous.4open.science/r/DA-GNN. Ethics is filled with a standard statement that we use public citation/co-purchase benchmarks and release no private personal data, dual-use risk for fraud/medical settings is noted as a motivation for better misclassification detection, not as a claim of clinical deployment.