- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies post-hoc refinement of frozen node classifiers using only the graph G and predicted class distributions Q. The method requires no node features, model parameters, or training gradients. The authors motivate propagation and node wise sharpening through a Dirichlet–Gini decomposition of the Potts interaction. Their method, PtS, first propagates probabilities with restart and then restores its row mass. Setting the additional parameter η to zero recovers PPR-Prob exactly. The authors prove that sharpening preserves the current class ranking and decreases a single-node KL–Gini energy.
Critical strengths
- The method is simple, has clear input requirements, and provides a useful theoretical explanation. PtS uses only G and Q and adds one parameter that can be disabled, making it applicable when model updates are unavailable. Restoring row mass recovers the original probability propagation at η = 0. The local majorization proof explains sharpening’s effect, and the two-community construction illustrates its interaction with propagation. The authors distinguish local energy descent from full-graph descent, keeping the theoretical claims within their demonstrated scope.
Critical weaknesses
-
The specific sharpening formula lacks direct controls.The paper does not compare temperature sharpening (normalized p^γ) or hard thresholding under identical propagation, row-mass handling, and tuning budgets. Given prior diffusion–thresholding and Potts methods, these controls are needed to establish whether the gains depend on the KL–Gini design or on sharpening generally.
-
Sharpening’s independent gains are modest and dataset-dependent. Gains over independently tuned PPR-Prob were 0.57 points on clean data and 1.09 under severe noise. Reaction OFF’s 5.31-point loss uses PtS-selected propagation parameters and does not establish the independent gain. Paired uncertainty estimates are needed to distinguish sharpening’s contribution from probability-space propagation.
-
Global sharpening can reinforce incorrect predictions. Table 18 reports negative gains in two low-homophily bins, and PtS underperformed APPNP on severely noisy products. Sharpening preserves rankings, while Proposition 2 starts from correct predictions. The prevalence of affected nodes and whether validation can flag these failures remain unclear, limiting recommendations for general use.
Questions for authors
-
Under identical probability propagation, row-mass handling, and independent tuning budgets, how do temperature sharpening and hard thresholding perform? Does the KL–Gini reaction retain a consistent advantage at both large depths and the depths selected in practice?
-
If output temperature is fitted only on clean validation data and then tested at σ = 2, how do PtS and APPNP compare in NLL and ECE? Could you report per-graph results, including the three graphs omitted from Table 20, to assess whether average improvements mask calibration losses?
-
What fraction of each graph’s test nodes falls into Table 18’s low-homophily bins, and are negative gains concentrated in particular graphs or types of incorrect priors? For severely noisy products, is the degradation mainly due to propagation space, sharpening, or selection bias? Can validation results flag this failure?
2 comments
Response Bxxx- Part 2/2
Global sharpening can reinforce incorrect predictions. Table 18 reports negative gains in two low-homophily bins, and PtS underperformed APPNP on severely noisy products. Sharpening preserves rankings, while Proposition 2 starts from correct predictions. The prevalence of affected nodes and whether validation can flag these failures remain unclear, limiting recommendations for general use.
Within one step, sharpening preserves each node’s class ranking, so it strengthens a wrong leading class just as it strengthens a right one. It does not freeze predictions, however: they change in the following propagation steps, and the net effect is positive. Against PPR-Prob at , sharpening corrects 2.8% of all test nodes and introduces errors on 1.7% (Appendix Q.1, Table 25). All nodes of a graph share one , which we state as a limitation, but sharpening is not forced on a graph. The search includes , and validation switches sharpening off in 67% of the ogbn-products units and 56% of the ogbn-arxiv units at (Table 7). Proposition 2 now starts from five wrong predictions, which PtS corrects (Appendix D.2).
What fraction of each graph’s test nodes falls into Table 18’s low-homophily bins, and are negative gains concentrated in particular graphs or types of incorrect priors?
The two bins hold 14.2% of the test nodes on average, from 8.3% on Ele-Computers to 27.1% on ogbn-arxiv. Inside them PtS loses 1.12 and 0.99 percentage points against PPR-Prob, and 2.13 and 0.92 points against APPNP, which takes 0.14 and 0.17 points off the overall increase. Against PPR-Prob the losses concentrate on WikiCS, Ele-Computers, Cora-TAPE and TAPE-Arxiv23, with up to 6.17 points in the lowest WikiCS bin (Appendix Q.3, Table 27). By type of frozen prediction, over all test nodes, sharpening gains on every group of wrong predictions, 4.03 points on confidently wrong ones, and loses 0.82 points on confidently correct ones (Appendix Q.2, Table 26). Within the two low bins, the loss comes from initially correct predictions, which contribute points to the overall gain over PPR-Prob, while initially wrong ones contribute points.
For severely noisy products, is the degradation mainly due to propagation space, sharpening, or selection bias?
With selection at the test severity, as in Table 1, the loss comes from the propagation space: PPR-Prob is already 1.30 percentage points below APPNP, and sharpening recovers 0.09 points (Table 11). With settings selected on clean data, it comes mainly from sharpening, which accounts for 1.17 of the 1.52 points lost against APPNP. The PPR-Prob deficit remains when both methods are selected on test labels, as a diagnostic only: 0.94 points (Appendix J.3).
Can validation results flag this failure?
Partly. With validation data at the test severity, PtS is also below APPNP on validation in 41 of the 55 cases in which it is below APPNP on test, among 648 evaluations at , including all nine ogbn-products losses (Appendix J.4, Table 13). The other 14 losses are not flagged, and these counts do not cover clean validation under a later shift.
If output temperature is fitted only on clean validation data and then tested at = 2, how do PtS and APPNP compare in NLL and ECE? Could you report per-graph results, including the three graphs omitted from Table 20, to assess whether average improvements mask calibration losses?
With settings selected and the output temperature fitted on clean validation data, both applied unchanged at , PtS has lower mean NLL than APPNP (1.744 against 1.924) and a similar mean ECE (0.165 against 0.163). The averages do mask losses: ECE is 0.178 for PtS against 0.033 for APPNP on ogbn-arxiv, and 0.219 against 0.189 on TAPE-Arxiv23. Appendix S.1, Table 30 gives all nine graphs, including the three omitted from the original Table 20, which is now Table 29.
We hope that these answers and the added analyses resolve your questions.
Response Bxxx- Part 1/2
Thank you for the careful review and the precise questions, and for noting that our theoretical claims stay within their demonstrated scope. New material is marked in red in the revised PDF.
The gains come from soft sharpening generally, not from the KL–Gini design specifically. With the same probability propagation, row-mass handling, off option and 250-trial validation budget, PtS does not significantly outperform temperature sharpening: the mean differences are +0.04 percentage points (95% CI: [−0.003,0.09]) on clean inputs and +0.02 points (95% CI: [−0.02,0.06]) at σ=2. Both paired bootstrap intervals include zero. Hard thresholding is 0.53 points (95% CI: [0.45,0.60]) below PtS on clean inputs and 0.92 points (95% CI: [0.79,1.13]) below it at σ=2 (Section 4.2, Table 2). The two soft rules are the Gini and the Shannon-entropy versions of one KL-regularized sharpening step, and hard thresholding is their common limit (new remark in Appendix B, and our answer to Reviewer s4Ta).
At the selected depths, PtS does not show a statistically significant mean-accuracy advantage over temperature sharpening: the paired intervals for the differences of 0.04 and 0.02 percentage points both include zero. At K=100 without restart, PtS, temperature sharpening and hard thresholding reach 73.15%, 73.14% and 73.76% on clean inputs, and 57.76%, 57.74% and 57.30% at σ=2 (Appendix O.1, Table 22). Against hard thresholding the advantage therefore holds at the selected depths, by 0.53 and 0.92 points, but not consistently at K=100. Against propagation without sharpening the advantage holds at both. At the selected depths it is 0.57 and 1.09 points over PPR-Prob and 1.71 and 3.90 points over APPNP (Tables 1 and 2). At K=100 without restart, PPR-Prob reaches 44.6% and 37.2%, and APPNP 40.5% and 34.7% (Figure 2, Table 21).
We agree on Reaction OFF and on paired uncertainty. Reaction OFF keeps the (α,K) selected for PtS and only sets η=0, so the loss of 5.31 percentage points shows how much those settings depend on sharpening, not what sharpening adds. The submission used it in this sense, see (Section 5, Table 11) of the revised paper. To avoid any ambiguity, Table 2 now contains only independently tuned differences. These are 0.57 points [0.48,0.66] on clean inputs and 1.09 points [0.99,1.19] at σ=2 for sharpening, and 1.14 points [1.04,1.25] and 2.81 points [2.55,3.11] for probability propagation over APPNP, with paired 95% bootstrap intervals. With all settings selected on clean validation data, sharpening still adds 0.61 points at σ=2 (Section 4.3). The gain is modest in the mean but not on every graph: at σ=2 it is 3.17 points on WikiCS and 2.72 points on Ele-Computers, both with p<0.001. It is dataset-dependent: positive on all nine graphs, with p<0.05 on five at σ=2 and on three on clean inputs, in paired t-tests corrected for overlapping splits but not for multiple comparisons (Appendix J.2, Table 12).