- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies label-efficient classification of designated subgraph sets in a shared background graph. The authors note that two structurally similar queries may have different relations to the same labeled source. Existing subgraph representations often do not preserve this instance-relation correspondence. The authors propose RAISE. RAISE constructs a fixed, label-free relational kernel on common background vertex coordinates from membership relations and diffusion relations. Kernel ridge regression learns class-specific source coefficients. Class-wise diffusion aggregation supplies complementary evidence. One round of self-training adds high-confidence pseudo-labeled examples from the unlabeled training pool. It then performs a weighted refit on the same fixed kernel that accounts for dependence. Experiments use four benchmarks: PPI-BP, HPO-METAB, HPO-NEURO, and EM-USER. Label budgets range from two percent to twenty percent. RAISE improves average micro-F1 and macro-F1 over PADEL by 4.71 and 6.74 percentage points, respectively. It also achieves the highest cross-subset Cohen's kappa. The authors also use labeled-source exchange, relation reassignment, synthetic relation-strength experiments, and missing-member experiments. These support the claim that relation correspondence is predictive.
Critical strengths
- The fixed, label-free relational kernel is separated from sparse-label inference, and the computational efficiency is notable. RAISE constructs membership and diffusion relations once into a fixed kernel, and labels enter prediction only through kernel ridge regression coefficients and class-wise aggregation. Across the 120 main-table conditions, the total time is 0.22 hours, far below PADEL’s 139.6 hours. This idea of reusing relations plus lightweight fitting has practical value for low-label subgraph classification and also makes the label-use path easier to audit.
- The correspondence interventions and counterexample experiments are relatively convincing. The labeled-source exchange fixes the query, the number of labels, and the class composition, and replaces only one labeled source; it shows that the relational kernel is more sensitive to sources with stronger diffusion relations to the query. Relation reassignment preserves the full kernel spectrum, rank, and scale while only shuffling instance–relation pairing, and RAISE degrades monotonically as the permutation fraction increases. Synthetic experiments further show that RAISE is advantageous when relations carry class information, whereas the structural kernel is advantageous when internal shape carries class information. These results support the core mechanism of the paper and honestly delimit its applicability.
Critical weaknesses
- The incremental contribution of the complete method is small. The core gains come mainly from the relational kernel and self-training. The ablations show that replacing the structural kernel with the fixed relational kernel brings about 5.68 points of micro-F1, and self-training adds about 1.85 points. Class-wise diffusion aggregation adds only about 0.35 points, and dependence-aware refitting adds only about 0.20 points, mainly from HPO-NEURO. The advantage of complete RAISE over the fixed relational kernel plus self-training is very small. The title and abstract emphasize the complete RAISE pipeline, but the ablations indicate that what really matters is the fixed relational kernel and one round of self-training. The complexity of dependence-aware refitting is disproportionate to its gain. The authors should position the contribution more clearly as a relational kernel plus lightweight sparse-label inference, rather than presenting the complete pipeline as the main innovation.
- The paper performs column standardization and prediction rules on the prediction set, which includes validation and test queries. Although labels are not used, the moments of test-query scores are used. All methods are claimed to share this rule, but readers may still worry that this transductive standardization favors RAISE’s kernel scores. In addition, on PPI-BP at two percent labels, RAISE’s micro-F1 is 35.47, lower than PADEL’s 37.61 and Member + 1-hop Ridge’s 38.11. The main text mainly reports the average over two to twenty percent and does not sufficiently explain the failure mode at extremely low labels. If the method claims applicability from two to twenty percent, this cell should be reported and analyzed.
Questions for authors
- Can the authors report a version that does not use test-query moments—for example, estimating standardization moments only on the labeled set or the unlabeled training pool, and then applying them to test data?
- On PPI-BP at two percent labels, RAISE is lower than PADEL and Member + 1-hop Ridge. Can the authors analyze the reason: is it overfitting of kernel ridge regression with very few sources, pseudo-label selection bias, or instability of diffusion aggregation under extremely low labels?
- Labeled-source exchange is performed only on PPI-BP. Can the authors reproduce the same pattern on HPO-METAB or EM-USER—that same-class sources with stronger relations raise the margin, while other-class sources with stronger relations lower it?
4 comments
Response to Reviewer Xu3g (2/2)
Q3. Source exchange beyond PPI-BP. Yes; the margin pattern holds on HPO-METAB and EM-USER at all four budgets. Same-class sources with stronger relations raise the true-class margin and other-class sources lower it, with larger shifts under K0 than under WL. Because few pairs meet the strict WL tolerance on these tasks, we widen it in tiers. The table below reports the tier with the most swaps, which also has the smallest contrast; Appendix J.4 lists all tiers. Entries are mean margin changes over all four budgets:
| Task (tier; swaps) | Same-class source: K0 / WL | Other-class source: K0 / WL | Contrast, K0−WL [95% CI, query bootstrap] |
|---|---|---|---|
| PPI-BP (strict; 2,344) | +0.979 / +0.001 | −0.541 / +0.002 | +1.52 [1.37, 1.68] |
| HPO-METAB (4× WL tolerance; 4,700) | +0.632 / +0.253 | −0.234 / −0.077 | +0.54 [0.41, 0.68] |
| EM-USER (4× WL tolerance, size ratio 2; 978) | +1.595 / +0.736 | −1.400 / −0.776 | +1.48 [1.08, 1.89] |
The contrast is the K0−WL difference on same-class swaps minus the same difference on other-class swaps.
W1. Contribution of the late components. The complete method targets two results, both reported in the abstract and Table 1: accuracy with few labels and agreement of predictions across labeled subsets. As you note, accuracy comes mostly from the relational kernel and one self-training round, which add 5.68 and 1.85 micro-F1 on the 50 fresh subsets (Table 2b). Class-wise aggregation acts mainly on agreement. In the setting of Table 1 (four tasks, 2/5/10/20% labels, the original five labeled subsets, RAISE's hyperparameters unchanged; Appendix J.7):
| Setting | Micro-F1 | Cross-subset κ |
|---|---|---|
| K0-KRR + self-training (no aggregation, independent refit) | 54.92 | 0.414 |
| + class-wise aggregation | 55.14 | 0.469 |
| + dependence-aware refit (RAISE) | 55.40 | 0.469 |
| SSNP (highest κ among baselines, Table 1) | 40.30 | 0.380 |
Aggregation raises κ from 0.414 to 0.469, more than half of RAISE's 0.089 lead over the most stable baseline. On the 50 fresh subsets at 5% and 10% labels, it likewise raises κ from 0.397 to 0.462, more than the 0.038 from self-training (Table 2b), and it lowers cross-subset prediction disagreement by 5.47 points for the quarter of queries least related to the labeled sources and by 3.00 for the most related quarter (Appendix J.7). Much of the agreement result therefore comes from aggregation, a gain that the small micro-F1 increment does not show. Aggregation itself is a class-wise average with one fixed weight.
The dependence-aware refit addresses a property of self-training on overlapping subgraphs: on three tasks, the pseudo-label errors of sources that share members are positively correlated beyond a permutation baseline (Table E.7). It costs little. It adds no hyperparameter, solves the same form of linear system as the first fit, and changes only the residual weighting. The 0.22 hours for all 120 main-table conditions that you note include aggregation and the refit, and a complete fit from cached relations averages 0.14 s. On the 50 fresh subsets, the refit adds 0.20 [0.13, 0.27] micro-F1 over independent weighting. The gain grows when errors are shared more strongly: under controlled error injection it rises from +0.28 to +0.80 against a diagonal treatment as the shared-error strength goes from 0 to 0.9, and it reaches +2.43 to +3.33 for pseudo-source sets with high member overlap on three tasks (Appendix J.2).
The revised Introduction now presents the evidence that instance-specific relations add predictive information beyond structural similarity before the method, and states the role of each component: the kernel records the relations once, KRR fits class-specific source contributions, aggregation stabilizes the inference, and one self-training round with the refit extends supervision. Section 4.3 now describes source deletion in one sentence, with the details in Appendix A.3.
We hope these results answer your questions and concerns, and we would be grateful if you would reconsider your score.
Dear Reviewer Xu3g,
Thank you again for your review. As the discussion period ends on November 18, we wanted to check whether our responses and the revised PDF (changes shown in blue) have addressed your questions. In brief, moments from the unlabeled training pool alone give 55.30 micro-F1, against 55.40 in the paper version (Appendix J.6); RAISE exceeds PADEL in 13 of the 16 task–budget cells on micro-F1 and in 15 on macro-F1 (Section 5.2); and class-wise aggregation raises cross-subset κ from 0.414 to 0.469, against 0.380 for SSNP (Appendix J.7).
If any point remains open, in particular on the PPI-BP 2% cell or the role of the late components, we would be glad to clarify it before the deadline.
Best regards,
The Authors
Dear Reviewer Xu3g,
Thank you again for your review. With the discussion period closing at the end of November 18 (AoE), we wanted to follow up once more in case our earlier message was missed. We would be glad to hear whether our responses address your questions, in particular the check without test-query moments (Appendix J.6) and the PPI-BP 2% analysis (Appendix J.7), and to clarify anything that remains.
Best regards,
The Authors
Response to Reviewer Xu3g (1/2)
Thank you for the detailed reading.
Q1 / W2. A version without test-query moments. Yes; removing test-query moments leaves performance virtually unchanged. With every other setting frozen:
So RAISE does not need test-query scores for standardization: moments from the unlabeled training pool give essentially the same results. Against the paper version, the paired difference is −0.10, and per-subset differences range from −0.49 to +0.16. In the third row, no moment or decision threshold uses test-query scores. The fixed kernel remains transductive as defined in Section 3: it is built once from the graph and all designated member sets, without labels.
On whether standardization favors RAISE: applied to every method's saved test scores, the same standardization still leaves RAISE first. In the paper, column standardization applies to RAISE and the matched KRR controls of Table 2, while external baselines keep their native scores. Under the shared standardization, the best external baseline reaches 49.79 micro-F1 (PADEL) and 47.35 macro-F1 (Member + 1-hop Ridge), against 55.13 and 52.76 for RAISE.
Q2 / W2. PPI-BP at 2% labels (25 labeled sources). In this cell, the micro-F1 gap to PADEL already appears at the first KRR fit (31.45 vs 37.61), before pseudo-labels or aggregation enter. Each subsequent stage narrows the gap and raises both scores:
So neither pseudo-label selection nor aggregation causes this gap.
The initial gap stems from the decision rule, not from a weaker KRR fit. RAISE standardizes each class column before the argmax, whereas the baselines take the argmax of their native scores. Keeping every fit fixed, we swap only this step (using saved test scores for the baselines). Entries are micro / macro-F1:
Under a matched decision rule, the first fit is level with or ahead of both baselines in micro-F1. Under a native argmax, it assigns 84.3% of test queries to the largest class, much as Ridge does. Standardization changes how predictions are distributed across classes:
In this skewed cell, standardization trades micro-F1 for macro-F1 by spreading predictions across smaller classes. PADEL and Ridge concentrate their predictions on the largest classes, and PADEL recalls only 4.4% and 5.5% of two smaller classes. RAISE covers every class but predicts the largest class less often than its true share.
The micro-F1 difference in this cell lies within test-query sampling variation: the paired bootstrap interval for RAISE minus PADEL is −2.14 [−6.79, 2.52]. On three new random partitions it averages −0.2 points, with a macro-F1 advantage of +7.5. At the same 2% budget, RAISE leads PADEL on the other three tasks (HPO-METAB 34.13 vs 27.49, HPO-NEURO 51.48 vs 46.78, EM-USER 59.59 vs 48.98; Tables C.2–C.4). Over all 16 task–budget cells, RAISE exceeds PADEL in 13 on micro-F1 and 15 on macro-F1. Section 5.2 now reports this cell, and Appendix J.7 gives the analysis.