- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates open-set cross-network node classification, where a model transfers knowledge from a labeled source graph to an unlabeled target graph containing both shared and unknown classes. The authors propose OGSA, an OOD-guided selective alignment framework that uses distributional evidence to identify unknown target nodes. The method integrates prediction-based screening, prototype-initialized clustering, and structure-aware OOD scoring to refine target pseudo-labels, which then guide target supervision and signed domain-adversarial alignment. The paper further derives an open-set target risk bound to characterize the trade-off between identifying unknown nodes and incorrectly rejecting known ones.
Experiments across 15 transfer tasks demonstrate consistent improvements over a broad set of graph OOD detection and domain adaptation baselines. The results are encouraging and support the effectiveness of selective alignment in open-set graph transfer. However, the robustness of the proposed approach under varying open-set conditions and the contribution of its individual components require further examination.
Critical strengths
S1. The paper addresses negative transfer caused by aligning unknown target nodes with known source classes. Incorporating distributional evidence beyond prediction confidence is a reasonable approach to improving unknown-node identification.
S2. OGSA achieves the best reported ACC and H-score across all 15 transfer tasks, outperforming competitive methods such as UAGA and GraphRTA. The consistent gains, particularly in unknown-class recognition, support the effectiveness of the proposed framework.
S3. The paper examines the effects of OOD refinement, signed domain alignment, alternative OOD scores, and auxiliary objectives. The analysis of pseudo-label quality during training also helps explain how the iterative refinement process affects performance.
Critical weaknesses
W1. The paper argues that unknown nodes can be confidently assigned to known classes despite substantial distributional differences. However, OOD refinement is restricted to candidates passing the initial prediction-based screening ((T=T_1\cap T_2), (\Omega=T\cap U)). Thus, confidently misclassified unknown nodes may be excluded regardless of their OOD scores. This appears inconsistent with the central motivation. The authors should quantify how often this occurs and examine whether OOD evidence can recover nodes outside the initial candidate set.
W2. OGSA builds on clustering and signed adversarial alignment closely related to UAGA, while incorporating OOD scoring from GEBM. Moreover, replacing the GEBM score with a simpler Mahalanobis distance yields a nearly identical average H-score. This raises questions about the necessity of the full OOD refinement design. A controlled comparison with UAGA augmented by the same OOD scores, using matched selection budgets and training settings, would better isolate OGSA's contribution.
Questions for authors
-
Since (\Omega=T\cap U), unknown nodes excluded by the initial prediction-based screening cannot be recovered by OOD refinement. What proportion of confidently misclassified unknown nodes are excluded at this stage, and how does this affect unknown-class detection?
-
How does OGSA compare with UAGA augmented by the same OOD scores under matched training settings and candidate budgets? This would help isolate the contribution of the proposed selection mechanism.
-
How are the Top-k budget and other hyperparameters selected without labeled target data? How sensitive is performance to the proportion of unknown target nodes?
-
How are the Gaussian statistics for the unknown class estimated when the source graph contains only known classes? Are unresolved target nodes included in ordinary domain alignment, and if so, how is potential negative transfer avoided?
3 comments
Official Comment by Authors (2/3)
2. Relation to UAGA and necessity of the OOD refinement design. (W2/Q2)
Response: This is a valuable point. We agree that the clustering and signed adversarial alignment in OGSA are closely related to UAGA and that the OOD score is adopted from GEBM. Our core contribution lies in the OOD-guided refinement. It uses distribution discrepancy as complementary evidence for identifying unknown nodes and combines it with prediction-based evidence through the intersection . To isolate this contribution, we follow your suggestion and augment UAGA with the same GEBM score under the same intersection rule. UAGA+GEBM is tuned per direction over the same hyperparameter ranges, including the candidate budget . The results (ACC / HS, %) on the five-class citation benchmarks are reported below.
| Method | C1→A1 | D1→A1 | A1→C1 | C1→D1 | D1→C1 | A1→D1 | Avg. |
|---|---|---|---|---|---|---|---|
| UAGA | / | / | 62.23 / 59.14 | 52.13 / | 60.41 / 52.26 | 62.23 / 58.29 | 59.80 / 58.98 |
| UAGA+GEBM | 58.74 / 56.95 | 59.76 / 59.58 | / | / 61.97 | / | / | / |
| OGSA | 71.98 / 75.37 | 68.76 / 70.53 | 81.43 / 80.53 | 74.54 / 76.87 | 75.94 / 77.04 | 73.54 / 74.36 | 74.37 / 75.78 |
Adding the GEBM score improves UAGA on average, which confirms that distributional deviation is useful evidence beyond our framework. However, UAGA+GEBM still falls clearly behind OGSA under the same score and selection rule. This shows that the gain of OGSA does not come from simply attaching an OOD score to an existing method. It comes from how the OOD evidence is integrated into adaptation, including how pseudo-labels are constructed and how unresolved nodes are excluded from alignment.
Regarding the Mahalanobis distance, we view its nearly identical performance as evidence for our motivation rather than against it. Even the simplest distance to source known classes is sufficient to bring a large improvement. This indicates that existing methods largely overlook distributional information, and that the benefit of OGSA does not depend on a particularly strong OOD detector. This is also why we directly adopt an existing OOD score instead of designing a tailored one. Developing OOD scores specific to open-set graph domain adaptation is a promising direction for future work.
To further justify the necessity of the full OOD refinement design, in particular the role of the intersection, we compare it with several alternative strategies for fusing the prediction-based score and the OOD score:
(1) Weighted sum: since the two scores lie on different scales, we consider three normalization schemes (min-max, z-score, and rank normalization) and combine the normalized scores by a weighted sum, with the weight searched over 0 to 1; (2) Product: the two scores are also normalized and then multiplied; (3) Rank average: the two scores are converted to ranks and averaged; (4) Union: the intersection in OGSA is replaced by the union, i.e., .
The results (ACC / HS, %) are summarized below.
| Model | Five-class citation | Nine-class citation | ogbn-arxiv |
|---|---|---|---|
| OGSA | 74.37 / 75.78 | 65.47 / 65.91 | 53.56 / 55.20 |
| w/o OOD refinement | 68.07 / 69.29 | 55.77 / 48.00 | 46.78 / 40.74 |
| OOD refinement only | 68.91 / 71.58 | 59.00 / 58.13 | 44.02 / 44.24 |
| Weighted sum | / | / | 52.07 / 53.90 |
| Product | 70.57 / 72.41 | 64.46 / 64.89 | / |
| Rank average | 70.74 / 72.72 | 62.30 / 62.31 | 48.55 / 54.04 |
| Union | 70.64 / 70.63 | 61.00 / 61.00 | 52.37 / 54.24 |
As shown in the table, all fusion strategies outperform both w/o OOD refinement and OOD refinement only, confirming that distributional deviation provides complementary evidence for unknown identification. Among them, the intersection achieves the best ACC and HS on all benchmark groups without additional tuning, while the alternative strategies require extra hyperparameters and show less stable performance across benchmarks. This is consistent with our theoretical analysis: incorporating OOD evidence helps recover missed unknown nodes, but known nodes that deviate from the source only due to domain shift may be incorrectly rejected. Strategies such as Union, which accept a node as unknown if either score flags it, are more prone to such errors, whereas the intersection limits them by requiring agreement between prediction-based and distributional evidence. We will add these comparisons and discussions in the revised version.
Official Comment by Authors (3/3)
3. Hyperparameter selection and sensitivity to the unknown proportion. (Q3)
Response: Thank you for the question. Following the common practice in open-set graph domain adaptation, the results of all methods are reported under their best hyperparameters on the target domain, so the comparison is conducted under the same protocol. More importantly, even without access to target labels for selection, OGSA remains stable over wide ranges of most hyperparameters, as shown in Appendix D.6. This indicates that the performance of OGSA does not rely on fine-grained tuning with target labels, and a reasonable default configuration is sufficient in practice.
To examine the sensitivity to the proportion of unknown target nodes, we keep only 75%, 50%, or 25% of the unknown target nodes on the five-class citation benchmarks, while using the same configurations as in the original setting without re-tuning. We compare OGSA with w/o OOD refinement and OOD refinement only. The results (ACC / HS, %) are reported below.
| Unknown kept | OGSA | w/o OOD refinement | OOD refinement only |
|---|---|---|---|
| 100% | 74.37 / 75.78 | 68.07 / 69.29 | / |
| 75% | 69.82 / 71.69 | 57.08 / 47.13 | / |
| 50% | 64.78 / 66.42 | 52.36 / 35.78 | / |
| 25% | 59.78 / 58.96 | 47.18 / 32.84 | / |
As the proportion of unknown nodes decreases, the performance of all methods degrades, but OGSA consistently achieves the best ACC and HS. Removing OOD refinement leads to the largest degradation, since prediction-based signals alone fail to identify the few remaining unknown nodes. Using OOD refinement alone also degrades notably, since shifted known nodes dominate the high-OOD nodes when unknown nodes become rare and are thus incorrectly rejected. By requiring agreement between the two sources of evidence, OGSA avoids both issues and remains robust across different unknown proportions. We will add this result in the revised version.
4. Gaussian statistics for the unknown class and alignment of unresolved nodes. (Q4)
Response: Thank you for the careful reading. First, we do not estimate any Gaussian statistics for the unknown class. The GEBM score is fitted only on source nodes, where a class-conditional Gaussian is estimated for each known class. A target node that is far from all known-class Gaussians therefore receives a high energy and is regarded as potentially unknown. Second, unresolved target nodes are not included in the alignment of the encoder. They are neither aligned with nor pushed away from the source domain, which mitigates potential negative transfer caused by uncertain nodes. We will clarify both points in the revised version.
Thank you again for your constructive comments. We will incorporate these results into the revised version and are happy to address any further questions.
Official Comment by Authors (1/3)
Thank you for the constructive and detailed review. Our responses to each point are provided below.
1. Unknown nodes excluded by the prediction-based screening. (W1/Q1)
Response: Thank you for this insightful comment. We agree that confidently misclassified unknown nodes can be excluded by the prediction-based screening regardless of their OOD scores. This restriction is a deliberate trade-off rather than an oversight. OOD scores alone may also flag known nodes that deviate from the source distribution due to domain shift, and requiring agreement with prediction-based evidence keeps such incorrect rejections under control. Moreover, the candidate set T is recomputed at every epoch, so a node excluded in one epoch can still be identified later as the predictions evolve during adaptation. To quantify this effect, we take the top-k target nodes by OOD score over all target nodes, denoted by Uall, and examine the high-OOD nodes that are not pseudo-labeled as unknown by OGSA, i.e., the excluded set X=Uall∖B, where B denotes the nodes pseudo-labeled as unknown after OOD refinement. We report:
(1) the fraction of truly unknown nodes that fall in X,
(2) the fraction of truly unknown nodes within X,
(3) the fraction of truly known nodes that fall in X, i.e., high-OOD known nodes protected from incorrect rejection,
(4) the fraction of unknown nodes in X at the first adaptation epoch that are pseudo-labeled as unknown at the last epoch.
Results (%) on the five-class citation benchmarks are averaged over 5 seeds.
The results lead to three observations. First, only 10.1% of unknown nodes are excluded despite their high OOD scores on average. Second, truly unknown nodes account for only 34.6% of the excluded set on average, so admitting these nodes would incorrectly reject more known nodes than it recovers unknown ones. In exchange, 7.5% of known nodes with high OOD scores are protected from incorrect rejection. Third, 81.6% of the unknown nodes excluded at the first adaptation epoch are pseudo-labeled as unknown at the last epoch, showing that the exclusion is largely temporary and that OOD evidence does help recover nodes outside the initial candidate set as adaptation proceeds.
To further examine this trade-off under domain shifts, we apply controlled target-side perturbations on the five-class citation benchmarks. For attribute shift, a fraction of feature dimensions of each target known-class node are resampled from their empirical marginal distributions. For structure shift, a fraction of intra-class edges among target known-class nodes are rewired to nodes of other classes. We then compare the fraction of target known nodes wrongly flagged by the OOD score alone (U) with that after intersecting with the prediction-based candidates (Ω=T∩U). Results (%) are averaged over 5 seeds and 6 transfer directions.
As attribute or structure shifts become stronger, the fraction of known nodes wrongly flagged by the OOD score alone increases from 13.3% to 20.9%, whereas it remains below 5.6% after the prediction-based screening is applied. This shows that the advantage of the prediction-based screening becomes more evident under domain shifts, where more known nodes deviate from the source distribution. Therefore, the prediction-based screening has a limited impact on unknown-class detection while effectively preventing incorrect rejections of known nodes.
We will add this analysis to the revised version.