- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents HADES, a node-level structural OOD detection method for hypergraphs. HADES introduces a prototype-based energy score that compares confidence propagation between the current hypergraph and a reference built from clean training representations (Lines 176 - 178). Moreover, it generates synthetic structural shifts on validation data to evaluate eight candidate detectors, then selecting the best-performing one for detection. The paper presents a practical approach to structural OOD detection with diverse experiments. However, I have some concerns with the method and experiments, particularly on how the improvements appear to rely heavily on selecting existing detectors using synthetic validation shifts, and whether this method generalizes to unknown shifts.
Critical strengths
-
The overall storytelling is clear and generally easy to follow.
-
I really enjoyed the presented figures, particularly HADES overview.
-
The paper evaluates HADES across multiple datasets and compares with several baselines, showing improvements in detection metrics like AUROC and FPR95.
-
HADES also investigates an important problem of structural OOD detection in hypergraphs, a relatively underexplored setting compared with traditional graph OOD detection.
Critical weaknesses
-
The main experiments use the same perturbation approach and settings for generating validation and test OOD samples (section 4.1), which could potentially make detector selection easier. Although the authors explored different perturbation severities and natural structural shifts in the appendix, their results reveal potential failures in some settings. For example, HADES achieves only 37.7 AUROC on the cocitation --> coauthorship transfer, compared with 70.4 for ViM (Table 7 in appendix). This raises concerns about the reliability of detector selection when the actual structural shifts differ from those used during validation.
-
Also, a single fixed train/validation/test split was used for each dataset (Line 287), while results were reported over three random seeds (Line 335). Since the training and validation data directly affect prototype construction and detector selection, it is unclear how sensitive the results are to different data splits.
-
Although the paper analyses the theoretical complexity, it does not provide empirical runtime or memory comparisons. Given that HADES requires evaluating numerous detectors before selecting one, clearly reporting the additional computational cost maybe beneficial to demonstrate the efficacy.
-
While the prototypes are learned from clean training data, the prototype assignments are computed using embeddings from the potentially shifted hypergraph. As a result, both the current and reference views may be affected by structural changes. It is unclear whether their difference can reliably identify structural OOD samples.
-
Based on Alg 1, HADES selects detectors solely based on validation AUROC. It would be interesting to explore whether using alternative metrics, such as FPR95, which is more sensitive and constrained, could lead to better OOD detection performance.
-
General note: It is recommended to review and discuss more recent graph OOD detection papers in the extended related work section, as the current discussion focuses primarily on works published before 2024. E.g.,
[1] Fan, X, et al., HGOOD: Hypergraph-enhanced Graph Contrastive Learning for Graph Out-of-Distribution Detection. IJCAI26
[2] Wang D, et al., What Information Matters? Graph Out-of-Distribution Detection via Tri-Component Information Decomposition. ICML26.
[3] Wang, Y, et al., CLINIC: Towards High-quality Graph Out-Of-Distribution Detection. ICML26
Questions for authors
-
Why does HADES select only one detector instead of combining multiple detectors? Could the authors compare their approach with simple averaging, weighted averaging, or other detector combination methods? Do the authors anticipate any benefits with this approach?
-
How sensitive is HADES to different train/validation/test splits?
-
The ablation results suggest that most improvements come from detector selection rather than the proposed discrepancy score. Could the authors further clarify the contribution of the new score and compare against selecting from a larger pool of existing detectors?
-
How does HADES perform under shifts like feature or label shifts, that are also commonly explored in graph OOD studies?
5 comments
I appreciate the authors' response and additional experiments to address my concerns. However, several important issues remain:
From my understanding, the Q4 response further shows that HADES relies heavily on how closely the surrogate validation shifts has to resemble the actual OOD shifts in order to improve performance. For example, changing from structural to feature surrogates improves AUROC from 63.6 to 72.1. However, in practice, the distribution of the test OOD shifts are unknown, making it difficult to ensure such alignment. So this raises fundamental concern for me about the robustness and practical applicability of HADES under unseen distribution shifts.
Additionally, the learned detector combination substantially outperforms HADES's single-detector selection (77.5 versus 67.9 AUROC) using the same candidates and surrogate data. Although the authors stated that single selection is computationally cheaper, no empirical runtime or memory comparisons are provided to justify this performance trade-off.
Also, based on the Q3 results, i am hesitant about how expanding the pool of existing detectors improves AUROC from 65.0 to 70.1, while adding the proposed discrepancy score provides only a further 1.5 point improvement. This suggests that most of the performance gain comes from selecting existing detectors rather than the proposed score.
Therefore, given my concerns still remain and several of the weaknesses are not addressed (e.g., W2, W3, W5). I would like to maintain my current rating.
We provide additional evidence on unseen shifts, runtime, prototype stability, and FPR95 selection, while clarifying data splits and recent related work.
Q1. Shifts unseen during validation
We appreciate the concrete Cora transfer example. We tested a stricter setting in which the target shift type is excluded from the surrogate library. For each of eight target types, validation uses only the other seven types. The natural transfer between cocitation and coauthorship has no corresponding surrogate.
Table R4. AUROC (%). Synthetic: 48 settings per type. Natural: 24 settings.
| Rule | Mean across eight types | Weakest type | Natural transfer |
|---|---|---|---|
| Best fixed detector for each column | 59.1 | 55.3 | 51.6 |
| Submitted single selection | 62.1 | 55.0 | 58.6 |
| Single selection, target type excluded | 62.9 | 55.4 | 63.9 |
| Learned combination, target type excluded | 67.0 | 56.0 | 77.2 |
The submitted selector exceeds the best fixed detector on average, and withholding the target type does not erase that advantage. Combining candidates trained on the other types raises the mean by another 4.1 points. For the cited HGNN cocitation to coauthorship case, AUROC rises from 37.7 to 52.9 with combination (ViM: 70.4). Across all four backbones in that direction, combination averages 71.3 versus 51.8 for the best fixed detector. This fusion uses the same surrogate validation framework and candidate pool as single selection.
Q2. Data splits
All three master seeds use the same data partition. Across them, HADES has 2.3 AUROC points of mean standard deviation, versus 2.4 to 4.6 for the other detectors. Repartitioning nodes outside the test set once gives 75.7 AUROC on mixed nodes, compared with 76.4 under the submitted reference and validation partition. These checks support stability to training randomness and this validation partition. We will evaluate new data splits with retrained backbones in the revision.
Q3. Runtime and memory
We profiled self validation and target scoring over 48 settings. Self validation takes 1.13 seconds and uses 644 MB of additional peak GPU memory on average per trained backbone. The median fixed detector takes 0.013 seconds to score a target hypergraph.
Table R5. Deployment costs, seed 42. GPU memory is above the resident state.
| Rule | Mean time | Mean peak GPU memory |
|---|---|---|
| Submitted single selection | 0.019 s | 198 MB |
| Learned combination | 0.13 s | 323 MB |
The combination evaluates eight candidates sequentially. Its time sums their separately measured scoring times, and its memory uses the largest measured candidate peak. Logistic fitting takes a few hundredths of a second and reuses the surrogate scores computed during self validation.
Q4. Prototype view under shift
Fixed centers constrain the prototype view even though shifted embeddings change node assignments. Across settings, its drift is 0.52 to 0.76 of the current incidence view drift. Removing the discrepancy score lowers the learned combination by 3.3 AUROC points with matched surrogates and 1.5 points when the target type is excluded. Thus the view difference contributes structural evidence even when assignments move.
Q5. Selection metric
We compared selecting the single detector by surrogate AUROC and by surrogate FPR95. The latter improves target FPR95 in both validation settings.
Table R6. Target FPR95 (%), lower is better. Mean over eight shift types.
| Selection criterion | Matched type | Target type excluded |
|---|---|---|
| Surrogate AUROC | 79.8 | 82.5 |
| Surrogate FPR95 | 77.0 | 81.5 |
The submitted AUROC criterion remains best for target AUROC. The FPR95 criterion lowers target FPR95 by 2.8 points with matched surrogates and 1.0 point when the shift type is excluded. We will make the criterion configurable for the intended operating metric.
Q6. Recent graph OOD work
We agree that the extended discussion should better cover recent work. Our main text already discusses UniGOD, BaCa, PGOS, and HyperGOOD from 2026. We will also position HGOOD, TIDE, and CLINIC. HGOOD and CLINIC detect entire graphs. TIDE detects OOD nodes but trains a new graph model. Their published results do not directly compare with node detection using a frozen hypergraph backbone.
We address detector combination, split sensitivity, the contribution of the discrepancy score, feature shifts, and the three suggested methods.
Q1. Single detector or combination
We thank the reviewer for suggesting combinations. To compare them on equal terms, we used the eight HADES candidates, clean validation nodes, and surrogate copies. We standardized candidate scores on clean validation nodes. Averaging weights came from surrogate AUROC, while logistic regression learned from proxy labels on the surrogate task.
Table R1. Mean AUROC (%) across dataset and backbone settings.
Equal averaging dilutes strong candidates when other scores respond in the opposite direction. Top two averaging and softmax weighting nearly match single selection. In contrast, the learned combination improves AUROC by 9.6 points in the paired protocol and 2.7 points on mixed nodes. The same surrogate validation task supports stronger fusion without target OOD labels. We retain single selection as the submitted method because it scores only one detector at deployment and has the regret guarantee in Proposition 2. We report learned fusion as a separate variant.
Q2. Sensitivity to data splits
We thank the reviewer for raising split sensitivity. The submission uses one fixed train, validation, and test split with three master seeds. Across those seeds, HADES has a mean AUROC standard deviation of 2.3 points, compared with 2.4 to 4.6 for other detectors, and its mean rank stays between 2.5 and 2.7. We also tried a new random partition of nodes outside the test set into reference and validation sets. On mixed nodes at q = 0.3, this variant reaches 75.7 AUROC versus 76.4 with the submitted partition. These checks address training randomness and one reference partition. They do not measure sensitivity to independently redrawn train and test splits, which would require retraining the backbones.
Q3. Discrepancy score and a larger pool
We thank the reviewer for separating the score from selection. We removed the discrepancy score from both the submitted pool and an expanded pool of existing post hoc detectors, while keeping the selection procedure fixed.
Table R2. Selected AUROC (%) in the paired protocol, mean over 48 settings.
A larger pool alone raises AUROC from 65.0 to 70.1. The discrepancy score then adds a further 1.5 points, with improvement in nine settings and a decline in one. In the submitted pool it adds 2.8 points, with improvement in 15 settings and a decline in two. Selection is a major source of the gain, while the score supplies complementary evidence from the difference between current incidence and prototype anchored views.
We also examined the three suggested papers. HGOOD[1] and CLINIC [3] detect OOD graphs among graph samples. Their reported graph level results therefore do not directly match node level detection in one hypergraph. TIDE [2] is closer because it detects OOD nodes under feature and structure shifts. It trains a new graph model with an information bottleneck, whereas HADES evaluates scores on a frozen hypergraph backbone. These distinctions explain why we discuss them rather than insert them into the post hoc score pool.
Q4. Feature and label shifts
We thank the reviewer for asking about other shift types. We evaluated the feature interpolation of Appendix C.2 at strengths 0.1 and 0.5, applied either to all nodes or to 30% of nodes in one hypergraph. The submitted selector used structural surrogates. The second HADES column used feature surrogates but the same detector pool and validation rule.
Table R3. Feature shift AUROC (%), mean over 48 settings.
At strength 0.1, all methods are near chance. At strength 0.5, feature surrogates raise HADES from 63.6 to 72.1 on all nodes and from 55.0 to 58.2 on mixed nodes. This shows that the validation rule can transfer to a feature shift when its surrogate reflects that shift. Unseen class label shift remains untested because a class holdout changes the backbone training set and requires retraining.