- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes CalibHyper, a chance-corrected relational hypergraph framework for few-shot molecular property prediction. The method introduces C4RC to model four-state property relations while accounting for chance agreement and observation reliability. It further uses MSRHA to select auxiliary properties and propagate signed relation messages for molecular prediction. Experiments on MoleculeNet benchmarks show promising results in both 1-shot and 10-shot settings.
Critical strengths
-
The paper identifies a meaningful limitation of binary label agreement, particularly under highly imbalanced property labels. The analysis of real assay data provides clear motivation for chance-corrected relation modeling.
-
The combination of C4RC and MSRHA is well motivated. The proposed framework connects relation calibration with molecule-conditioned auxiliary-property selection and signed message passing, rather than using relation learning only as an auxiliary training objective.
-
The experimental evaluation covers multiple MoleculeNet benchmarks under both 1-shot and 10-shot settings. The component ablations and additional analyses of auxiliary labels and routing strategies provide useful insights into the proposed framework.
Critical weaknesses
-
The necessity of four-state relation modeling is not fully established. Since chance correction itself addresses the limitations of binary agreement, a simpler chance-corrected scalar relation could also be effective. Comparing C4RC with such a relation under the same MSRHA framework would better clarify the contribution of four-state modeling.
-
The reliability shrinkage in Eq. (2) depends only on the number of jointly observed molecules. However, for highly imbalanced datasets such as MUV and PCBA, a large observation count does not necessarily imply reliable estimates of rare label combinations. The authors should clarify whether the proposed shrinkage adequately handles this imbalance.
-
The effectiveness of C4RC is mainly demonstrated through downstream prediction results. However, the paper provides limited direct evidence that the learned relation scores accurately reflect property dependence. An analysis of the predicted relation signs and strengths, especially for negatively related properties such as those in CEETOX, would better support the motivation of relation calibration.
-
The empirical benefit of hypergraph-based communication is not fully demonstrated. Although MSRHA jointly encodes molecule-target-auxiliary triples, the current experiments do not directly compare it with simpler relation-aware pairwise message passing. A controlled comparison using the same C4RC relation scores would help clarify the contribution of the proposed hypergraph design.
-
The effectiveness of C4RC under sparse auxiliary labels remains unclear. In the single-run experiments in Figure 4(a), switching to four-state targets substantially reduces performance on MUV when auxiliary labels are missing. Since reliability shrinkage is intended to account for limited joint observations, a clearer explanation of this behavior would strengthen the analysis.
Questions for authors
-
In Section 5.2, MSRHA selects auxiliary properties using the largest signed relation scores. How does this strategy retain strongly negative relations when weaker positive relations receive higher scores? Would using the absolute relation scores for Top-k selection be more appropriate?
-
Table 2 shows that removing MSRHA while retaining C4RC performs worse than removing both modules, especially on MUV. Could the authors explain why relation supervision alone reduces prediction performance and how MSRHA addresses this issue?
-
In MSRHA, signed hyperedge messages update only molecule and target property nodes, while auxiliary property nodes remain unchanged. What motivates this design, and would updating auxiliary property nodes affect the performance?
-
In Section 5.1, C4RC uses a support-label-conditioned independence baseline for unseen target properties, which differs from the joint-marginal baseline used for meta-training property pairs. Could the authors clarify how the relation head handles this difference when estimating target-auxiliary relations for query molecules? Does this change in calibration affect the reliability of relation transfer to unseen tasks?