- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper asks whether unlabeled graph observables can indicate when a source-calibrated conformal predictor will continue to achieve acceptable coverage under graph-environment shift. It first gives an identifiability observation: for a fixed observable marginal over (G, X), different label conditionals can induce arbitrarily different miscoverage, so unlabeled graph statistics cannot provide a universal coverage certificate. The proposed TOPOTRANSFER procedure therefore assumes exchangeable labeled calibration environments from a declared shift family, computes topology/representation signatures, fits a cross-fitted monotone predictor of excess miscoverage, conformalizes environment-level residuals, and abstains when a target is outside calibrated support or fails a leave-one-environment-out stability check.
Critical strengths
-
The paper makes a useful conceptual distinction between a universal coverage certificate and a selective environment-level diagnostic calibrated only within a declared shift family.
-
The single reported GOOD-Cora experiment at least evaluates the relevant selective trade-off (false certificates jointly with acceptance) and includes support/refusal ablations.
Critical weaknesses
- The central positive validity claim is not backed by a theorem for the final procedure. Equation (2) conformalizes residuals, but the actual decision additionally depends on learned signatures, component-wise support checks, leave-one-environment-out stability, the finite-environment correction b_E, and the acceptance threshold. I could not find a formal finite-sample statement showing what error probability the complete selected decision controls, under precisely what exchangeability assumptions, and how the data-dependent selection preserves that guarantee. For a paper whose main contribution is framed as a selective validity decision, this is a fundamental gap rather than a presentation issue.
- The empirical evaluation is far below the evidence required by the paper’s own stated hypotheses.
Questions for authors
- Can the authors state and prove a finite-sample theorem for the final TOPOTRANSFER decision rule, including cross-fitting, estimation of environment miscoverage, b_E, component-wise support filtering, and the stability gate? Please state exactly what event/probability is controlled and over which randomness.
- Are the support ranges and stability thresholds fixed independently of the residual calibration scores, or selected using the same environments? If they are data-dependent, why does the conformal residual argument remain valid after this selection?