- Rating
- 1
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper introduces PI-CSRN, a graph-based surrogate model for solving partial differential equations (PDEs) on variable geometries. The framework integrates a Graph U-Net with Graclus-based graph coarsening, attention pooling, and TransformerConv layers. It incorporates a physics-informed loss based on discrete mesh operators and uses conformalized quantile regression (CQR) to construct prediction intervals. In addition, a stacked residual network is designed to refine the ensemble predictions and adjust the associated prediction intervals under distribution shifts. The proposed PI-CSRN is evaluated on procedurally generated benchmarks covering two-dimensional electrostatics and three-dimensional elasticity.
Critical strengths
-
The problem is well motivated, addressing the important challenge of uncertainty quantification in mesh-based surrogate models when test geometries differ from those encountered during training.
-
The use of relative features and discrete mesh operators to preserve invariance to rigid-body transformations is a coherent and well-justified design choice.
Critical weaknesses
-
The claimed coverage result comes from labelled near-target data, not from the method’s design.
-
The obvious baseline comparisons are missing including recalibrating CQR on the same small dataset, fine-tuning the base model on it, or weighted and non-exchangeable conformal methods. The current comparisons rely primarily on the authors’ own model variants, limiting the evidence for the proposed method’s advantages.
-
The distribution shift and benchmark are insufficiently characterized, with no quantitative description of dataset sizes, mesh sizes, or differences between training and test geometries. Results are based on single runs without random-seed analysis, uncertainty estimates, or confidence intervals. Key implementation details, benchmark data, and quantitative results for 3D elasticity are also missing.
-
The framework combines largely established components, while claims about preserving topology and geodesic structure and preventing representation collapse lack empirical validation. Feature ablations are absent, making it difficult to determine whether the proposed architectural choices provide meaningful benefits.
-
The use of invariant scalar features raises questions about whether vector displacement fields can be represented correctly without an appropriate coordinate frame or equivariant architecture. The traction-free boundary condition appears incorrectly formulated, and the elasticity physics loss and material parameters are insufficiently specified.
-
The evaluation focuses on linear, homogeneous, static PDEs that conventional FEM solvers can solve efficiently. Without runtime comparisons or more challenging test cases, the practical advantages of the proposed surrogate remain unestablished.
-
Several claims, including strict guarantees, optimal performance, and cross-physics generalization, are not adequately supported by the reported experiments. The sigmoid output layer raises questions about predicting unrestricted displacement values, and the discrepancy between training and validation errors requires clarification.
Questions for authors
-
Does the physics loss apply to the central prediction only, or also to the quantile outputs?
-
The output layer in Fig. 1 ends in a sigmoid. How does that work for displacements outside [0, 1]?
-
Why is the single stacked model’s train MAE above its validation MAE? Is it a typo?
2 comments
I thank the authors for their response, which answered my three clarification questions and added useful dataset statistics. However, it leaves most of my main concerns unaddressed. First, valid coverage still depends on labelled near-target data. Without fine-tuning, OOD coverage reaches only 0.800–0.840, below the nominal 0.9 level. The rebuttal also does not say whether the fine-tuning data are separate from the conformal calibration data. If they overlap, the coverage guarantee does not hold. Second, the baselines I requested are still missing: recalibrating CQR on the same near-target data, fine-tuning the base model on that data, and weighted or non-exchangeable conformal methods. Without these, it is unclear whether the stacked residual network adds anything beyond access to labelled near-target data. Third, the distribution shift is still not characterised quantitatively. All results come from single runs without variance or confidence intervals. The 3D elasticity results and the size of the fine-tuning set are still not reported. Finally, the rebuttal does not respond to my questions on the correctness and scope of the method: representing vector fields with invariant features, the boundary-condition and physics-loss formulation, feature ablations, a runtime comparison with FEM, and the claims of strict guarantees and cross-physics generalization. The reported OOD coverage contradicts the claim of strict guarantees.
We would like to thank the reviewer for recognizing the motivation of uncertainty quantification, especially under geometric distribution shifts, and the design choice of combining relative geometric invariants with discrete mesh operators to preserve rigid-body invariance.
Regarding the questions addressed:
1) Does the physics loss apply to the central prediction only, or also to the quantile outputs?
Both Lpde and Lbc of the multi-objective loss function in Equation 4 apply directly to the central prediction y^. However, all three output heads (q^low, q^high, and y^) share a common encode-process-decode backbone. So, the two CQR heads whose predictions are correlated with the central prediction head also benefit, in line with Multi-Task Learning principles. This is demonstrated in Section 5 (Table 1), where adding Lpde and Lbc to the
Conformal Anisotropic U-Net GNNimproves all test-set metrics simultaneously: Test MAE drops from 0.0250 to 0.0220, coverage rises from 0.740 to 0.800, and interval width tightens from 0.066 to 0.064 compared to theConformal Anisotropic U-Net GNNwithout these loss terms.2) The output layer in Fig. 1 ends in a sigmoid. How does that work for displacements outside [0, 1]?
The sigmoid activation in Fig. 1 was applied only to the 2D Electrostatics experiments, where Dirichlet ports are ϕ∈[0,1], thus, the interior potential is strictly bounded. For 3D Linear Elasticity, an unrestricted linear output head was used to predict signed displacement vectors. We have clarified in the revised manuscript that the use of sigmoid activation is optional and task-dependent.
3) Why is the single stacked model’s train MAE above its validation MAE? Is it a typo?
That was indeed a typographic error in Table 1 (
Conformal Stacked Residual GNN), which was corrected in the revised manuscript from0.0190to0.0120.Regarding the weaknesses identified:
Section 5 (Table 1) systematically ablates each architectural and loss component, alongside standard Encode-Process-Decode baselines (
Simple NN,SAGE,Residual SAGE, andResidual Transformer GNN), following the paradigm ofMeshGraphNets(Pfaff et al., 2021), alongsideIsotropic U-Net GNNand a Deep Ensemble UQ baseline (Conformal Anisotropic Ensemble GNN). Crucially, coverage improvement does not actually occur from near-target data. Even without fine-tuning, ourConformal Stacked Residual Ensemble GNNimproves OOD coverage from 0.800 to 0.840 and lowers Test MAE from 0.0220 to 0.0208 over base CQR purely through stacked residual learning. Fine-tuning the residual corrector on a small, disjoint, previously unseen set (†) with reduced complexity relative to the test set led to further improvements in coverage (0.901) and achieved the best Test MAE (0.0159).Across more than 10,000 simulations spanning 2D and 3D topologies, each experiment used 5,000 auto-generated topologies and 500 independent test topologies. Training meshes contain 2,000 to 10,000 nodes, whereas test meshes range from 2,000 to 100,000 nodes and include perturbed procedural parameters to achieve higher complexity with non-convex topologies with multiple holes and sharp concavities. The statistics mentioned are also included in the revised manuscript.