- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper investigates generalization to unseen domain topology in neural PDE operators. It introduces TopoBox-3D, a benchmark with independently varying tunnels and cavities, and evaluates six architectures on Hodge heat diffusion. The analysis compares geometry and topology shifts, examines associations between spectral descriptors and prediction error, and measures harmonic preservation and nonharmonic-component accuracy.
Critical strengths
The motivation is relevant: compatibility with arbitrary meshes does not establish generalization to topological configurations absent from training. The independently controlled tunnels and cavities provide a concrete experimental setting, and Figure 1(a) helps communicate the domain families. The reported topology audits, geometry-level data splits, and comparisons across several architectures are useful features of the benchmark.
Critical weaknesses
My main concern is that the benchmark covers a narrow PDE setting and the experimental analysis provides limited explanation of why models perform poorly under particular geometric or topological changes. The following issues limit the significance of the contribution.
-
The implications beyond Hodge diffusion remain unclear. All three cochain degrees belong to the same linear, autonomous diffusion family, evaluated at a fixed time under homogeneous absolute boundary conditions. Hodge diffusion is a reasonable diagnostic choice because its harmonic kernel has a direct relationship to topology and its spectral evolution is analytically tractable. However, this special structure also limits what the experiments establish about neural PDE operators more broadly. The scalar, circulation, and flux interpretations do not demonstrate applicability to distinct physical PDE systems. Section 6 acknowledges these limitations, but I would like a clearer account of which findings are expected to transfer to other operators or boundary conditions, and why. At present, the broader implications are primarily hypotheses.
-
The heterogeneous results need a stronger explanatory interpretation. Section 5.2 reports that 37 of 45 comparisons for the implicit-topology models have (Q>1), but several cells show (Q<1). These indicate less degradation than the fixed-topology geometry baseline. Separately, Table 8 contains cases with (R<1), indicating lower aggregate OOD error than IID error. These exceptions do not invalidate the study, but they raise an important question: under what conditions does an unseen topology make prediction harder, leave it largely unaffected, or coincide with an easier test distribution? The aggregate tendency does not answer this question. Since topology, boundary structure, mesh size, and spectrum change together, the results need more analysis connecting particular domain changes to the observed model behavior.
-
The diagnostics describe associations more convincingly than they explain model failures. The Rayleigh-quotient correlations and component-level errors provide useful summaries, but they do not establish why particular architectures succeed or fail in different settings. The paper itself notes that the association with relative error may partly reflect target attenuation in the error denominator. Likewise, TNO’s nonharmonic advantage cannot be attributed separately to incidence structure or harmonic information. These qualifications are appropriately acknowledged, but they leave the proposed architectural explanation unresolved. My concern is therefore whether the analysis yields sufficiently actionable insight beyond documenting performance differences within this benchmark.
-
The mathematical exposition makes the contribution difficult to assess. The introduction assumes considerable familiarity with Betti numbers, homology/cohomology, differential forms, and Hodge decomposition. Section 3 then introduces tetrahedralization, chain and cochain complexes, boundary matrices, matrix ranks, and the Euler characteristic with little conceptual explanation. Although Appendices A and B provide additional details, the main text needs a clearer progression from the geometric construction to its topological invariants and then to the diffusion operator. In particular, explaining why removing through-cylinders and enclosed balls gives the stated Betti vector would help readers understand the benchmark. Equation (1)’s kernel–cohomology relationship also deserves a concrete example illustrating harmonic preservation and nonharmonic decay.
Questions for authors
-
Why hodge diffusion is chosen in the benchmark? Which conclusions are specific to Hodge heat diffusion, and what evidence or mathematical argument supports their relevance to other PDEs?
-
Can the authors explain representative (Q<1) and (R<1) cases using the existing geometric, spectral, and component-level diagnostics? What distinguishes them from settings with substantial degradation?
-
What specific mechanism explains the differences between architectures? Could an analysis using absolute or input-normalized error, or a focused structural ablation, strengthen that interpretation?
5 comments
Thank you for the detailed response and substantial revision. Several of my earlier concerns have been addressed. Sections 5.2–5.3 now clarify the distinction between and , and the composition and matching analyses help explain why some aggregate OOD results appear favorable. The input-normalized error controls address the missing normalization check and appropriately limit the interpretation of the Rayleigh–error association. I also appreciate the correction of the TNO probe construction, the expanded audit across three checkpoints, and the withdrawal of the earlier harmonic/complementary tradeoff claim.
However, the mathematical background remains insufficiently developed for readers without prior training in topology and Hodge theory. The added examples provide useful intuition, but essential concepts—including Betti numbers, homology/cohomology, differential forms, chain/cochain complexes, and harmonic subspaces—still appear without a coherent introduction. Section 2 primarily discusses related work. A concise background subsection explaining these concepts and their relationships before the central equations would substantially improve accessibility. A worked example connecting a tunnel or cavity to a nonbounding cycle, its Betti number, and the corresponding harmonic space would be particularly helpful. These foundations are necessary for understanding what the benchmark measures and why its diagnostics are meaningful.
My other main concern is what actionable guidance the benchmark provides for improving models. The revisions strengthen evaluation practice through better treatment of normalization and input composition. However, the connection between the diagnostics and specific model improvements remains unclear.
For example, TNO’s corrected performance suggests that its combined structural access is promising. The comparison does not establish which ingredient contributes to that advantage: incidence information, harmonic coordinates, processor design, or their interaction. Consequently, it is difficult to infer which change a practitioner should make to an existing model after observing poor performance on this benchmark.
Could the authors clarify two points?
-
How can the benchmark guide a concrete intervention? Can a particular diagnostic—such as harmonic distortion, cross-subspace leakage, or complementary-component error—identify a model modification likely to improve OOD performance? A focused ablation or mechanistic argument connecting one diagnosed failure to one intervention would strengthen the benchmark’s value for model development.
-
Under what conditions might that intervention benefit other PDEs? Section 6 appropriately acknowledges that the numerical results and harmonic identities cannot be assumed to transfer. It would nevertheless be helpful to explain which proposed improvements depend on Hodge diffusion’s particular structure and which have a plausible basis for improving geometry/topology OOD generalization in other PDE settings.
I recognize that a diagnostic benchmark can be valuable without introducing a new architecture. My remaining concerns are whether the presentation gives readers the background needed to understand the diagnostic framework, and whether the evaluation supplies sufficient guidance for model improvement with potential relevance beyond this operator family.
Thank you for the careful follow-up and for acknowledging the improvements. We have revised Sections 2.1, 5.6, and 6 to connect the mathematical background, a concrete model intervention, and its conditions for transfer.
1. A connected background and worked example
Section 2.1 now provides a connected primer and worked example before the first Hodge-operator equation; related work follows in Section 2.2. The sequence is:
Chains and cycles → homology and Betti numbers → cochains and differential forms → harmonic representatives.
A chain is a weighted collection of oriented simplices; its boundary is given by an incidence matrix. A cycle has zero boundary. Homology identifies cycles differing by a boundary, and the Betti number counts independent classes over the reals. Cochains assign linear measurements to chains; integration of differential forms gives such measurements. The transpose incidence maps form the cochain complex. Cohomology identifies closed cochains modulo exact ones; with the mass inner product and our boundary conditions, each class has a unique harmonic representative.
Worked example: a box with one through-tunnel and no cavities has one independent loop that cannot bound a surface within the material, so . Its dual cohomology class has a harmonic 1-form representative with nonzero circulation around that loop. Thus and heat evolution preserves . By contrast, removing an enclosed ball produces a nonbounding surrounding surface, contributing to and the harmonic 2-form space. This connects the geometric construction directly to the stationary components being audited.
2. A concrete intervention with a pointwise error guarantee
Section 5.6 adds harmonic-output correction with a pointwise guarantee (Equations 7–8). For any frozen predictor , define
This replaces only the predicted harmonic component by its known, conserved value. The projector is computed from the test-domain discretization, without test solution labels. Since , mass orthogonality gives
=\\|F(w_0)-w_T\\|_M^2 -\\|P_H(F(w_0)-w_T)\\|_M^2.$$ **The diagnosed harmonic error is exactly the removable squared error.** With an exact projector and conserved target component, the correction cannot increase absolute or target-/input-normalized error on any IID or OOD input. Improvement is strict when the original harmonic error is nonzero; the degradation ratios $R$ and $Q$ need not decrease. This yields a specific decision rule: - **Substantial harmonic error or complement-to-harmonic leakage:** apply the correction; the harmonic error energy measures its recoverable benefit. - **Predominantly complementary error:** this correction offers little gain, directing attention to the learned complementary dynamics instead. It removes within-harmonic distortion and complement-to-harmonic leakage, while leaving complementary error, including harmonic-to-complement leakage, unchanged. It requires additional structural information for models not already given $P_H$; its additional basis and projection costs must also be reported. This is an explicit mechanistic guarantee, not a new empirical ablation or a claim to isolate the ingredients behind TNO's advantage. Section 5.6 explains both when this modification helps and when it cannot address the dominant error. ### 3. Conditions for transfer to other PDEs Section 6 states the transfer condition (Equation 9): **enforce a known solution component**. Suppose a domain-specific orthogonal projector $P$ satisfies $$Pw_T=g(w_0,f,\mathrm{BC}),$$ where $g$ is known from the initial data, forcing, and boundary conditions. Then $g+(I-P)F(w_0)$ obeys the same error-reduction identity. This does not require the full PDE to be linear: a known mass balance, for example, can determine the constant component in a conservative system. For Hodge heat, $P=P_H$ and $g=P_Hw_0$. For another PDE, the relevant component and its evolution must be established from that equation and its boundary conditions; blindly preserving a Hodge component is not justified. Approximate projectors or an incorrect balance law also remove the exact guarantee. This provides a concrete route from diagnosed invariant-component error to model correction, with explicit conditions for broader applicability. We hope this mechanism and the revised background address the remaining concerns about accessibility and model-development value.
Thank you for the detailed review. We added normalization and composition controls, geometric matching, and a three-seed subspace audit to address your questions, and clarified the mathematical exposition.
1. What explains the cases with Q<1 or R<1?
The inequalities answer different questions: Rp is the median seed-wise OOD/IID mean-error ratio, whereas Qp=Rp/RA compares degradation against Protocol A.
For MGN D2, R=1.129 but Q=0.815: error increases, but less than in A2 (RA=1.385). Thus, Q<1 does not imply improved OOD accuracy.
For TNO C2, normalization changes the conclusion:
\text{Error normalization} & R & 95\\%\text{ geometry interval} \\\\ \hline \text{Target norm} & 0.776 & [0.737,0.822] \\\\ \text{Input norm} & 1.035 & [1.019,1.052] \end{array}$$ Target-relative $R$ is 1.059 on nonharmonic inputs, versus 0.738, 0.515, and 0.419 for weak, balanced, and strong harmonic mixtures. Standardizing realized harmonic-fraction composition on common support gives $R=1.080$; adding Rayleigh-quartile stratification gives 0.949. Geometric matching yields 1.016 and 1.015, with both intervals including one. These are descriptive contrasts on restricted populations. **They show that aggregate OOD improvement depends on composition and normalization**, rather than establishing intrinsically easier unseen topology (Sections 5.2–5.3; Appendices D.3–D.4). ### 2. Absolute/input-normalized error control Following your suggestion, we use $$E_{\mathrm{in}}=\frac{\\|\widehat w_T-w_T\\|_M}{\\|w_0\\|_M}, \qquad q_T=\frac{\\|w_T\\|_M}{\\|w_0\\|_M}.$$ Ignoring the small numerical stabilizer, $E_{\mathrm{rel}}=E_{\mathrm{in}}/q_T$. Inputs have unit mass norm to numerical precision, so absolute and input-normalized errors coincide. The median Rayleigh–error correlations are: $$\begin{array}{lrrrr} \text{Normalization} & k=0 & k=1 & k=2 & \text{All 72 cells} \\\\ \hline \text{Target} & .837 & .729 & .783 & .784 \\\\ \text{Input} & .770 & -.254 & -.433 & .024 \end{array}$$ **This confirms the importance of target attenuation.** We revised the difficulty interpretation accordingly. The main finding remains: input normalization gives $Q>1$ in **41/45** implicit-model cells, with geometric mean 1.183. The control refines the explanation while preserving the widespread generalization gap (Section 5.4; Appendix D.2). ### 3. What can be concluded about architectural mechanisms? The expanded audit uses three frozen checkpoints and ten geometries per split across six tasks and six models: **2,160 evaluations**. It identified a sparse-input construction fault in the earlier TNO probes. We corrected and reference-verified the inputs and recomputed all 360 TNO records. Corrected OOD results are: $$\begin{array}{lrr} \text{Model} & I_H & L_{H\to\perp} \\\\ \hline \text{MGN} & .1887 & .1279 \\\\ \text{RIGNO} & .1874 & .1088 \\\\ \text{Transolver} & .2910 & .1473 \\\\ \text{GNOT} & .1984 & .1386 \\\\ \text{GAOT} & .5913 & .0538 \\\\ \text{TNO} & \mathbf{.0231} & \mathbf{.0226} \end{array}$$ Here $I_H$ is total harmonic-input error and $L_{H\to\perp}$ outward leakage. These are pooled medians of three-seed medians over 60 unit-mass task–geometry probes per model; some tasks share geometries. **We withdrew the earlier harmonic/complementary tradeoff claim:** TNO now shows low harmonic error and still ranks first in complementary accuracy on all six tasks. We verified 24 incidence matrices; a 192-sample conventional-inference spot check agrees with stored relative errors within $4.36\times10^{-5}$, without constituting a full benchmark rerun. Appendix D.5 provides task-wise intervals and full-test nonharmonic leakage. The evidence supports TNO's combined structural access; separating incidence from harmonic information requires a focused ablation. Zero/identity baselines show why harmonic preservation alone is insufficient to establish topology recognition (Appendix D.6). ### 4. Hodge diffusion and clearer exposition Hodge heat links topology to testable operator identities. The Introduction now illustrates preservation versus decay: $$w_0=a h+b u_\lambda \quad\Longrightarrow\quad w_T=a h+b e^{-\kappa T\lambda}u_\lambda,$$ Here $h$ is harmonic and $u_\lambda$ has positive eigenvalue. Before introducing the chain complex, Section 3 explains $(1,g,c,0)$ through connected material, tunnel loops, cavity cycles, and the absence of a closed three-dimensional cycle. Section 6 separates reusable evaluation lessons from Hodge-specific identities and rankings; the three degrees remain one PDE family. We hope these controls and clearer explanations strengthen the empirical contribution and address your concerns.