acceptodds
Under review as a conference paper at ICLR 2027

PINN-ffusion: Physics-Informed Neural Network via Diffusion-based Adaptive Sampling and Anisotropic Domain Decomposition

Abstract

Physics-Informed Neural Networks (PINNs) provide a flexible framework for solving partial differential equations (PDEs) by incorporating physical constraints into neural network training. However, their performance is often limited by the non-uniform distribution of PDE residuals, which are typically concentrated in localized regions such as boundary layers and sharp gradients. Existing approaches address this issue through either adaptive sampling as a minimax optimization problem or domain decomposition based on fixed partitioning schemes, which lead to instability and limited flexibility. In this study, we propose PINN-ffusion, a unified framework that reformulates adaptive sampling and domain decomposition as a coupled problem of probability evolution and geometry learning. Instead of adversarial optimization, we construct the sampling distribution via a diffusion process, enabling stable adaptation while preserving the original PINN objective. In addition, we introduce a geometry-aware domain decomposition based on diversity and uncertainty metrics, generalizing isotropic partitions into anisotropic structures aligned with the residual landscape. The proposed framework allows sampling and domain geometry to co-evolve during training, improving the allocation of computational resources to complex regions. We provide a theoretical justification of consistency with the PINN objective and demonstrate through experiments that PINN-ffusion achieves improved accuracy and stability, particularly in problems with localized and anisotropic error structures.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Partial Differential Equation, Adaptive Sampling, Domain Decomposition, Deep Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_XCupOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PINN-ffusion, a physics-informed neural network framework that combines adaptive collocation sampling with anisotropic domain decomposition. Building on PINN-ball, it replaces adversarial sampling with heat-flow smoothing of a residual-informed density and generates collocation points via a probability-flow ODE. It further introduces ellipsoidal subdomains whose geometry is determined by sample covariance and residual-gradient information, allowing local subnetworks to adapt to directional solution structures.

Critical strengths
  1. Both proposed modifications are motivated by clear, targeted objectives. The diffusion-based sampling mechanism is designed to enhance sampling stability, while the ellipsoidal subdomain strategy enables local subnetworks to better model solution structures with pronounced directional patterns.

  2. The experimental setup delivers rigorous supporting evidence. With matched optimizers and iteration budgets for both PINN-ball and PINN-ffusion, ablation studies confirm the additive contribution of each component.

Critical weaknesses
  1. Training stability needs more direct tests as the number of subdomains increases. Table 7 reports results across initializations, and Figure 4 shows training curves for Burgers. However, these tests do not show whether diffusion reduces the training oscillations observed in PINN-ball when more balls are used. Please compare both methods across several subdomain counts and random seeds. Training curves, measures of oscillation, and failure rates would better support the stability claim. The PINN-ball baseline should use a well-tuned learning-rate schedule.

  2. The paper tests five PDE benchmarks, including coupled systems and cylinder flow, but the tests remain limited to low-dimensional settings. More demanding cases, such as coupled multiphysics problems, would help assess its wider use. A comparison with a suitable classical numerical solver is also missing. Comparing runtime and memory use at a common accuracy target would help show when the proposed method offers a practical benefit.

Questions for authors
  1. Eq. (13) keeps the metric bounded and handles S=0. However, how does the method handle a very small, noisy S? Is there a threshold or smoothing rule to avoid unnecessary geometry changes near convergence?

  2. Appendix F.2 states that anisotropic scaling is constrained to avoid unstable training. What exact constraints are used for axis ratios, subdomain sizes, and overlap? How do these rules maintain domain coverage and control the derivatives of the partition weights?

7 comments
AuthorsOpenReview ↗

Response to Reviewer XCup (1/5)

We feel grateful for the constructive reviews. We provide responses to the reviews as well as supporting experimental results below. Also, we uploaded the revised manuscript by highlighting our changes in colored fonts. All section, equation, table, and figure numbers below refer to the revised manuscript, in which revised and newly added parts are marked in red. References [1]–[10] are listed at the end of Part 3/5.

(W1) Training stability with respect to the number of subdomains.

We thank the reviewer for the concrete experimental design. Following the suggestion, we have added the comparison to Appendix G.12 of the revised manuscript with the following setup. On Burgers, we vary the number of subdomains as K∈{4,8,16}K \in \lbrace 4, 8, 16 \rbrace and repeat each configuration with five random seeds, using the same optimizer and number of iterations for both methods. The learning-rate schedule of PINN-ball (initial value and exponential decay rate) was selected from a grid of three values based on the validation error at K=8K = 8 and used for all KK, and the same tuning budget was applied to PINN-ffusion.

As the evaluation metric, we report the mean and standard deviation of the relative L2L^{2} error (Rel. L2) over the five seeds. The results are reported in Table 18, and the training curves are shown in Fig. 13 (Appendix G.12).

Table 18. Training stability on Burgers with respect to the number of subdomains (five seeds)

MethodKKRel. L2 mean (std)
PINN-ball47.9e-4 (1.6e-4)
88.6e-4 (3.9e-4)
161.2e-3 (1.1e-2)
PINN-ffusion45.4e-4 (7.8e-5)
83.7e-4 (1.6e-5)
163.7e-4 (6.2e-5)

For all values of KK, the mean error and standard deviation of PINN-ffusion are smaller than those of PINN-ball, and the difference widens as KK increases. For PINN-ball, increasing KK from 4 to 16 causes the mean error to increase from 7.9×10−47.9 \times 10^{-4} to 1.2×10−31.2 \times 10^{-3}, and the standard deviation to increase from 1.6×10−41.6 \times 10^{-4} to 1.1×10−21.1 \times 10^{-2}, a rise of approximately 70 times. In contrast, for PINN-ffusion, the mean error does not increase even as KK increases (from 5.4×10−45.4 \times 10^{-4} to 3.7×10−43.7 \times 10^{-4}), and the standard deviation remains at or below 7.8×10−57.8 \times 10^{-5} for all values of KK. As shown in the learning curve (Fig. 13), PINN-ball exhibits increased oscillations as KK increases, whereas PINN-ffusion converges stably for all values of KK. In other words, the oscillations and deviations from the seed that increase in PINN-ball when the number of subdomains is increased are significantly reduced in diffusion sampling. This result is consistent with a design that uses a smoothing density without trainable parameters, rather than a trainable adversarial sampler.

AuthorsOpenReview ↗

Response to Reviewer XCup (2/5)

(W2) Scope of application and comparison with a classical numerical solver.

We thank the reviewer for this valid point. We have strengthened the manuscript in two respects.

First, we have clarified the scope of the method. Our method aims at the stable training of low-regularity solutions of low- to moderate-dimensional spatiotemporal PDEs, which is the common setting of PINN research employing adaptive sampling and domain decomposition. Domain-decomposition PINNs were proposed to handle large domains and multi-scale solutions [1, 2], and adaptive sampling likewise targets low-dimensional, low-regularity problems. The authors of AAS [3] note that the difficulty of high-dimensional problems shares similarities with that of low-dimensional problems of low regularity and adopt a two-dimensional peak problem as their central example, and PINN Balls [4], the direct predecessor that combines the two components, addresses the memory cost of second-order optimizers with experiments that all have at most three inputs. In contrast, high-dimensional PDEs are approached with fundamentally different techniques, such as stochastic dimension gradient descent (SDGD) [5]. Moreover, even in recent work, higher-dimensional problems and multiphysics are treated as separate problems that require sequential training over time slices [6] or dedicated architectures that handle the disparity of gradient scales across fields [7]. This supporting evidence is summarized in Appendix G.3 of the revised manuscript.

The current benchmarks already include a coupled system (Boussinesq–Burger) and a flow around a complex geometry (Navier–Stokes around a cylinder). Since extending the method to (3+1)(3 + 1)-dimensional problems and strongly coupled multiphysics would require mesh-free smoothing as well as field-wise sampling and decomposition, we have stated this extension as future work in Section 6 (Limitations) and Appendix G.3 of the revised manuscript.

Second, we have compared our method with a classical numerical solver. On Burgers, we compare with a recent high-order compact finite-difference (FD) scheme [8]. We implemented this scheme (fourth-order backward differentiation (BDF4) in time, fourth-order Padé and compact differences in space, and an extrapolation-based linearization of the nonlinear term) in one spatial dimension and measured the relative L2L^{2} error, run time, and peak memory under grid refinement. The FD error is computed against the exact solution obtained by the Cole–Hopf transformation. This comparison has been added to Appendix G.13 of the revised manuscript (Table 19).

Table 19. Comparison of PINN-ffusion and a classical FD solver on Burgers

MethodSettingRel. L2 errorTimePeak memoryHardware
PINN-ffusion5 subdomains, 2,000 epochs4.62e-436.5 s/epoch36 GBGPU (RTX A6000 Ada)
FD [8] (matched accuracy)510 intervals, τ=1.25×10−3\tau = 1.25 \times 10^{-3}3.14e-40.071 s0.31 MB1 CPU core
FD [8] (best accuracy)4,080 intervals, τ=1.56×10−4\tau = 1.56 \times 10^{-4}5.72e-81.53 s1.10 MB1 CPU core

FD reaches the accuracy of PINN-ffusion in 0.07 s with 0.31 MB of memory and reduces the error to 5.72×10−85.72 \times 10^{-8} within 1.5 s. Consistent with recent systematic comparisons [9, 10], classical solvers are considerably faster and more accurate for forward problems with known solutions.

AuthorsOpenReview ↗

Response to Reviewer XCup (3/5)

(W2, continued)

However, the motivation of PINN research lies not in replacing classical solvers for such forward problems but in addressing problems that are difficult for classical solvers. Examples include inverse problems and data assimilation, in which unknown coefficients or initial conditions are estimated from sparse and noisy observations; parametric problems and surrogate modeling, which would otherwise require a new solve for every parameter value; and problems on geometries for which mesh generation is difficult, as well as high-dimensional problems. PINNs are well suited to these problems because they combine observational data and governing equations in a single loss and can learn the unknowns jointly through automatic differentiation.

For PINNs to be used in these settings, however, they must first be trained stably and accurately on forward problems with known solutions. Current PINNs frequently exhibit unstable training or failure for solutions with shocks, thin layers, or directional structures, which explains the active research on improving PINN training through adaptive sampling, domain decomposition, and second-order optimizers. Since our benchmarks are designed to measure precisely this training capability against reference solutions, we have clarified in Section 1 (Introduction) and Appendix G.13 of the revised manuscript that our contribution is an improvement in the accuracy and stability of PINN training. In addition, we have stated in Section 6 (Limitations) that applying the improved training to inverse and parametric problems is left for future work.

References

[1] Finite Basis Physics-Informed Neural Networks (FBPINNs): A Scalable Domain Decomposition Approach for Solving Differential Equations (Ben Moseley et al., 2023)

[2] Multilevel Domain Decomposition-based Architectures for Physics-informed Neural Networks (Victorita Dolean et al., 2024)

[3] Adversarial Adaptive Sampling: Unify PINN and Optimal Transport for the Approximation of PDEs (Kejun Tang et al., 2024)

[4] PINN Balls: Scaling Second-Order Methods for PINNs with Domain Decomposition and Adaptive Sampling (Andrea Bonfanti et al., 2025)

[5] Tackling the Curse of Dimensionality with Physics-Informed Neural Networks (Zheyuan Hu et al., 2024)

[6] Time-slices Based Training of Physics-Informed Neural Networks for 3D Non-Stationary Thermal Inversion Simulations (Maciej Sikora and Maciej Paszyński, 2026)

[7] LSTM-PINN for Steady-State Electrothermal Transport: Preserving Multi-Field Consistency in Strongly Coupled Heat and Fluid Flow (Yuqing Zhou et al., 2026)

[8] A New High-order Compact Numerical Method for the Two-dimensional Nonlinear Burgers' Equation (Shengdi Wang et al., 2026)

[9] Can Physics-Informed Neural Networks Beat the Finite Element Method? (Tamara G. Grossmann et al., 2024)

[10] Weak Baselines and Reporting Biases Lead to Overoptimism in Machine Learning for Fluid-related Partial Differential Equations (Nick McGreivy and Ammar Hakim, 2024)

Reviewer_TRYROpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This manuscript combines adaptive sampling with domain decomposition methods, which is not new. The paper proposes PINN-fussion, which replaces the adversarial minimax adaptive sampling with a diffusion-based density evolution, and replaces isotropic ball-shaped subdomains with Mahalanobis anisotropic ellipsoids. The claimed contributions are a collapse-free sampling scheme and a geometry-aware decomposition that better captures directional error structures. Combining domain decomposition is also well studied in many published works. The motivation of this work is not clear because it is like a rigid combination. See the details critical weaknesses. Based on these points, this work is not suitable for publication on ICLR.

Critical strengths

Establishing the consistency of the diffusion-based sampling loss with the original PINN residual objective using kernel function is interesting.

Critical weaknesses

Sampling strategies are important for high-dimensional PDEs because the geometric properties of high-dimensional spaces are different from the low-dimensional one. Domain decomposition works well for low-dimensional problems, but it becomes very challenging when applied to high-dimensional problems. Hence, there is no compelling motivation for combining adaptive sampling with domain decomposition. This is the biggest concern.

Experimental comparison omits the adaptive sampling baselines. The paper compares against PINN, XPINN, APINN, and PINN-ball, but not against adaptive sampling itself, where PINN-ball’s sampling mechanism is derived. If the diffusion sampler is superior to adversarial sampling, the natural baseline is adaptive sampling, not just its domain-decomposition version.

In addition, the authors’ literature review on adaptive sampling is severely deficient. Many relevant studies are not mentioned. For example, I go through the reference in this manuscript to know that adversarial adaptive sampling (AAS) served as the inspiration for PINN balls, which inspired the current manuscript. Yet, the manuscript contains no reference to AAS, which clearly indicates that the authors lack a clear understanding of the research trajectory in this field.

Questions for authors

See Critical Weaknesses.

8 comments
AuthorsOpenReview ↗

Response to Reviewer TRYR (1/3)

We feel grateful for the constructive reviews. We provide responses to the reviews as well as supporting experimental results below. Also, we uploaded the revised manuscript by highlighting our changes in colored fonts. All section, equation, table, and figure numbers below refer to the revised manuscript. References [1]–[6] are listed at the end of Part 3/3.

(W1) Motivation for combining adaptive sampling with domain decomposition.

We thank the reviewer for articulating this central concern so clearly. We agree with both premises: sampling is particularly important in high dimensions, and domain decomposition becomes difficult in high dimensions. However, we respectfully disagree with the conclusion that there is no compelling motivation for combining the two components, for the following three reasons.

First, adaptive sampling is also essential for low-dimensional problems with low regularity. RAR [1], RAD/RAR-D [2], R3 [3], and QR-DEIM [4] have all demonstrated their effectiveness on shocks and thin layers in low-dimensional PDEs such as the Burgers and Allen–Cahn equations. The authors of AAS [5] likewise note that the difficulty of high-dimensional problems shares similarities with that of low-dimensional problems of low regularity, which motivates adaptive sampling. Our method targets precisely this regime, which is also the regime of PINN Balls [6], whose benchmarks all have at most three inputs. We do not claim applicability to high-dimensional problems; we have stated this scope in Sections 1 (Introduction) and 6 (Limitations) of the revised manuscript and summarized the supporting literature in Appendix G.3.

Second, the two components are structurally coupled rather than simply stacked. In u=∑jλjuju = \sum_{j} \lambda_j u_j, expert jj is trained only on points within supp⁡(λj)=Ωj\operatorname{supp}(\lambda_j) = \Omega_j; therefore, when NN points are drawn from a density pp, the expected number of samples received by expert jj is

E[nj]=N∫Ωjp(x),dx.\mathbb{E}\left[ n_j \right] = N \int_{\Omega_j} p(x) \\, dx.

Hence, without adaptive sampling, a small subdomain assigned to a sharp structure receives very few samples; conversely, without decomposition, a single network must simultaneously fit all regions in which the samples concentrate. Since the allocation of points determined by the sampler and the allocation of capacity determined by the decomposition govern each other's effectiveness, the two should be designed jointly. In our method, the covariance of the sampled points (Σ\Sigma in Eq. 13) and the residual gradient (SS in Eq. 14) directly determine the shape of each subdomain, so that this coupling is built into the method. This discussion has been added to Appendix G.3 of the revised manuscript.

Third, the effect of this coupling is confirmed experimentally. In the 2×22 \times 2 ablation on Navier–Stokes (Table 3), each component alone reduces the error of PINN-ball by 45.0–58.945.0–58.9%, and combining both components further reduces the error on every variable relative to either component alone (by 18.1–30.018.1–30.0% on the velocity components and by 71.7–75.871.7–75.8% on the pressure). For the pressure, the combined error of 1.18×10−21.18 \times 10^{-2} is even lower than 2.29×10−22.29 \times 10^{-2}, the value predicted by multiplying the two individual reduction ratios. Moreover, the anisotropic decomposition attains comparable accuracy with fewer subdomains: on Burgers, PINN-ffusion with four subdomains is more accurate than PINN-ball with five (Fig. 2; see Fig. 6 in Appendix D for Helmholtz and Navier–Stokes). Table 17 (Appendix G.11), presented in our response to W2, further shows that combining the anisotropic decomposition with our sampler reduces the error by 16.3–45.616.3–45.6% relative to using the sampler alone without decomposition.

AuthorsOpenReview ↗

Response to Reviewer TRYR (2/3)

(W2) Comparison with adaptive sampling baselines.

We thank the reviewer for this valuable suggestion. To begin with, we first describe two comparisons that are already present in the manuscript and then present the newly added experiment.

First, the manuscript already contains a direct comparison with adversarial sampling. In Table 3, "PINN-ball" and "w/o Anisotropic" share the isotropic decomposition, the optimizer, and the number of iterations, and differ only in the sampler. This constitutes a direct comparison between the adversarial sampler of PINN-ball, which originates from AAS [5], and our diffusion sampler; our sampler reduces the errors in uu, vv, and pp by 55.7%, 55.8%, and 52.9%, respectively. In Section 4.3 of the revised manuscript, we have stated explicitly that the sampler of PINN-ball is based on AAS.

Second, the manuscript also contains comparisons with other adaptive sampling methods. In Table 10 of Appendix G.7 (referenced in Section 4.3), RAD [2], QR-DEIM [4], and QR-DEIM-R [4] are compared by replacing only the sampler. The strongest of these baselines, QR-DEIM-R, attains errors of 8.16×10−38.16 \times 10^{-3}, 8.30×10−38.30 \times 10^{-3}, and 3.27×10−23.27 \times 10^{-2} in uu, vv, and pp, whereas our method attains 5.89×10−35.89 \times 10^{-3}, 5.82×10−35.82 \times 10^{-3}, and 1.18×10−21.18 \times 10^{-2}.

Having said that, as suggested by the reviewer, we have added an experiment that compares our method with adaptive sampling itself, without domain decomposition. On Navier–Stokes, we compare a single network trained with AAS [5] against the same single network trained with only our diffusion sampler (probability flow). The two settings share the network architecture, optimizer, number of iterations, and number of collocation points, and differ only in the sampler. The results are reported in Table 17, and this experiment has been added to Appendix G.11 of the revised manuscript.

Table 17. Relative L2L^{2} errors on Navier–Stokes comparing samplers without domain decomposition (PINN-ffusion includes the anisotropic decomposition)

Methoduvp
AAS [5]9.34e-39.05e-33.45e-2
Probability flow7.04e-37.46e-32.17e-2
PINN-ffusion5.89e-35.82e-31.18e-2

Even without decomposition, our sampler reduces the errors of AAS in uu, vv, and pp by 24.6%, 17.6%, and 37.1%, respectively, indicating that the advantage of the sampler does not depend on the decomposition. Combining it with the anisotropic decomposition (PINN-ffusion) further reduces the errors by 16.3%, 22.0%, and 45.6%, yielding errors 36.9%, 35.7%, and 65.8% lower than those of AAS. Section 4.3 of the main text now also refers to this comparison.

AuthorsOpenReview ↗

Response to Reviewer TRYR (3/3)

(W3) Literature on adaptive sampling and the missing citation of AAS.

We thank the reviewer for pointing out the omission of AAS. As the reviewer correctly notes, AAS [5] is the direct origin of the sampler of PINN Balls [6] and should have been cited. In Appendix G.2 of the revised manuscript, we have added a related-work paragraph that summarizes the development of adaptive sampling, together with a comparison table (Table 7), and we now cite AAS and the line of adaptive sampling methods in Sections 1 and 2.1 of the main text.

Research on adaptive sampling has developed as follows. RAR [1] began as a greedy refinement strategy that adds candidate points with large residuals to the training set, and RAD/RAR-D [2] generalized this approach to drawing points from a residual-based density. R3 [3] combines the retention of high-residual points with uniform resampling to mitigate propagation failures with a fixed number of points, and QR-DEIM [4] applies QR column pivoting to residual snapshots so that the selection also reflects how the residual evolves during training. AAS [5] models the sampler as a deep generative model trained jointly with the solution through a min-max objective, and PINN Balls [6] employs this adversarial sampler to learn the domain decomposition as well.

Table 7. Comparison of adaptive sampling methods for PINNs

MethodPoint update#PointsLearned samplerDomain decomposition
RAR [1]Greedy additionGrowing✗✗
RAD / RAR-D [2]Density resamplingFixed/Growing✗✗
R3 [3]Retain + resampleFixed✗✗
QR-DEIM [4]QR pivotingGrowing✗✗
AAS [5]Adversarial generatorFixed✓✗
PINN-ffusion (Ours)Heat diffusionFixed✗✓ (anisotropic)

Our method inherits from RAD and AAS the use of a target density derived from the residual, but differs in two respects. First, the sampler has no learnable parameters. The density is constructed analytically by heat-semigroup smoothing, and the points are transported deterministically by the probability-flow ODE; therefore, unlike AAS, neither a sampler network nor adversarial min-max updates are required. Second, sampling is coupled with domain decomposition: the covariance of the sampled points (Eq. 13) and the residual gradient (Eq. 14) directly determine the shape of the anisotropic elliptical subdomains. Among the methods in Table 7, PINN-ffusion is the only one coupled with domain decomposition, whereas PINN Balls [6] combines AAS with an isotropic ball decomposition (Table 1).

The effect of these differences is confirmed experimentally. Because the sampler has no learnable parameters, our method is 19% faster per epoch than PINN-ball with the same second-order optimizer (Table 16), and replacing only the adversarial sampler with ours under a fixed decomposition reduces the error by 52.9–55.852.9–55.8% (Table 3). In comparisons in which only the sampler is replaced, our method also attains lower errors than RAD, QR-DEIM, and QR-DEIM-R (Table 10), and the direct comparison with AAS without decomposition is reported in Table 17 (Appendix G.11) in our response to W2.

(Q) Questions regarding the critical weaknesses.

All points raised in this question are addressed in our responses to W1–W3 above.

References

[1] DeepXDE: A Deep Learning Library for Solving Differential Equations (Lu Lu et al., 2021)

[2] A Comprehensive Study of Non-adaptive and Residual-based Adaptive Sampling for Physics-informed Neural Networks (Chenxi Wu et al., 2023)

[3] Mitigating Propagation Failures in Physics-informed Neural Networks using Retain-Resample-Release (R3) Sampling (Arka Daw et al., 2023)

[4] Adaptive Collocation Point Strategies For Physics Informed Neural Networks via the QR Discrete Empirical Interpolation Method (Adrian Celaya et al., 2025)

[5] Adversarial Adaptive Sampling: Unify PINN and Optimal Transport for the Approximation of PDEs (Kejun Tang et al., 2024)

[6] PINN Balls: Scaling Second-Order Methods for PINNs with Domain Decomposition and Adaptive Sampling (Andrea Bonfanti et al., 2025)

Reviewer_nZHXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PINN-ffusion, which combines diffusion-based adaptive sampling with anisotropic domain decomposition for physics-informed neural networks (PINNs). The method is evaluated on five PDE benchmarks and achieves improvements over PINN-ball and several conventional PINN baselines.

Critical strengths
  1. The experimental results are encouraging, especially on the Klein–Gordon and Navier–Stokes benchmarks.
  2. the motivation for jointly considering adaptive sampling and domain decomposition is quite novel.
Critical weaknesses
  1. Some benchmark specifications appear inconsistent. In Appendix E.3, the Klein–Gordon benchmark is described as having the initial conditions (u(x,0)=ut(x,0)=0u(x,0)=u_t(x,0)=0). However, the stated reference solution is (u(x,t)=xcos⁡(5πt)+(xt)3u(x,t)=x\cos(5\pi t)+(xt)^3), which gives (u(x,0)=x).
  2. The computational complexity of diffusion sampling is not sufficiently explained. The initial density depends on the PDE residual and is not necessarily separable across dimensions. As a result, the separability of the kernel alone does not appear sufficient to establish this complexity.
Questions for authors
  1. Could the authors clarify the inconsistency in the Klein–Gordon initial conditions and confirm the exact benchmark configuration used?
  2. Could the authors clarify the implementation of the diffusion sampler and justify the stated (O(dN)) complexity for non-separable residual distributions?
3 comments
AuthorsOpenReview ↗

Response to Reviewer nZHX

We feel grateful for the constructive reviews. We provide responses to the reviews as well as supporting experimental results below. Also, we uploaded the revised manuscript by highlighting our changes in colored fonts. All section, equation, table, and figure numbers below refer to the revised manuscript.

(W1) Inconsistency in the initial conditions of the Klein–Gordon benchmark.

We thank the reviewer for the careful reading. As the reviewer correctly points out, the reference solution

u(x,t)=xcos⁡(5πt)+(xt)3u(x,t) = x\cos(5\pi t) + (xt)^{3}

satisfies u(x,0)=xu(x,0) = x and ut(x,0)=0u_t(x,0) = 0; hence, the initial conditions stated in Eq. (70) of Appendix E.3 of the submitted version contain a typographical error. This error is confined to the text and does not affect the experiments. As stated in Appendix E.3, both the boundary terms and the source term ff are computed from the reference solution (Eq. (75) of the revised manuscript), and the target values at the Nb=3,000N_b = 3{,}000 points that impose the initial and boundary conditions are also taken from this reference solution. Consequently, all methods are trained under identical conditions and evaluated against the same reference solution.

The exact configuration used in the experiments is as follows. The domain is (x,t)∈[0,1]2(x,t) \in [0,1]^{2}, the governing equation is utt−uxx+u3=fu_{tt} - u_{xx} + u^{3} = f, and ff is computed from the reference solution. The initial conditions are u(x,0)=xu(x,0) = x and ut(x,0)=0u_t(x,0) = 0, and the boundary conditions are u(0,t)=0u(0,t) = 0 and u(1,t)=cos⁡(5πt)+t3u(1,t) = \cos(5\pi t) + t^{3}. We use Nr=10,000N_r = 10{,}000 collocation points and Nb=3,000N_b = 3{,}000 initial and boundary points, with four subdomains and networks of two layers of width 28 (Table 5). In Appendix E.3 of the revised manuscript, the initial conditions (Eq. (73)) and the boundary conditions (Eq. (74)) have been corrected accordingly.

(W2) Computational complexity of diffusion sampling.

We appreciate this precise observation. The statement in the original manuscript that "the product structure reduces a dd-dimensional evaluation to dd one-dimensional evaluations" does not account for the yy-integral in Eq. (45) when p0p_0 is non-separable. In the revision, we have corrected the corresponding sentences in Section 3.1 and Appendix C.1.3 and added an exact cost analysis in Appendix C.1.4.

What the separability of the kernel guarantees is a factorization of the operator, not the separability of p0p_0. Since the Neumann heat semigroup on a rectangular domain is diagonal in the cosine eigenbasis of each axis, evaluating p0p_0 on a tensor grid with nin_i nodes along axis ii (Ng=∏iniN_g = \prod_{i} n_i) yields the following (Eq. (51) of the revised manuscript):

Pt=P0×1H1(t)×2⋯×dHd(t),Hi(t)=Cidiag⁡(e−μi,nt)nCi−1,μi,n=(nπLi)2\mathbf{P}_t = \mathbf{P}_0 \times_1 \mathbf{H}_1(t) \times_2 \cdots \times_d \mathbf{H}_d(t), \qquad \mathbf{H}_i(t) = \mathbf{C}_i \operatorname{diag}\left( e^{-\mu_{i,n} t} \right)_{n} \mathbf{C}_i^{-1}, \qquad \mu_{i,n} = \left( \frac{n\pi}{L_i} \right)^{2}

This factorization holds for non-separable p0p_0 as well. That is, instead of integrating the dd-dimensional heat equation in time, the exact solution is obtained by dd one-dimensional transforms, one per axis, which is what the original sentence intended to convey. Accordingly, we have added the following cost analysis as Proposition C.6 (Appendix C.1.4). Smoothing costs O(Ng∑ini)O\left( N_g \sum_{i} n_i \right) with dense axis-wise transforms and O(Nglog⁡Ng)O\left( N_g \log N_g \right) with the fast DCT, and computing the score with axis-wise differentiation transforms has the same order of cost. Evaluation at NN collocation points by multilinear interpolation costs O(Nd⋅2d)O\left( N d \cdot 2^{d} \right), which is linear in NN.

The practical overhead is also small. Since our sampler has no learnable parameters, it reduces the time per epoch by 19% (from 45.2 s to 36.5 s) and the peak memory from 38 GB to 36 GB compared with PINN-ball under the same optimizer and number of iterations (Table 16). Nevertheless, since NgN_g grows exponentially with dd, we have stated explicitly in Sections 1 and 6 (Limitations) that the method targets low- to moderate-dimensional problems.

(Q1) Clarification of the Klein–Gordon benchmark configuration.

In our response to W1, we corrected the typographical error in the initial conditions and provided the complete configuration used in the experiments. The corrections appear in Eqs. (73) and (74) of Appendix E.3.

(Q2) Implementation of the diffusion sampler and its complexity for non-separable residual distributions.

In our response to W2, we explained that the sampler is implemented through an axis-wise operator factorization that remains valid for non-separable p0p_0, and we replaced the O(dN)O(dN) statement in the manuscript with an exact cost analysis (Proposition C.6 in Appendix C.1.4).

AuthorsOpenReview ↗

Follow-up on Author Rebuttal and Discussion

Dear Reviewer nZHX,

Thank you again for your careful review. As the discussion period progresses, we briefly summarize how we addressed your concerns.

  • Klein–Gordon initial conditions (W1, Q1). The inconsistency was a typographical error confined to the text of Appendix E.3. In all experiments, the source term and the initial/boundary targets were computed from the reference solution, so all methods were trained and evaluated under identical and correct conditions. We provided the full experimental configuration in our response and corrected the initial and boundary conditions in Eqs. (73)–(74) of the revised manuscript.
  • Complexity of diffusion sampling (W2, Q2). We agree that the original statement was imprecise for non-separable residual densities. We clarified that the separability of the kernel yields an axis-wise factorization of the heat-semigroup operator on a tensor grid (Eq. (51)), which remains exact for non-separable densities, and we replaced the O(dN) statement with an exact cost analysis (Proposition C.6, Appendix C.1.4): O(N_g log N_g) for smoothing and score computation with the fast DCT, and a cost linear in N for evaluation at the collocation points. In practice, the sampler reduces the time per epoch by 19% and the peak memory compared with PINN-ball (Table 16). We also stated the low- to moderate-dimensional scope explicitly in Sections 1 and 6.

We hope these responses resolve your concerns. If any point remains unclear, we would be glad to provide further clarification. If you find our responses satisfactory, we would be grateful if you would consider reflecting this in your evaluation.

Best regards, The Authors

Dear Reviewer nZHX,

Thank you again for your valuable feedback and constructive comments. We have carefully addressed your concerns and provided additional explanations and experimental results in our rebuttal.

As the discussion period is approaching its end, we would greatly appreciate it if you could take a moment to review our responses and share your thoughts on whether your concerns have been adequately addressed. If any issues remain, we would be happy to provide further clarification.

Thank you for your time and consideration.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Follow-up on Author Rebuttal and Discussion

Dear Area Chair, Thank you for overseeing the review process of our submission.

We have carefully addressed the reviewers’ concerns in our rebuttal, providing additional clarifications, supporting evidence, and experimental results where appropriate.

Reviewer nZHX

  • (W1, Q1) Inconsistent initial conditions in the Klein–Gordon benchmark. This was a typographical error confined to the text of Appendix E.3. In all experiments, the source term and the initial/boundary targets were computed from the reference solution, so all methods were trained and evaluated under identical, correct conditions. We provided the complete configuration and corrected the initial and boundary conditions (Eqs. (73)–(74)).
  • (W2, Q2) Complexity of diffusion sampling for non-separable residual distributions. We agreed that the original statement was imprecise and clarified that the separable kernel yields an axis-wise factorization of the heat-semigroup operator on a tensor grid (Eq. (51)), exact for non-separable densities. We added an exact cost analysis (Proposition C.6, Appendix C.1.4): O(N_g log N_g) with the fast DCT and linear in N for evaluation. The sampler also reduces the time per epoch by 19% and the peak memory relative to PINN-ball (Table 16).

Reviewer TRYR

  • (W1) Motivation for combining adaptive sampling with domain decomposition. (i) Adaptive sampling is well established for low-dimensional, low-regularity problems (RAR, RAD/RAR-D, R3, QR-DEIM, AAS), and PINN Balls also uses at most three inputs (Sections 1, 6, Appendix G.3). (ii) The components are structurally coupled: each expert is trained only on the points in its subdomain, so the sampler determines the data each expert receives, and the sampled covariance and residual gradient shape the subdomains (Eqs. (13)–(14)). (iii) In the 2x2 ablation (Table 3), combining both components further reduces the error on every variable (18.1–30.0% for velocity, 71.7–75.8% for pressure), with fewer subdomains needed (Fig. 2, Fig. 6).
  • (W2) Comparison with adaptive sampling baselines. Beyond the existing sampler-only comparisons (Table 3 against the AAS-based sampler of PINN-ball, 52.9–55.8% lower error; Table 10 against RAD, QR-DEIM, QR-DEIM-R), we added a comparison with AAS without decomposition (Table 17, Appendix G.11): our sampler alone lowers the errors of AAS by 17.6–37.1%, and 35.7–65.8% with the anisotropic decomposition.
  • (W3) Missing citation of AAS. We now cite AAS in Sections 1, 2.1, and 4.3, and added related work with a comparison table (Table 7, Appendix G.2), clarifying our differences: a parameter-free sampler built analytically by heat-semigroup smoothing and the probability-flow ODE, coupled with anisotropic decomposition.

Reviewer XCup

  • (W1) Training stability with respect to the number of subdomains. We added an experiment on Burgers with K in {4, 8, 16}, five seeds, and equal tuning budgets (Table 18, Fig. 13, Appendix G.12). PINN-ffusion attains lower mean error and standard deviation for every K; the error and variance of PINN-ball grow with K, whereas PINN-ffusion remains stable.
  • (W2) Scope and comparison with a classical solver. We clarified the scope as low- to moderate-dimensional spatiotemporal PDEs with low-regularity solutions (Appendix G.3) and listed (3+1)-dimensional and strongly coupled multiphysics problems as future work. We added a comparison with a high-order compact FD scheme on Burgers (Table 19, Appendix G.13), acknowledging that it is faster and more accurate for this forward problem, and clarified that our contribution is more accurate and stable PINN training, a prerequisite for inverse and parametric problems.
  • (Q1) Small and noisy S. Proposition C.7 (Appendix C.4.1) shows that, by trace normalization, the geometry is invariant to the magnitude of S and its change is bounded by the relative noise level. Errors vary only within 5.82–5.98e-3 as the anisotropy strength varies from 1 to 7 (Table 11).
  • (Q2) Constraints on anisotropic scaling, coverage, and derivative control. We specified the exact constraints and their values (Appendices F.2 and F.2.1) and reported the geometric quantities of the subdomains for each benchmark (Table 6), where the maximum axis ratio is at most 3.62 and the minimum overlap is positive. We also described how coverage is maintained and proved bounds on the derivatives of the partition weights in terms of the minimum semi-axis and the overlap lower bound (Proposition C.8, Appendix C.4.2).

As we have not yet received further feedback from the reviewers, we would greatly appreciate it if you could encourage them to revisit our responses and participate in the discussion before the discussion period closes.

We believe that our rebuttal addresses the main concerns raised during the review, and we would welcome any additional questions or clarifications from the reviewers or the Area Chair.

Thank you very much for your time and consideration.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.