acceptodds
Under review as a conference paper at ICLR 2027

Right Feature, Right Place: Residual-Driven Construction of Cauchy-Adaptive Physics-Informed Neural Networks

Abstract

Physics-informed neural networks (PINNs) solve partial differential equations by fitting a neural representation to the equation residual. Many solutions of scientific interest, from rogue waves and solitons to anomalous diffusion, carry isolated complex poles and algebraic tails, and at a finite size a PINN's accuracy depends on whether its neurons can represent this structure. Progress on PINN accuracy has centred on optimization, and adaptive methods tune features within one representation family chosen in advance. We introduce the Cauchy-Adaptive PINN (CAPINN), which lets the PDE residual choose each neuron's class, centre, and scale from a Cauchy-enriched dictionary of Cauchy, tanh-derived, and Gaussian neurons, building a compact basis over successive time intervals without solution labels. Standard PINN benchmarks favour tanh networks through their locally tanh-like fronts, so we evaluate CAPINN on a problem roster balanced for analytic structure. CAPINN is more accurate than vanilla PINNs and physics-informed Gaussians on every problem with poles or algebraic tails, and its basis adapts to each solution: Cauchy neurons place their poles along the withheld singularity trajectories, while smooth solutions are fitted with Gaussians alone. Front-dominated benchmarks remain the territory of tanh-based solvers. At a fixed neuron budget, how the representation is allocated is a lever complementary to optimization, and residual-driven allocation yields compact, interpretable solvers.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
physics-informed neural networks scientific machine learning PDE solvers frozen-feature networks rational approximation Cauchy activation representation selection interpretability

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_Z3D3OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Cauchy-Adaptive Physics-Informed Neural Networks (CAPINN) and studies a relatively underexplored limitation of PINNs. In addition to optimization difficulties, the choice of a fixed representation family can itself limit the accuracy of PDE solvers. The authors argue that different PDE solutions can exhibit distinct analytic structures, including complex poles, algebraic tails, localized smooth profiles, and sharp fronts. Under a limited neuron budget, selecting basis functions that match these structures can therefore be important for obtaining accurate and compact representations. CAPINN constructs the solution representation from a predefined analytic dictionary containing Cauchy, tanh-derived, and Gaussian basis functions. The PDE residual is used to greedily select the basis class, location, and scale during the solve. The paper demonstrates that this residual-guided selection can produce highly compact representations for several one-dimensional PDEs. The gains are particularly strong for problems with rational poles or algebraic tails, where Cauchy features closely match the underlying analytic structure.

Critical strengths
  1. The paper combines analytic structure with residual-guided basis construction in a simple and interpretable framework. The design is conceptually clean, and the selected basis functions provide a direct connection between the learned representation and the underlying structure of the PDE solution.

  2. The matched representation ablations and the inclusion of negative controls provide reasonably convincing evidence for the paper's focused claim that representation choice matters. In particular, the comparisons help isolate the effect of different basis families under comparable settings.

  3. The paper clearly presents the failure cases of Cauchy features and does not overstate their general applicability. This makes the scope of the method easier to assess and also motivates further mechanistic analysis of when different analytic representations are beneficial.

Critical weaknesses

The claimed balanced benchmark appears to introduce a reverse benchmark bias. The paper argues that conventional PINN benchmarks often favor tanh-based representations because many standard problems contain tanh-like fronts. However, the proposed benchmark roster is also strongly aligned with the inductive bias of CAPINN. The first five configurations, including two Peregrine settings, Benjamin–Ono, Voigt, and Michelson–Sivashinsky, contain isolated complex poles or algebraic tails. These are precisely the structures that Cauchy neurons are designed to represent. Only the remaining three configurations serve as controls without this strong alignment. This makes the benchmark balance somewhat questionable. The experiments convincingly show that Cauchy features work well on problems with Cauchy-like analytic structure. They provide weaker evidence that CAPINN is generally preferable as a representation-learning strategy across a broader class of PDEs.

The empirical evaluation is limited to relatively simple 1D PDEs. This makes it difficult to assess whether the proposed basis-selection strategy remains effective when the geometry, solution structure, and computational cost become substantially more complex. I would strongly encourage the authors to include at least one higher-dimensional benchmark. A problem with complexity comparable to a two-dimensional Navier–Stokes system would make the empirical claims considerably more convincing. Such an experiment would also help clarify whether residual-guided dictionary search remains computationally practical in higher dimensions.

The method does not automatically discover new feature families. CAPINN selects a representation from a manually specified dictionary: Cauchy, tanh and Gaussian. This is an important distinction. The PDE residual determines which candidate is selected, together with its centre and scale, but the candidate families themselves are provided by the designer.

If the true solution is better characterized by structures such as Airy, Bessel, log singularity and branch point, CAPINN cannot discover such a representation unless the corresponding family is already included in the dictionary. For this reason, claims suggesting that the residual alone discovers the appropriate representation or pole structure appear overstated. The search space already contains substantial prior knowledge about the expected solution structure. A more precise interpretation is that CAPINN performs adaptive model selection within a manually constructed hypothesis space.

The dictionary design introduces substantial prior information about the singularity structure. The training guide states that, for rational solutions, the width interval should extend down to the smallest pole distance from the real axis. This weakens the rhetoric in the abstract and conclusion that pole information is not provided and that the poles emerge purely from the PDE residual. It is true that the exact pole locations are not used as supervision. However, the method is still given the correct analytic family together with a scale range designed to cover the relevant singularity scale. These two statements should be clearly distinguished. The current experiments therefore demonstrate that the residual can locate useful Cauchy elements inside a carefully designed search space. They provide weaker evidence for genuine representation discovery. A stronger discovery experiment would use a substantially overcomplete dictionary containing many distractor families and a much broader range of scales. The authors could then test whether residual-based selection still consistently recovers the appropriate analytic structure. Another useful experiment would remove the Cauchy family and test whether the algorithm can approximate the same solutions through alternative rational families. Under the current setup, the method is better characterized as model selection among carefully curated hypotheses.

The methodological novelty appears more incremental than the title and contribution statements suggest. Several central components of CAPINN have close precedents. Frozen-PINN already studies frozen features, space-time separation, and coefficient fitting without standard gradient-based training. AG-RaNN uses the residual as an error indicator to adaptively grow randomized networks and constructs new neuron parameters during this process. IFeF-PINN separates basis construction from coefficient regression and provides convergence analysis for the linear setting. XNet directly introduces Cauchy activations and applies them to PDE solving. Given these connections, I believe the main novelty of CAPINN should be stated more narrowly as the specific combination of heterogeneous analytic dictionary + block-OLS residual scoring + local geometry refinement + nonlinear acceptance check.

This combination is meaningful and technically interesting. However, broader claims that CAPINN is the first method that allows the PDE itself to construct the representation appear too strong given the existing literature.

An important related baseline, AC-PKAN, is missing from the discussion and comparison. AC-PKAN [1] is closely related to the motivation of this paper. Both works directly address the representational limitations of standard PINNs and explore structured basis or activation functions under physics-informed objectives. AC-PKAN combines Chebyshev-based Kolmogorov-Arnold layers with learnable wavelet activations and residual-gradient attention to improve representational capacity and optimization. At minimum, I believe AC-PKAN should be discussed in the related work section. An empirical comparison would be even more informative if a compatible implementation is available.

A possible distinction could be stated as follows: Recent PINN architectures have also explored structured basis functions beyond standard MLP activations. AC-PKAN combines Chebyshev-based Kolmogorov-Arnold layers with learnable wavelet activations and residual-gradient attention to improve expressivity and optimization under physics-informed objectives. CAPINN differs in that it explicitly selects the analytic class, centre, and scale of individual frozen basis elements according to their predicted reduction of the PDE residual.

[1] Zhang et al. AC-PKAN: Attention-Enhanced and Chebyshev Polynomial-Based Physics-Informed Kolmogorov-Arnold Networks. TMLR, 2026.

Questions for authors

Could the authors provide a quantitative analysis of pole recovery in addition to the qualitative alignment shown in Figure 3? Figure 3 is interesting, but the current evidence is mainly visual and post-hoc. It would be useful to quantify how accurately the selected Cauchy neurons recover the underlying pole geometry. For example, the authors could report ∣ck−Re⁡z⋆(t)∣|c_k-\operatorname{Re} z^\star(t)| and ∣dk−∣Im⁡z⋆(t)∣∣.|d_k-|\operatorname{Im} z^\star(t)||. I would also encourage the authors to analyze how these errors change with the candidate grid resolution, time-slab resolution, neuron budget, and dictionary width range. This is particularly relevant for the Benjamin–Ono experiment. The paper notes that the selected widths form several bands, so the agreement in pole distance is only approximate. A quantitative sensitivity analysis would help clarify how robust the observed pole alignment is to the search design. If the alignment is highly sensitive to these design choices, I think the claim should be framed more conservatively as interpretable fitting behavior. A stronger claim of singularity discovery would require evidence that the recovered pole geometry remains stable across reasonable changes in the search space and discretization.


All in all, regarding the weaknesses and questions I raised, if they are not adequately addressed, I will lower my recommendation rating.

5 comments
AuthorsOpenReview ↗

Response 1/4

Thank you for pressing us to distinguish useful finite-budget representation construction from broader discovery claims. Your review motivated coupled 2D flow tests, experiments on the non-rational and oscillatory structures you named, bidirectional AC-PKAN comparisons, and quantitative pole-alignment diagnostics. While we substantially tightened the framing across Sections 1–5 to reflect these boundaries, the core CAPINN algorithm (Algorithm 1), the {Cauchy, tanh-derived, Gaussian}\{\text{Cauchy, tanh-derived, Gaussian}\} dictionary (§3.1, Eq. 1), and our primary 1D benchmark results (Tables 1–2) remain unchanged. The resulting evidence sharpens our contribution: residual-guided joint selection of family and geometry within a supplied dictionary, with explicit regime boundaries.

[W1] Benchmark composition, dropping the "balanced benchmark" claim, and cross-regime evaluation.

We agree with your critique that calling a five-of-eight pole-and-tail suite a "balanced benchmark" overstates its generality and introduces reverse benchmark bias. We removed the "balanced benchmark" characterization throughout Sections 1, 4.1, and 5, reframed the original 1D suite as a regime-diagnostic testbed—where smooth periodic heat selects only Gaussians (0/0/72 Cauchy/tanh-derived/Gaussian blocks, 3.23e-7 in Table 2), No C (the tanh-derived + Gaussian dictionary without Cauchy blocks) is more accurate than Mixed on 1D Allen–Cahn (2.24e-5 vs. 4.09e-5 in Appendix B.11), and non-adaptive Frozen-PINN is faster and more accurate on Burgers (3.43e-7 in 3.3 s in Tables 1, 10–11)—and added cross-regime validation across standard reaction/wave, coupled 2D fluid/wave, non-rational oscillatory, and 5D–100D radial PDEs:

  • Added cross-regime validation across 1D reaction/wave, coupled 2D flows, and non-rational ODEs (§4.1, Table 1; §4.3, Table 5 & Figure 4; Appendices D.1, D.3–D.5): Unchanged three-family CAPINN solves 1D-Reaction and 1D-Wave by adapting its family mixture and Gaussian width across time intervals (see [W6] below), combines Cauchy boundary-layer and Gaussian interior blocks on coupled 2D steady Navier–Stokes (Re=100,500\mathrm{Re}=100, 500) and tracks 2D rational lumps on (2+1)(2+1)D KP-I (see [W2] below), and solves Airy and Bessel ODEs from equation residuals without interior labels (see [W3] below).
  • Conditional family selection in 5D–100D radial cross-regime controls (§4.3, Table 6; Appendix D.5): Under the same construction procedure and block cap (K≤6K\le 6 isotropic radial blocks/interval evaluated on ambient coordinates), Cauchy access reduces nonlinear radial Allen–Cahn error relative to No C by 39.5× in 10D (3.04e-6 vs. 1.20e-4) and 110.5× in 50D (1.71e-5 vs. 1.89e-3), whereas No C matches Mixed on smooth 5D/10D Neumann heat (1.04e-6–1.14e-6) and all three 100D polynomial-heat seeds select zero Cauchy blocks (0/5/25 Cauchy/tanh-derived/Gaussian blocks, reaching 6.62e-15 ± 1.25e-16), confirming that Cauchy selection remains strictly regime-dependent in higher dimensions.
AuthorsOpenReview ↗

Response 2/4

[W2] Scaling beyond 1D: coupled 2D steady Navier–Stokes and time-dependent (2+1)(2+1)D KP-I.

You encouraged us to test whether residual-guided dictionary search remains effective and computationally practical on a coupled higher-dimensional fluid system such as 2D Navier–Stokes. Following your recommendation, we added coupled 2D steady incompressible Navier–Stokes on [0,1]2[0,1]^2 (solving simultaneously for velocity and pressure (u,v,p)(u,v,p) at Re=100,500\mathrm{Re}=100, 500) and time-dependent (2+1)(2+1)D Kadomtsev–Petviashvili (KP-I) lump propagation (§4.3, Table 5 & Figure 4; Appendix D.5, Tables 24–26):

  • Parameter-matched 2D PIG comparison on sine-lid cavity (Re=100\mathrm{Re}=100; Table 5, Figure 4a–d, & Table 24): Mixed CAPINN selects 61 Cauchy and 38 Gaussian blocks covering the cavity boundary layers and interior vortex, achieving 0.0228% full-vector velocity error on 1932193^2 Taylor–Hood FEM128 versus 0.1518% ± 0.0104% (3 seeds) for our local adaptation of 2D PIG (6.65× lower error at 1,290 vs. 1,291 total scalars including geometry on the same 97297^2 interior and 4×974\times 97 wall collocation grid).
  • Published COMSOL reference at Re=100\mathrm{Re}=100 and Re=500\mathrm{Re}=500 (Table 5 & Table 24): Mixed CAPINN reaches 0.0279% speed error at Re=100\mathrm{Re}=100 (vs. 0.0404% for published PIG and 0.122% for PGCAN) and 1.45% speed error at Re=500\mathrm{Re}=500 (comparable to PGCAN's 1.42% and 1.6×–4.1× below single-family Cauchy 2.36%, Gaussian 2.52%, and validation-tuned Fixed Gaussian 5.94%).
  • Singular unit-lid cavity (Re=100\mathrm{Re}=100; Table 5 & Table 25): On the singular unit-lid cavity (ulid=1u_{\mathrm{lid}}=1) evaluated without corner cropping, three Sobol candidate-pool seeds reach 2.83% ± 0.07% (Cauchy) and 3.49% ± 0.66% (Mixed, 0.22% ± 0.03% Ghia centerline RMS) versus 8.93% for validation-tuned Fixed Gaussian.
  • Time-dependent (2+1)(2+1)D KP-I lump propagation (Table 26): Across 10 time intervals, radial Cauchy blocks reach 0.335% ± 0.018% full-time error versus 5.30% ± 1.62% for radial Gaussian (15.8× lower error at K≤32K\le 32 blocks/interval) and 2.78% ± 1.23% for static initial-geometry Cauchy (8.3× lower error from per-interval basis renewal).

[W3] Representing Airy, Bessel, and regularized log/branch profiles within a supplied dictionary.

We agree that CAPINN selects and refines atoms within a designer-supplied dictionary; it does not invent new symbolic families (§1, §3.1, §5). The distinction is between discovering a new family and constructing an accurate representation of a solution: the latter need not require an atom with the same functional form. Appendix D.4 tests this distinction in two complementary ways:

  • Residual-guided Airy/Bessel solves (Appendix D.4, Figure 17; Tables 22–23): Without adding Airy or Bessel functions, the original Cauchy/tanh-derived/Gaussian dictionary (CTG) reaches relative L2L^2 errors of 1.46e-8 on the Airy initial-value problem and 4.96e-8 on the unit-weighted Bessel problem, using equation residuals and prescribed initial/boundary data rather than interior solution labels. Explicitly adding paired Airy and Bessel blocks (+Ai,J+\mathrm{Ai},J) further reduces turning-point Airy error from 6.93e-6 to 2.49e-10 (and radial Helmholtz error from 5.95e-5 to 6.03e-14), showing the benefit of richer supplied building blocks.
  • Known-profile approximation (Appendix D.4, Table 21): On regularized logarithmic log⁡(x2+ε2)\log(x^2+\varepsilon^2) and fractional-power (x2+ε2)1/2,(x2+ε2)1/4(x^2+\varepsilon^2)^{1/2}, (x^2+\varepsilon^2)^{1/4} profiles on [−1,1][-1,1] (ε=0.02\varepsilon=0.02), 33-column (K=32K=32) Cauchy spans attain 1.02e-9–5.75e-9 discrete least-squares fitting error with widths in [0.002,20][0.002,20]. No C is more accurate on the narrower width interval [0.002,1][0.002,1], demonstrating scale dependence. These experiments fit known, regularized functions; they do not solve PDEs or test approximation of functions with singularities on the real axis.

Appendix D.4 thus separates representation capacity, residual-guided construction, and explicit dictionary enrichment. Together, the results establish useful approximation beyond the individual forms of the supplied atoms, without claiming symbolic family discovery.

AuthorsOpenReview ↗

Response 3/4

[W4] Distinguishing dictionary and scale-range priors from withheld pole coordinates.

You pointed out an important tension in our initial draft: our original Appendix B note advised setting the minimum width down to "the smallest pole distance from the real axis," which made it appear that CAPINN relied on problem-specific pole-depth priors. For a Cauchy feature 1/((x−c)2+d2)1/((x-c)^2+d^2) evaluated on the real axis, dd is simply its half-width at half-height (Appendix A.1), so that note was an imprecise description of a spatial resolution floor rather than an injection of analytical pole coordinates. Prompted by your critique, we systematically re-examined our width-bracket sensitivity and found that problem-tailored width brackets are largely unnecessary: a single mesh-derived rule [wmin⁡,wmax⁡]=[2Δx,L][w_{\min}, w_{\max}] = [2\Delta x, L] without pole-depth input preserves comparable accuracy on most of the eight problems, while also revealing width-density-sensitive exceptions. We therefore removed the confusing pole-distance note and replaced it with this quantitative sensitivity study in Appendix B.2 (Figure 5 & Table 9), while stating in Sections 1 and 5 that the candidate dictionary itself remains a supplied hypothesis space:

  • Uniform mesh rule [wmin⁡,wmax⁡]=[2Δx,L][w_{\min}, w_{\max}] = [2\Delta x, L] (Appendix B.2, Table 9): Setting the width bracket purely from the spatial grid spacing Δx=2L/nx\Delta x = 2L/n_x and domain half-width L=∣Ix∣/2L=|I_x|/2 stays within 0.58×–1.51× of the Table 8 brackets on seven of eight PDEs at nw=11n_w=11 (improving accuracy on four of eight, including 6.09e-6 vs. 6.74e-6 on Voigt and 2.70e-6 vs. 3.91e-6 on Burgers) and within 0.82×–1.45× on seven of eight at nw=7n_w=7, while documenting larger width-density sensitivity on Michelson–Sivashinsky (1.71e-3 at nw=7n_w=7 vs. 8.56e-3 at nw=11n_w=11) and Allen–Cahn (8.25e-4 at nw=7n_w=7 vs. 6.12e-5 at nw=11n_w=11).
  • Broad scale-bracket expansion (Appendix B.2, Figure 5f): Widening [wmin⁡,wmax⁡][w_{\min}, w_{\max}] up to a 512×512\times span keeps relative L2L^2 error in 3.5e-7–5.3e-7 on Benjamin–Ono and wide Peregrine as long as wmin⁡w_{\min} resolves the finest feature scale.

[W5] Positioning relative to frozen-feature, greedy, and Cauchy precedents, and the viewpoint motivating our construction.

We agree that the individual building blocks have close precedents (Frozen-PINN, AG-RaNN, IFeF-PINN, XNet, and OMP/OLS/greedy PDE solvers such as Siegel et al., 2023), and we revised Sections 1–3 and Appendix A.1–A.3 to credit each work explicitly without any "first-ever" claim for residual-guided construction:

  • Motivation, novelty, and significance ("right feature, right place" at finite budget; §1, §2, Appendix A.1–A.3): Although Gaussian, tanh, and Cauchy networks each enjoy universal approximation and favorable theoretical approximation rates (Mhaskar, 1996; Li et al., 2025a,b), those results quantify approximation potential while leaving the task of finding useful features to the solver (Appendix A.3). A single PDE solution can combine smooth regions, sharp transitions, and localized rational peaks or power-law tails whose locations, scales, and relative prominence evolve over time: feature shape governs how compactly a basis captures each local structure, yet a dictionary of suitable functions does not specify which ones to use or where to place them without interior solution labels (§1). Existing methods adapt parts of this representation separately—through single-family frozen-feature renewal, end-to-end adaptive activations or Gaussian centers, or greedy basis expansion. Our contribution is studying feature-family choice and geometric placement together within a constructive frozen-feature framework (§1, §2, Appendix A): using the PDE residual to jointly select family, location, and scale across {Cauchy, tanh-derived, Gaussian}\{\text{Cauchy, tanh-derived, Gaussian}\} blocks and renew the active representation across time intervals so a candidate dictionary is turned into a compact PDE basis during the solve.
  • Controlled evidence that family choice and geometric placement are both necessary (§4.2, Tables 2–3; Appendix D.2, Table 17): Even though each single family is a universal approximator, combining all three families in Table 2 improves Voigt by 7.7× over the best single family (6.74e-6 vs. 5.19e-5 for Cauchy-only and 2.65e-4 for tanh-only), lowers rational-wave error by 1,404×–4,900× over No C, and selects pure Gaussians on periodic heat (88× below Cauchy-only). Conversely, holding the {Cauchy, tanh-derived, Gaussian}\{\text{Cauchy, tanh-derived, Gaussian}\} dictionary, Gauss–Newton refitting, and the same coarse entrance-initialization procedure fixed at K=5K=5 blocks/interval in Table 3 (and Table 17), five-seed random placement leaves 0.475 ± 0.011 error on wide Peregrine and 1.01 ± 0.033 on Benjamin–Ono, whereas discrete Block-OLS selection lowers error to 1.16e-3 and 2.91e-2, and four-round local (c,d)(c,d) refinement lowers it to 1.46e-6 and 8.52e-6.
Reviewer_1aAjOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes the Cauchy-Adaptive Physics-Informed Neural Network (CAPINN) which includes the Cauchy kernels in equation (3) into the basis/neuron bank to represent the PDE solution. CAPINN adds a basis/neuron if the basis/neuron is estimated to provide largest residual reduction; then CAPINN fits the coefficients by Gauss-Newton updates. Inclusion of Cauchy kernels is shown to capture the solution with poles or algebraic tails better than other representation-focused methods such as Frozen-PINN and PIG. CAPINN is evaluated across 8 configures of 7 one-dimensional equations and outperforms baselines on most of the test instances. Qualitative study on solution errors is provided to demonstrate Cauchy kernels help to capture the poles and algebraically decaying tails.

Critical strengths
  1. Novel residual-driven adaptive construction on representation family with Cauchy kernels. The paper proposes a novel residual-guided procedure to tackle the neuron selection issues in representation fitting. And their numerical experiment shows that smooth functions are fitted with Gaussian alone, meaning the proposed residual-guided procedure is adaptive to richer classes of neurons and not bias to Cauchy kernels.

  2. CAPINN demonstrates strong numerical results and captures poles and algebraically decaying tails. The evaluation is conducted across a long list of PDEs and consistently outperforms the baselines. On the configurations with poles or algebraic tails, CAPINN either beats the best known published results or provides the first error results. Case study also include qualitative solution error analysis showcasing the effect of Cauchy kernels to capture poles and algebraically decaying tails.

Critical weaknesses
  1. Presentation, organization, and clarity. The current presentation of the paper is difficult to follow, especially for audience not familiar with the adaptive feature fitted approach. Section 3.2 is the main contribution of the paper but is broken down to several separated and not coherent pieces. I would recommend the authors to explain their main methods following the flow chart in Figure 1, so that the reader can follow more easily. The chronological order of the procedure also helps clarity.
Questions for authors
  1. The work does not claim the introduction of Cauchy kernels as one of the contribution and mentions other works related to Cauchy activations and XNet. Can you clarify the novelty relative to XNet and Cauchy-basis network?

  2. Most of PINN approaches exhibits variability caused by weight initialization, optimization, and/or collocation sampling. Can you clarify whether CAPINN is a deterministic approach or is there any randomness that could occur even with fixed configuration? It would be better if the authors can provide multiple-seed baseline results and comment on the statistical robustness of CAPINN.

  3. How sensitive is CAPINN to the dictionary and selection hyperparameters such as center-grid resolution, width range, acceptance threshold etc? How much prior knowledge is needed to specify these ranges to include enough and/or helpful basis functions in the neuron bank? Can you clarify how do you select these parameters in practice?

3 comments
AuthorsOpenReview ↗

Response 1/2

Thank you for your constructive review and for highlighting the adaptive multi-family behavior on periodic heat. Following your suggestions, we reorganized Section 3.2 chronologically and added fixed-budget construction controls, determinism and multi-seed variability records, and an 89-configuration hyperparameter sensitivity study. Because these revisions touched much of the exposition in Sections 1–3, we note that the underlying CAPINN representation (§3.1, Eq. 1), construction algorithm (Algorithm 1), and 1D benchmark conclusions (Tables 1–2) remain unchanged.

[W1] Reorganizing Section 3.2 in chronological order following Figure 1.

We agree that Section 3.2 in the initial submission broke the construction procedure into disconnected pieces that were hard to follow. Following your recommendation, we reorganized Section 3.2 into five numbered chronological steps matching the exact execution flow of Figure 1 and Algorithm 1, immediately after Section 3.1 (Eq. 1) defines the feature blocks and Chebyshev time coefficients:

(1) Entrance initialization →\to (2) Coarse candidate scoring →\to (3) Local coordinate refinement →\to (4) Coefficient refitting and acceptance →\to (5) Validation and propagation (§3.2, Eqs. 2–4).

We also documented our parallel float64 GPU implementation of Steps 2–3, which accelerates candidate evaluation across the eight-problem 1D suite by 37.4× (reducing total suite fitting time from 17,759.4 s on single-thread CPU to 474.3 s on GPU) while preserving the algorithm's behavior (§3.2, §4.1, and Appendix B.3, Figure 6 & Table 10).


[Q1] Novelty relative to XNet and Cauchy-basis networks.

Thank you for asking us to sharpen our positioning relative to XNet (Li et al., 2025a,b). We build on the Cauchy activation family and approximation results introduced by XNet (§1, §2, and Appendix A.3), while our revised Introduction (§1) motivates a distinct constructive problem for PDEs: because a single PDE field can combine smooth regions, sharp transitions, and rational peaks or tails whose locations and scales evolve over time, feature family and geometric placement are complementary rather than interchangeable ("right feature, right place"). Rather than training a single Cauchy activation family end-to-end by gradient descent, CAPINN uses the PDE residual to jointly select feature family, center cc, and width dd across {Cauchy, tanh-derived, Gaussian}\{\text{Cauchy, tanh-derived, Gaussian}\} blocks—scoring candidates orthogonal to the current basis via Block-OLS (Step 2), locally refining (c,d)(c,d) coordinates within each family before comparing families (Step 3), freezing accepted geometry during second-order Gauss–Newton coefficient refits (Step 4), and renewing the basis across time intervals (Step 5; §3.1–3.2). The complementary roles of multi-family dictionary choice (including selecting Gaussians alone on smooth periodic heat, as you noted in Strength 1) and residual-guided placement are isolated in §4.2 (Tables 2–3) and Appendix D.2 (Table 17).


[Q2] Determinism of CAPINN at fixed configuration and multi-seed statistical robustness.

You asked whether CAPINN is deterministic at a fixed configuration or subject to initialization and sampling randomness like standard PINNs, and requested multi-seed baseline robustness statistics. In short: the 1D CAPINN solver uses fixed collocation grids, deterministic candidate pools, and deterministic local coordinate refinement without random weight initialization or stochastic minibatch sampling; for the stochastic neural baselines, we added three- to five-seed statistics in Appendix D.1:

  • 1D determinism and cross-backend agreement (§3.2, Reproducibility Statement, & Appendix B.3, Table 10): On CPU, eight-worker parallel scoring (8W+cache) produces bitwise-identical saved geometry and coefficients to single-thread CPU across all 199 intervals of the eight-problem suite. On float64 GPU (which pre-screens a top-32 Kronecker–Gram shortlist per family before exact QR projection), the solver selects the identical block sequence across all 127 intervals of conventional and wide Peregrine, Benjamin–Ono, Voigt, and periodic heat at relative tolerance 10−510^{-5} (maximum relative L2L^2 difference 0.0033%), while the shortlist shifts the discrete selection trajectory slightly on Michelson–Sivashinsky, Burgers, and Allen–Cahn (reaching 0.67×–1.05× of the CPU error).
  • Multi-seed neural baselines (Appendix D.1, Tables 14–16, mean ± sample SD): Across the stochastic neural baselines, three-seed evaluations give 0.430 ± 0.037 for PINN and 0.431 ± 0.052 for resampled PIG on wide Peregrine, 2.98e-4 ± 6.2e-5 for physics-validated PINN on Burgers, and 1.62e-3 ± 1.8e-4 for PINN + RAD on Benjamin–Ono (Table 14), while five-seed Frozen-PINN gives 1.66e-4 ± 2.0e-5 on 1D-Reaction and 6.23e-3 ± 1.7e-6 on repaired periodic Allen–Cahn (Tables 15–16).
AuthorsOpenReview ↗

Response 2/2

[Q3] Sensitivity to dictionary hyperparameters, prior knowledge needed, and practical selection.

Thank you for raising this practical question on how sensitive CAPINN is to its dictionary and selection parameters—coarse center count ncn_c, candidate width range [wmin⁡,wmax⁡][w_{\min}, w_{\max}], relative loss-acceptance threshold η\eta (which accepts a refitted block only when Lnew≤ηLprevL_{\text{new}} \le \eta L_{\text{prev}}), and per-interval block cap Kmax⁡K_{\max}—how much prior knowledge is needed to set them, and how to choose them on a new PDE. Across the swept center densities ncn_c, acceptance thresholds η\eta, block caps Kmax⁡K_{\max}, and width expansions on Benjamin–Ono and wide Peregrine, relative L2L^2 error stays in the 10−710^{-7} range because the acceptance test stops growth automatically, while the width bracket can be initialized directly from the spatial mesh via [wmin⁡,wmax⁡]=[2Δx,L][w_{\min}, w_{\max}] = [2\Delta x, L] without prior knowledge of the solution's pole scales (Appendix B.2, Figure 5 & Table 9):

  • Sensitivity and self-limiting size (Appendix B.2, Figure 5c–f): Across our 89-configuration GPU sensitivity study (3,222 intervals), both Benjamin–Ono and wide Peregrine remain in the 10−710^{-7} error range across all swept coarse center counts (nc∈[17,129]n_c\in[17,129]; Panel c), acceptance thresholds (η∈[0.990,0.999]\eta\in[0.990,0.999]; Panel d), block caps (Kmax⁡∈[5,16]K_{\max}\in[5,16]; Panel e), and width-bracket expansions up to a 512×512\times span (3.5e-7–5.3e-7; Panel f). Because Step 4 halts once a candidate fails the η=0.995\eta=0.995 acceptance gate, Benjamin–Ono retains 5.00 blocks (77 linear coefficients) per interval across all seven caps Kmax⁡∈[5,16]K_{\max}\in[5,16], and wide Peregrine plateaus at 4.79 mean blocks (148.2 coefficients) from cap 8 through 14 (with 0 validation-pruned intervals across all 13 runs).
  • Mesh-based width range and configuration-sensitive cases (Appendix B.2, Table 9 & Figure 5a,b): Setting [wmin⁡,wmax⁡]=[2Δx,L][w_{\min}, w_{\max}] = [2\Delta x, L] purely from the grid spacing Δx=2L/nx\Delta x = 2L/n_x and domain half-width L=∣Ix∣/2L=|I_x|/2 stays within 0.58×–1.51× of the Table 8 brackets on seven of eight problems at nw=11n_w=11 (improving accuracy on four of eight, e.g., 6.09e-6 vs. 6.74e-6 on Voigt and 2.70e-6 vs. 3.91e-6 on Burgers, with 5.60e-7 on Benjamin–Ono and 6.10e-7 on wide Peregrine) and within 0.82×–1.45× on seven of eight at nw=7n_w=7. At the same time, width grid density affects stiff fronts on Michelson–Sivashinsky (1.71e-3 at nw=7n_w=7 vs. 8.56e-3 at nw=11n_w=11) and Allen–Cahn (8.25e-4 at nw=7n_w=7 vs. 6.12e-5 at nw=11n_w=11; Table 9), wmin⁡w_{\min} must resolve the finest spatial scale, and time-interval count NN and entrance weight λe\lambda_e matter on rapid transients and stiff fronts (Figure 5a,b).
  • Practical selection on a new PDE: Initialize widths from the mesh via [2Δx,L][2\Delta x, L] (nw∈{7,11}n_w\in\{7,11\}), uniform coarse centers, η=0.995\eta=0.995, and four local refinement rounds, then check two label-free diagnostics (Figure 5b,e): if retained block growth hits Kmax⁡K_{\max}, raise the cap; if the held-out off-grid PDE residual RMS remains large, refine the time-interval count NN, spatial grid nxn_x, or entrance weight λe\lambda_e.

The reviewer thanks the authors for detailed response. My concerns have been solved. I keep my positive rating and would be willing to support publication as long as authors revise their manuscript as they promised to.

Reviewer_USD6OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The authors argue that, at a small neuron budget, PINN accuracy depends on whether the neurons can represent the analytic structure of the solution, such as isolated complex poles and algebraic tails. They propose the Cauchy-Adaptive PINN (CAPINN), which builds a compact frozen-feature basis from a dictionary of Cauchy, tanh-derived, and Gaussian neurons. The PDE residual (via a block orthogonal-least-squares score) chooses each neuron's class, centre, and scale, while time-dependent coefficients are refitted over successive time slabs without solution labels. CAPINN is evaluated on several one-dimensional PDEs balanced for analytic structure (Peregrine rogue wave, Benjamin-Ono, Voigt, Michelson-Sivashinsky, heat, Burgers, Allen-Cahn), along with a two-dimensional heavy-tailed flow-matching experiment. The empirical evaluations show that CAPINN is more accurate than vanilla PINNs and physics-informed Gaussians (PIG) on problems with poles or algebraic tails, while front-dominated problems remain better served by tanh-based solvers such as Frozen-PINN.

Critical strengths
  1. The idea of matching the analytic form of each neuron to the structure of the solution is clear and well-motivated, and the Benjamin-Ono example illustrates it nicely.
  2. The Cauchy features are grounded in mathematics (conjugate pole pairs, compatibility with the Hilbert transform), and the residual-based selection is correctly attributed to orthogonal least squares.
  3. The representation ablations are convincing: removing Cauchy neurons degrades Benjamin-Ono by more than three orders of magnitude, and mixing neuron families helps on Voigt. The negative controls (heat selects only Gaussians; Allen-Cahn is better without Cauchy) are informative.
  4. I like that the paper is honest about where the method does not win, such as front-dominated problems, and discloses hardware, tuning, and baseline failures.
  5. The learned representation is interpretable, as the selected Cauchy poles track the analytic singularity trajectories.
Critical weaknesses
  1. My main concern is the experimental comparison, which confounds representation quality with sampling and optimization choices. CAPINN is tuned per problem, while vanilla PINN and PIG use untuned source recipes (PIG on fixed collocation points), mostly with a single seed. Moreover, adaptive collocation and quadrature methods, e.g., RAR-G/RAD/RAR-D [1], R3 [2], and LearnQuad [3], are neither cited nor compared, which is surprising given that the paper itself discusses weak baselines in machine learning for PDEs. Can author(s) include matched-budget comparisons with adaptive sampling methods, and run the baselines with symmetric tuning and multiple seeds?
  2. The benchmark favors the method's inductive bias (five of eight configurations have poles or tails, and all are one-dimensional in space), while standard benchmarks such as PINNacle and PDEBench are discussed but not used. This makes it hard to assess general applicability, including in higher dimensions.
  3. The ablations do not separate the selection rule from the dictionary; a fixed or random Cauchy-enriched basis with the same fitting is missing. Given prior greedy PDE solvers [4] (not cited) and adaptive activation blending for PINNs, the novelty lies mainly in the analytic dictionary.
  4. The computational cost is high and not compared in a controlled way (e.g., 9,116 s on Burgers vs. 3.3 s for Frozen-PINN, on a single CPU thread), and it is unclear how the dictionary search scales to 2D/3D problems.
  5. On the nonlocal problems, Cauchy features use analytic operator responses while the other families and baselines use finite domains or FFTs, so the comparisons do not isolate approximation quality. Also, the Allen-Cahn Frozen-PINN adaptation lacks a constant mode, so its 2.07 error reflects an invalid adaptation rather than a meaningful baseline.

[1] C. Wu, M. Zhu, Q. Tan, Y. Kartha, and L. Lu. A Comprehensive Study of Non-adaptive and Residual-based Adaptive Sampling for Physics-Informed Neural Networks. Computer Methods in Applied Mechanics and Engineering, 403:115671, 2023.

[2] A. Daw, J. Bu, S. Wang, P. Perdikaris, and A. Karpatne. Mitigating Propagation Failures in Physics-Informed Neural Networks using Retain-Resample-Release (R3) Sampling. ICML, 2023.

[3] S. Pal, K. Azizzadenesheli, and V. Singh. PINNs with Learnable Quadrature. NeurIPS, 2025.

[4] J. W. Siegel, Q. Hong, X. Jin, W. Hao, and J. Xu. Greedy Training Algorithms for Neural Networks and Applications to PDEs. Journal of Computational Physics, 484:112084, 2023.

Questions for authors
  1. Can authors compare against a fixed or randomly sampled Cauchy-enriched basis that uses the same coefficient budget, time slabs, and refitting?
  2. How do the baselines perform with resampled collocation, a symmetric tuning budget, and several seeds?
  3. How does the dictionary search scale to higher-dimensional PDEs?
  4. How sensitive are the results to the number of time slabs, and the entrance weights?
  5. Can authors include wall-clock comparisons with the baselines?
  6. Can authors report a tail-restricted error to directly test the claimed advantage on algebraic tails?

Any discussion around these will help improve the paper over its current state.

4 comments
AuthorsOpenReview ↗

Response 1/3

Thank you for your detailed review and constructive experimental suggestions. Your questions motivated multi-seed baseline and adaptive-sampling controls, fixed-budget construction and shared-FFT operator ablations, 1D-Reaction/Wave and 2D–100D benchmarks, and parallel GPU timing and sensitivity studies. While these additions and expository updates substantially expanded our empirical controls, the core CAPINN algorithm (Algorithm 1), the three-family {Cauchy, tanh-derived, Gaussian}\{\text{Cauchy, tanh-derived, Gaussian}\} dictionary (§3.1, Eq. 1), and our primary 1D benchmark results (Tables 1–2) remain unchanged.

[W1/Q2] Baseline performance under physics-validated tuning, multi-seed evaluation, and adaptive sampling (RAD/RAR-D).

We agree that comparing CAPINN against single-seed default baselines can confound representation quality with sampling and tuning choices. We added physics-validated PINN and PIG hyperparameter searches on Burgers and wide Peregrine, three-seed baseline evaluations across the suite, periodic PIG collocation resampling, and temporal residual-based adaptive distribution (RAD) and refinement with distribution (RAR-D) comparisons (§2, §4.1, and Appendix D.1, Table 14). Tuning and adaptive sampling meaningfully improve the baselines, but a three-to-six-order-of-magnitude accuracy gap remains on rational waves:

  • Physics-validated baseline tuning (Burgers, 600 s CPU cap): Selecting the hyperparameter configuration that minimizes the held-out physics-validation residual (evaluated on independent interior, initial, and boundary collocation points without interior solution labels) lowers three-seed mean relative L2L^2 error from 6.16e-4 to 2.98e-4 ± 6.2e-5 for PINN and from 0.274 to 7.45e-2 ± 8.0e-3 for resampled PIG.
  • Adaptive sampling on Benjamin–Ono (3,600 s GPU cap): Temporal PINN + RAD reaches 1.62e-3 ± 1.8e-4 (1,690 s) and PINN + RAR-D reaches 3.59e-3 ± 2.8e-4 (298 s; resampled PIG: 0.104 ± 0.028, uniform 600 s GPU PINN: 5.20e-3 ± 2.0e-3), versus 4.22e-7 (29.3 s) for GPU CAPINN.
  • Wide Peregrine (1,200 s GPU cap): Three-seed PINN (0.430 ± 0.037), RAD (0.443 ± 0.025), RAR-D (0.496 ± 0.0033), and resampled PIG (0.431 ± 0.052) plateau at or above 0.43, versus 4.23e-7 (18.8 s) for CAPINN.

See Appendix A.1 (Eq. 10) for the tail-decay analysis of finite exponential bases, Appendix D.1 (Table 14) for the validation protocol and full three-seed baseline and adaptive-sampling results, and Appendix B.3–B.4 (Tables 10–11) for paired timings.

[W2/Q3] Generalization to PINNacle/AC-PKAN benchmarks and scaling to 2D–100D PDEs.

We appreciate your concern that an eight-configuration 1D suite weighted toward poles and tails makes it hard to assess broader applicability and higher-dimensional scaling. We expanded the evaluation to 1D-Reaction and 1D-Wave from PINNacle/AC-PKAN, coupled 2D flows, and structured 5D–100D radial PDEs, showing that residual-guided selection adapts across both coupled 2D spatial structures and high ambient dimensions:

  • Coupled 2D Navier–Stokes & (2+1)(2+1)D KP-I (§4.3, Table 5 & Figure 4; Appendix D.5, Tables 24–26):
    • Sine-lid Navier–Stokes (Re=100,500\mathrm{Re}=100, 500): Mixed CAPINN (selecting 61 Cauchy and 38 Gaussian blocks; Figure 4a–d) achieves 0.0228% full-vector velocity error vs. 0.1518% ± 0.0104% for parameter-matched 2D PIG (6.65× lower error at 1,290 vs. 1,291 total scalars on the same 97297^2 collocation grid), 0.0279% COMSOL speed error at Re=100\mathrm{Re}=100, and 1.45% at Re=500\mathrm{Re}=500 (matching PGCAN's 1.42% and beating validation-tuned Fixed Gaussian 5.94%; Table 24).
    • Singular unit-lid Navier–Stokes (Re=100\mathrm{Re}=100; Table 5 & Table 25): On the uncropped unit lid (ulid=1u_{\mathrm{lid}}=1), three Sobol candidate-pool seeds reach 2.83% ± 0.07% (Cauchy) and 3.49% ± 0.66% (Mixed, 0.22% ± 0.03% Ghia centerline RMS) vs. 8.93% for validation-tuned Fixed Gaussian.
    • Time-dependent (2+1)(2+1)D KP-I lump propagation (Table 26): Across 10 time intervals, radial Cauchy reaches 0.335% ± 0.018% full-time error vs. 5.30% ± 1.62% for radial Gaussian (15.8× lower error at K≤32K\le 32 blocks/interval) and 2.78% ± 1.23% for static initial-geometry Cauchy (8.3×).
  • Structured 5D–100D radial PDEs (§4.3, Table 6; Appendix D.5): Evaluating isotropic radial blocks r=∥x−c∥2r=\|\mathbf{x}-\mathbf{c}\|_2 and their ambient dd-dimensional PDE derivatives directly on coordinates x∈Rd\mathbf{x}\in\mathbb{R}^d (K≤6K\le 6 blocks/interval, avoiding a Cartesian grid over Rd\mathbb{R}^d; general anisotropic high-D geometries remain open in §5), Cauchy access lowers nonlinear Allen–Cahn error relative to No C (the tanh-derived + Gaussian dictionary without Cauchy blocks) by 39.5× in 10D (3.04e-6 vs. 1.20e-4) and 110.5× in 50D (1.71e-5 vs. 1.89e-3 in 1.4–2.8 s on GPU).
AuthorsOpenReview ↗

Response 2/3

[W2/Q3 continued]

It matches No C on smooth 5D/10D PINNacle Neumann heat HNd (1.04e-6–1.14e-6), and selects zero Cauchy blocks (0/5/25 Cauchy/tanh-derived/Gaussian blocks, reaching 6.62e-15 ± 1.25e-16 in 5.4 s) on 100D polynomial heat.

  • 1D-Reaction & 1D-Wave (§4.1, Table 1; Appendix D.1, Table 15 & Appendix D.3): Unchanged CAPINN reaches 6.07e-5 (7.2 s GPU) on 1D-Reaction and 2.08e-6 (11.3 s GPU) on 1D-Wave, renewing both its family mixture (19/47/93 line and 42/50/65 periodic Cauchy/tanh-derived/Gaussian blocks) and median Gaussian width (broadening from 0.56 on the initial pulse to 2.46 across the late plateau) across time intervals on 1D-Reaction.

[W3/Q1] Isolating the selection rule from the Cauchy dictionary at fixed budget.

You pointed out that our original ablations did not separate the selection rule from the dictionary, noting prior greedy PDE solvers (Siegel et al., 2023) and asking for a comparison against a fixed or randomly sampled Cauchy-enriched basis using the same coefficient budget, time intervals, and refitting. Holding the {Cauchy, tanh-derived, Gaussian}\{\text{Cauchy, tanh-derived, Gaussian}\} candidate pool, time intervals, the same coarse entrance-initialization procedure, and frozen-geometry Gauss–Newton refitting fixed at an exact retained budget KK (without post-growth validation prefix pruning) shows that dictionary access alone leaves order-one errors (0.11–1.01), whereas orthogonal Block-OLS selection and local (c,d)(c,d) coordinate refinement reduce error by four to five orders of magnitude:

  • Benjamin–Ono (K=5K=5 blocks, 77 linear coefficients/interval): Error drops from 0.929 (fixed pool) and 1.01 ± 0.033 (5-seed random) to 2.91e-2 with coarse Block-OLS selection (35×) and 8.52e-6 with four-round local (c,d)(c,d) refinement (a further 3,414×; 1.18e5× total gain over random).
  • Wide Peregrine (K=5K=5 blocks, 154 coefficients/interval): Error drops from 0.469 (fixed) and 0.475 ± 0.011 (random) to 1.16e-3 with Block-OLS (410×) and 1.46e-6 with local refinement (a further 790×; 3.24e5× total gain).
  • Voigt (K=13K=13 blocks, 189 coefficients/interval): Error drops from 0.123 (fixed) and 0.114 ± 0.019 (random) to 3.19e-5 with Block-OLS (3,584×) and 2.60e-5 with refinement (1.23×, as its x=0x=0 center already lies on the coarse grid).

Held-out off-grid PDE residual RMS drops in tandem to 1.00e-6 on Benjamin–Ono and 7.05e-6 on wide Peregrine. Section 2 cites Siegel et al. (2023) alongside the OMP/OLS literature, Appendix A.2 (Eq. 11) gives the linear Block-OLS projection identity, and §4.2 (Table 3) and Appendix D.2 (Table 17) report the full construction and simultaneous-random roster.

[W4/Q5] Computational cost and controlled wall-clock comparisons with baselines.

You pointed out that our reported single-thread CPU cost was high (e.g., on Burgers vs. 3.3 s for Frozen-PINN) and asked for controlled wall-clock comparisons with the baselines. Your question motivated us to profile where that single-thread CPU time was spent: 83.6%–99.4% goes to independent candidate evaluation, Block-OLS scoring, and local coordinate refinement, whereas sequential Gauss–Newton coefficient refitting takes only 0.5%–15.8%. Evaluating candidates in parallel on the same float64 GPU (RTX 4080 SUPER) used for the neural baselines reduces eight-problem suite runtime by 37.4× without altering the algorithm:

  • Suite & representative speedups: Total fitting time across the eight original 1D configurations drops from 17,759.4 s (1T CPU, Intel Core i7-11700K) to 3,208.7 s (8 CPU workers with shared caches) and to 474.3 s (7.9 minutes, 8.3–134.8 s per PDE) on float64 GPU, including 3,014.0 s →\to 29.3 s (102.7×) on Benjamin–Ono (4.22e-7) and 2,445.9 s →\to 27.7 s (88.2×) on Voigt (6.74e-6), plus 7.2 s and 11.3 s on 1D-Reaction and 1D-Wave.
  • CPU/GPU trajectory parity: Across conventional and wide Peregrine, Benjamin–Ono, Voigt, and periodic heat, the GPU run selects the identical family and geometry sequence across all 127 intervals at relative tolerance 10−510^{-5} (maximum relative L2L^2 difference 0.0033%).
  • Where non-adaptive baselines remain faster: When fixed smooth features already match the solution, non-adaptive solvers are faster: Frozen-PINN solves Burgers to 3.43e-7 in 3.3 s (vs. 134.8 s and 3.91e-6 for GPU CAPINN, and 5,919.1 s on 1T CPU) and periodic heat to 2.1e-5 in 0.6 s/seed on CPU.

See §3.2 (Computation), §4.1, Appendix B.3 (Figure 6 & Table 10) for the per-step cost breakdown and CPU/GPU parity, and Appendix B.4 (Table 11) for the paired baseline wall-clock times.

[W5] Shared numerical FFT evaluation on nonlocal PDEs and repairing the Allen–Cahn Frozen-PINN baseline.

You identified two important confounders: on Benjamin–Ono and Voigt, Cauchy blocks used closed-form Hilbert and fractional-Laplacian formulas while other families used padded FFTs.

AuthorsOpenReview ↗

Response 3/3

[W5 continued]

And on Allen–Cahn, our periodic Frozen-PINN adaptation omitted a constant mode, resulting in a 2.07 error. We resolved both in the revised manuscript:

  • Shared 65,536-node padded FFT on [−800,800)[-800,800) (§4.2; Appendix D.2, Table 18): Disabling the analytic Cauchy operator formulas (Appendix A.1, Eq. 8) and evaluating each family with the identical padded FFT and cubic-spline interpolation gives 4.04e-7 for Mixed CAPINN on Benjamin–Ono (vs. 4.52e-4 for tanh alone [1,119× lower error] and 5.93e-4 for No C [1,468×]; doubling FFT resolution to 131,072 nodes gives 4.04e-7–4.20e-7) and 9.31e-6 on Voigt (vs. 2.32e-4 for No C, 24.9×), showing that the representation advantage persists when all families use the same numerical operator.
  • Repaired periodic Frozen-PINN on Allen–Cahn (§4.1, Table 1; Appendix B.11; Appendix D.1, Table 16): Because u0(x)=x2cos⁡πxu_0(x)=x^2\cos\pi x has mean −2/π2-2/\pi^2 and boundary slope jump ux(±1,0)=∓2u_x(\pm 1,0)=\mp 2, appending a constant mode to the zero-mean Kuramoto–Sivashinsky periodic basis (2.07 ± 7.0e-5 →\to 1.88 ± 1.8e-4) and replacing derivative-matching initialization with a value-only L2L^2 projection reduces five-seed error by 332× to 6.23e-3 ± 1.7e-6 (vs. 4.05e-5 for Mixed CAPINN and 2.24e-5 for No C in single-window periodic fitting).

[Q4] Sensitivity to the number of time intervals NN and entrance weight λe\lambda_e.

You asked how sensitive CAPINN is to the number of time intervals NN and the interval-entrance weight λe\lambda_e (which scales the interval-entrance continuity rows relative to interior PDE residual rows). Our 89-configuration GPU sensitivity study (3,222 intervals; Appendix B.2, Figure 5 & Table 9) characterizes how both parameters behave across dispersive, smooth, and stiff regimes:

  • Time intervals NN and off-grid residual diagnostic (Figure 5b): On wide Peregrine, increasing NN from 12 to 24 resolves the focusing transient, lowering relative L2L^2 error from 8.00e-5 to 4.23e-7 while the unsupervised held-out off-grid PDE residual RMS drops from 2.50e-4 to 6.49e-6. Beyond N=30N=30, interval-seam errors accumulate slightly while the per-interval residual stays small, so the off-grid residual serves as an under-resolution diagnostic rather than a monotonic global error bound.
  • Entrance weight λe\lambda_e (Figure 5a): Sensitivity is small on Benjamin–Ono, wide Peregrine, and periodic heat (which stay in the 10−710^{-7} and 10−910^{-9} ranges across λe∈[1,15]\lambda_e\in[1,15]), whereas on stiff Allen–Cahn, raising λe\lambda_e from 3 to 10–30 suppresses interval-seam accumulation (reducing error from 3.45e-4 to 1.66e-5–4.05e-5).
  • Candidate density, cap self-limiting, and [2Δx,L][2\Delta x, L] scale rule (Figure 5c–f & Table 9): Across the remaining panels of the same study, Benjamin–Ono and wide Peregrine remain in the 10−710^{-7} error range across coarse center counts nc∈[17,129]n_c\in[17,129], acceptance thresholds η∈[0.990,0.999]\eta\in[0.990,0.999], block caps Kmax⁡∈[5,16]K_{\max}\in[5,16] (halting automatically at 5.00 retained blocks per interval on Benjamin–Ono and plateauing at 4.79 mean blocks from cap 8 through 14 on wide Peregrine), and width-bracket expansion up to a 512×512\times span (3.5e-7–5.3e-7), while the mesh-derived width rule [2Δx,L][2\Delta x, L] stays within 0.58×–1.51× of the Table 8 brackets on seven of eight problems at nw=11n_w=11.

[Q6] Tail-restricted error metrics (∣x∣>x0|x| > x_0) testing accuracy in the algebraic tails.

Thank you for asking for a tail-restricted error metric to directly test the claimed advantage on algebraic tails. Evaluating relative L2L^2 error ER=∥uθ−u∥2,R/∥u∥2,RE_R=\|u_\theta-u\|_{2,R}/\|u\|_{2,R} normalized by the reference signal energy inside each spatial zone RR (core, mid-tail, and far-tail) ensures that a large core amplitude cannot mask tail errors, and shows 48× to 7,780× lower far-tail error for Mixed CAPINN relative to No C:

  • Wide Peregrine & Benjamin–Ono: In the far tail (∣x∣>5|x| > 5 on wide Peregrine; co-moving ∣x−0.3t∣>20|x - 0.3t| > 20 on Benjamin–Ono), Mixed CAPINN keeps relative L2L^2 error at 5.40e-6 on wide Peregrine (vs. 4.20e-2 for No C, 7,780× lower) and 3.35e-6 on Benjamin–Ono (vs. 1.05e-2 for No C [3,126×] and 1.35e-2 for three-seed PINN + RAD [4,032×]).
  • Voigt: In the Gaussian core (∣x∣≤5|x|\le 5), No C fits slightly better than Cauchy alone (3.72e-5 vs. 4.37e-5; tanh alone is 4.26e-5), but in the algebraic far tail (∣x∣>10|x| > 10), No C and Gaussian jump by 300× and 491× relative to their core errors (reaching 1.11e-2 and 4.17e-2, vs. 1.08e-3 for Cauchy). Mixed CAPINN improves both regions simultaneously, reaching 3.91e-6 in the core and 2.32e-4 in the far tail (48× below No C).

See §4.2 (Table 4) and Appendix D.3 (Figure 16 & Table 19) for the full three-zone breakdown across all family and adaptive-sampling arms.

Dear authors,

Thank you for your response. My main concerns and questions have been resolved, and I would like to keep the positive rating.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

Dear Area Chair and Reviewers,

We thank all reviewers for their time and constructive feedback. In revising the manuscript, we focused on improving the clarity of the presentation, tightening our claims to match the evidence, and running the empirical controls you requested. Throughout these updates, the core CAPINN formulation, construction algorithm (Algorithm 1), and primary 1D benchmark results (Tables 1–2) remain unchanged.

The revision makes three main updates:

  • Improved clarity and chronological presentation (§1–§3): We reorganized the Method section (§3.2) to walk through a single time interval in five chronological steps matching Figure 1 and Algorithm 1, and revised Sections 1–2 (and Appendix A) to clarify the motivation and design rationale behind CAPINN and to credit prior frozen-feature, greedy, Cauchy, and KAN work more clearly.
  • Tightened claims and explicit regime boundaries (§1, §4.1, §4.3, §5): Following your feedback, we removed the "balanced benchmark" characterization, clarified that CAPINN selects within a supplied dictionary rather than discovering new symbolic families, and reframed the pole-trajectory study (Figure 3) from inverse pole recovery to interpretable real-axis fitting with post-hoc alignment metrics.
  • Expanded benchmarks, ablations, and parallel GPU profiling (§4, Appendices B & D): Without changing the underlying solver, we parallelized candidate evaluation on float64 GPUs for fair wall-clock comparisons (37× suite speedup) and added the requested controls: fixed-budget selection ablations, shared-FFT operator controls, multi-seed physics-validated baseline tuning and adaptive sampling (RAD/RAR-D), regional tail metrics, an 89-run hyperparameter and mesh-width sensitivity study, and cross-regime evaluations on 1D-Reaction/Wave, coupled 2D Navier–Stokes and KP-I flows, 5D–100D radial PDEs, and Airy/Bessel equations.

We hope these revisions clarify the motivation for CAPINN, where it is effective, and the evidence supporting our claims. The updated supplementary material includes the code and configurations for all new experiments. Thank you again for your time and constructive feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.