- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Cauchy-Adaptive Physics-Informed Neural Networks (CAPINN) and studies a relatively underexplored limitation of PINNs. In addition to optimization difficulties, the choice of a fixed representation family can itself limit the accuracy of PDE solvers. The authors argue that different PDE solutions can exhibit distinct analytic structures, including complex poles, algebraic tails, localized smooth profiles, and sharp fronts. Under a limited neuron budget, selecting basis functions that match these structures can therefore be important for obtaining accurate and compact representations. CAPINN constructs the solution representation from a predefined analytic dictionary containing Cauchy, tanh-derived, and Gaussian basis functions. The PDE residual is used to greedily select the basis class, location, and scale during the solve. The paper demonstrates that this residual-guided selection can produce highly compact representations for several one-dimensional PDEs. The gains are particularly strong for problems with rational poles or algebraic tails, where Cauchy features closely match the underlying analytic structure.
Critical strengths
-
The paper combines analytic structure with residual-guided basis construction in a simple and interpretable framework. The design is conceptually clean, and the selected basis functions provide a direct connection between the learned representation and the underlying structure of the PDE solution.
-
The matched representation ablations and the inclusion of negative controls provide reasonably convincing evidence for the paper's focused claim that representation choice matters. In particular, the comparisons help isolate the effect of different basis families under comparable settings.
-
The paper clearly presents the failure cases of Cauchy features and does not overstate their general applicability. This makes the scope of the method easier to assess and also motivates further mechanistic analysis of when different analytic representations are beneficial.
Critical weaknesses
The claimed balanced benchmark appears to introduce a reverse benchmark bias. The paper argues that conventional PINN benchmarks often favor tanh-based representations because many standard problems contain tanh-like fronts. However, the proposed benchmark roster is also strongly aligned with the inductive bias of CAPINN. The first five configurations, including two Peregrine settings, Benjamin–Ono, Voigt, and Michelson–Sivashinsky, contain isolated complex poles or algebraic tails. These are precisely the structures that Cauchy neurons are designed to represent. Only the remaining three configurations serve as controls without this strong alignment. This makes the benchmark balance somewhat questionable. The experiments convincingly show that Cauchy features work well on problems with Cauchy-like analytic structure. They provide weaker evidence that CAPINN is generally preferable as a representation-learning strategy across a broader class of PDEs.
The empirical evaluation is limited to relatively simple 1D PDEs. This makes it difficult to assess whether the proposed basis-selection strategy remains effective when the geometry, solution structure, and computational cost become substantially more complex. I would strongly encourage the authors to include at least one higher-dimensional benchmark. A problem with complexity comparable to a two-dimensional Navier–Stokes system would make the empirical claims considerably more convincing. Such an experiment would also help clarify whether residual-guided dictionary search remains computationally practical in higher dimensions.
The method does not automatically discover new feature families. CAPINN selects a representation from a manually specified dictionary: Cauchy, tanh and Gaussian. This is an important distinction. The PDE residual determines which candidate is selected, together with its centre and scale, but the candidate families themselves are provided by the designer.
If the true solution is better characterized by structures such as Airy, Bessel, log singularity and branch point, CAPINN cannot discover such a representation unless the corresponding family is already included in the dictionary. For this reason, claims suggesting that the residual alone discovers the appropriate representation or pole structure appear overstated. The search space already contains substantial prior knowledge about the expected solution structure. A more precise interpretation is that CAPINN performs adaptive model selection within a manually constructed hypothesis space.
The dictionary design introduces substantial prior information about the singularity structure. The training guide states that, for rational solutions, the width interval should extend down to the smallest pole distance from the real axis. This weakens the rhetoric in the abstract and conclusion that pole information is not provided and that the poles emerge purely from the PDE residual. It is true that the exact pole locations are not used as supervision. However, the method is still given the correct analytic family together with a scale range designed to cover the relevant singularity scale. These two statements should be clearly distinguished. The current experiments therefore demonstrate that the residual can locate useful Cauchy elements inside a carefully designed search space. They provide weaker evidence for genuine representation discovery. A stronger discovery experiment would use a substantially overcomplete dictionary containing many distractor families and a much broader range of scales. The authors could then test whether residual-based selection still consistently recovers the appropriate analytic structure. Another useful experiment would remove the Cauchy family and test whether the algorithm can approximate the same solutions through alternative rational families. Under the current setup, the method is better characterized as model selection among carefully curated hypotheses.
The methodological novelty appears more incremental than the title and contribution statements suggest. Several central components of CAPINN have close precedents. Frozen-PINN already studies frozen features, space-time separation, and coefficient fitting without standard gradient-based training. AG-RaNN uses the residual as an error indicator to adaptively grow randomized networks and constructs new neuron parameters during this process. IFeF-PINN separates basis construction from coefficient regression and provides convergence analysis for the linear setting. XNet directly introduces Cauchy activations and applies them to PDE solving. Given these connections, I believe the main novelty of CAPINN should be stated more narrowly as the specific combination of heterogeneous analytic dictionary + block-OLS residual scoring + local geometry refinement + nonlinear acceptance check.
This combination is meaningful and technically interesting. However, broader claims that CAPINN is the first method that allows the PDE itself to construct the representation appear too strong given the existing literature.
An important related baseline, AC-PKAN, is missing from the discussion and comparison. AC-PKAN [1] is closely related to the motivation of this paper. Both works directly address the representational limitations of standard PINNs and explore structured basis or activation functions under physics-informed objectives. AC-PKAN combines Chebyshev-based Kolmogorov-Arnold layers with learnable wavelet activations and residual-gradient attention to improve representational capacity and optimization. At minimum, I believe AC-PKAN should be discussed in the related work section. An empirical comparison would be even more informative if a compatible implementation is available.
A possible distinction could be stated as follows: Recent PINN architectures have also explored structured basis functions beyond standard MLP activations. AC-PKAN combines Chebyshev-based Kolmogorov-Arnold layers with learnable wavelet activations and residual-gradient attention to improve expressivity and optimization under physics-informed objectives. CAPINN differs in that it explicitly selects the analytic class, centre, and scale of individual frozen basis elements according to their predicted reduction of the PDE residual.
[1] Zhang et al. AC-PKAN: Attention-Enhanced and Chebyshev Polynomial-Based Physics-Informed Kolmogorov-Arnold Networks. TMLR, 2026.
Questions for authors
Could the authors provide a quantitative analysis of pole recovery in addition to the qualitative alignment shown in Figure 3? Figure 3 is interesting, but the current evidence is mainly visual and post-hoc. It would be useful to quantify how accurately the selected Cauchy neurons recover the underlying pole geometry. For example, the authors could report and I would also encourage the authors to analyze how these errors change with the candidate grid resolution, time-slab resolution, neuron budget, and dictionary width range. This is particularly relevant for the Benjamin–Ono experiment. The paper notes that the selected widths form several bands, so the agreement in pole distance is only approximate. A quantitative sensitivity analysis would help clarify how robust the observed pole alignment is to the search design. If the alignment is highly sensitive to these design choices, I think the claim should be framed more conservatively as interpretable fitting behavior. A stronger claim of singularity discovery would require evidence that the recovered pole geometry remains stable across reasonable changes in the search space and discretization.
All in all, regarding the weaknesses and questions I raised, if they are not adequately addressed, I will lower my recommendation rating.
5 comments
Response 2/4
[W2] Scaling beyond 1D: coupled 2D steady Navier–Stokes and time-dependent D KP-I.
You encouraged us to test whether residual-guided dictionary search remains effective and computationally practical on a coupled higher-dimensional fluid system such as 2D Navier–Stokes. Following your recommendation, we added coupled 2D steady incompressible Navier–Stokes on (solving simultaneously for velocity and pressure at ) and time-dependent D Kadomtsev–Petviashvili (KP-I) lump propagation (§4.3, Table 5 & Figure 4; Appendix D.5, Tables 24–26):
- Parameter-matched 2D PIG comparison on sine-lid cavity (; Table 5, Figure 4a–d, & Table 24): Mixed CAPINN selects 61 Cauchy and 38 Gaussian blocks covering the cavity boundary layers and interior vortex, achieving 0.0228% full-vector velocity error on Taylor–Hood FEM128 versus 0.1518% ± 0.0104% (3 seeds) for our local adaptation of 2D PIG (6.65× lower error at 1,290 vs. 1,291 total scalars including geometry on the same interior and wall collocation grid).
- Published COMSOL reference at and (Table 5 & Table 24): Mixed CAPINN reaches 0.0279% speed error at (vs. 0.0404% for published PIG and 0.122% for PGCAN) and 1.45% speed error at (comparable to PGCAN's 1.42% and 1.6×–4.1× below single-family Cauchy 2.36%, Gaussian 2.52%, and validation-tuned Fixed Gaussian 5.94%).
- Singular unit-lid cavity (; Table 5 & Table 25): On the singular unit-lid cavity () evaluated without corner cropping, three Sobol candidate-pool seeds reach 2.83% ± 0.07% (Cauchy) and 3.49% ± 0.66% (Mixed, 0.22% ± 0.03% Ghia centerline RMS) versus 8.93% for validation-tuned Fixed Gaussian.
- Time-dependent D KP-I lump propagation (Table 26): Across 10 time intervals, radial Cauchy blocks reach 0.335% ± 0.018% full-time error versus 5.30% ± 1.62% for radial Gaussian (15.8× lower error at blocks/interval) and 2.78% ± 1.23% for static initial-geometry Cauchy (8.3× lower error from per-interval basis renewal).
[W3] Representing Airy, Bessel, and regularized log/branch profiles within a supplied dictionary.
We agree that CAPINN selects and refines atoms within a designer-supplied dictionary; it does not invent new symbolic families (§1, §3.1, §5). The distinction is between discovering a new family and constructing an accurate representation of a solution: the latter need not require an atom with the same functional form. Appendix D.4 tests this distinction in two complementary ways:
- Residual-guided Airy/Bessel solves (Appendix D.4, Figure 17; Tables 22–23): Without adding Airy or Bessel functions, the original Cauchy/tanh-derived/Gaussian dictionary (CTG) reaches relative errors of 1.46e-8 on the Airy initial-value problem and 4.96e-8 on the unit-weighted Bessel problem, using equation residuals and prescribed initial/boundary data rather than interior solution labels. Explicitly adding paired Airy and Bessel blocks () further reduces turning-point Airy error from 6.93e-6 to 2.49e-10 (and radial Helmholtz error from 5.95e-5 to 6.03e-14), showing the benefit of richer supplied building blocks.
- Known-profile approximation (Appendix D.4, Table 21): On regularized logarithmic and fractional-power profiles on (), 33-column () Cauchy spans attain 1.02e-9–5.75e-9 discrete least-squares fitting error with widths in . No C is more accurate on the narrower width interval , demonstrating scale dependence. These experiments fit known, regularized functions; they do not solve PDEs or test approximation of functions with singularities on the real axis.
Appendix D.4 thus separates representation capacity, residual-guided construction, and explicit dictionary enrichment. Together, the results establish useful approximation beyond the individual forms of the supplied atoms, without claiming symbolic family discovery.
Response 3/4
[W4] Distinguishing dictionary and scale-range priors from withheld pole coordinates.
You pointed out an important tension in our initial draft: our original Appendix B note advised setting the minimum width down to "the smallest pole distance from the real axis," which made it appear that CAPINN relied on problem-specific pole-depth priors. For a Cauchy feature evaluated on the real axis, is simply its half-width at half-height (Appendix A.1), so that note was an imprecise description of a spatial resolution floor rather than an injection of analytical pole coordinates. Prompted by your critique, we systematically re-examined our width-bracket sensitivity and found that problem-tailored width brackets are largely unnecessary: a single mesh-derived rule without pole-depth input preserves comparable accuracy on most of the eight problems, while also revealing width-density-sensitive exceptions. We therefore removed the confusing pole-distance note and replaced it with this quantitative sensitivity study in Appendix B.2 (Figure 5 & Table 9), while stating in Sections 1 and 5 that the candidate dictionary itself remains a supplied hypothesis space:
- Uniform mesh rule (Appendix B.2, Table 9): Setting the width bracket purely from the spatial grid spacing and domain half-width stays within 0.58×–1.51× of the Table 8 brackets on seven of eight PDEs at (improving accuracy on four of eight, including 6.09e-6 vs. 6.74e-6 on Voigt and 2.70e-6 vs. 3.91e-6 on Burgers) and within 0.82×–1.45× on seven of eight at , while documenting larger width-density sensitivity on Michelson–Sivashinsky (1.71e-3 at vs. 8.56e-3 at ) and Allen–Cahn (8.25e-4 at vs. 6.12e-5 at ).
- Broad scale-bracket expansion (Appendix B.2, Figure 5f): Widening up to a span keeps relative error in 3.5e-7–5.3e-7 on Benjamin–Ono and wide Peregrine as long as resolves the finest feature scale.
[W5] Positioning relative to frozen-feature, greedy, and Cauchy precedents, and the viewpoint motivating our construction.
We agree that the individual building blocks have close precedents (Frozen-PINN, AG-RaNN, IFeF-PINN, XNet, and OMP/OLS/greedy PDE solvers such as Siegel et al., 2023), and we revised Sections 1–3 and Appendix A.1–A.3 to credit each work explicitly without any "first-ever" claim for residual-guided construction:
- Motivation, novelty, and significance ("right feature, right place" at finite budget; §1, §2, Appendix A.1–A.3): Although Gaussian, tanh, and Cauchy networks each enjoy universal approximation and favorable theoretical approximation rates (Mhaskar, 1996; Li et al., 2025a,b), those results quantify approximation potential while leaving the task of finding useful features to the solver (Appendix A.3). A single PDE solution can combine smooth regions, sharp transitions, and localized rational peaks or power-law tails whose locations, scales, and relative prominence evolve over time: feature shape governs how compactly a basis captures each local structure, yet a dictionary of suitable functions does not specify which ones to use or where to place them without interior solution labels (§1). Existing methods adapt parts of this representation separately—through single-family frozen-feature renewal, end-to-end adaptive activations or Gaussian centers, or greedy basis expansion. Our contribution is studying feature-family choice and geometric placement together within a constructive frozen-feature framework (§1, §2, Appendix A): using the PDE residual to jointly select family, location, and scale across blocks and renew the active representation across time intervals so a candidate dictionary is turned into a compact PDE basis during the solve.
- Controlled evidence that family choice and geometric placement are both necessary (§4.2, Tables 2–3; Appendix D.2, Table 17): Even though each single family is a universal approximator, combining all three families in Table 2 improves Voigt by 7.7× over the best single family (6.74e-6 vs. 5.19e-5 for Cauchy-only and 2.65e-4 for tanh-only), lowers rational-wave error by 1,404×–4,900× over No C, and selects pure Gaussians on periodic heat (88× below Cauchy-only). Conversely, holding the dictionary, Gauss–Newton refitting, and the same coarse entrance-initialization procedure fixed at blocks/interval in Table 3 (and Table 17), five-seed random placement leaves 0.475 ± 0.011 error on wide Peregrine and 1.01 ± 0.033 on Benjamin–Ono, whereas discrete Block-OLS selection lowers error to 1.16e-3 and 2.91e-2, and four-round local refinement lowers it to 1.46e-6 and 8.52e-6.
Response 1/4
Thank you for pressing us to distinguish useful finite-budget representation construction from broader discovery claims. Your review motivated coupled 2D flow tests, experiments on the non-rational and oscillatory structures you named, bidirectional AC-PKAN comparisons, and quantitative pole-alignment diagnostics. While we substantially tightened the framing across Sections 1–5 to reflect these boundaries, the core CAPINN algorithm (Algorithm 1), the {Cauchy, tanh-derived, Gaussian} dictionary (§3.1, Eq. 1), and our primary 1D benchmark results (Tables 1–2) remain unchanged. The resulting evidence sharpens our contribution: residual-guided joint selection of family and geometry within a supplied dictionary, with explicit regime boundaries.
[W1] Benchmark composition, dropping the "balanced benchmark" claim, and cross-regime evaluation.
We agree with your critique that calling a five-of-eight pole-and-tail suite a "balanced benchmark" overstates its generality and introduces reverse benchmark bias. We removed the "balanced benchmark" characterization throughout Sections 1, 4.1, and 5, reframed the original 1D suite as a regime-diagnostic testbed—where smooth periodic heat selects only Gaussians (0/0/72 Cauchy/tanh-derived/Gaussian blocks, 3.23e-7 in Table 2), No C (the tanh-derived + Gaussian dictionary without Cauchy blocks) is more accurate than Mixed on 1D Allen–Cahn (2.24e-5 vs. 4.09e-5 in Appendix B.11), and non-adaptive Frozen-PINN is faster and more accurate on Burgers (3.43e-7 in 3.3 s in Tables 1, 10–11)—and added cross-regime validation across standard reaction/wave, coupled 2D fluid/wave, non-rational oscillatory, and 5D–100D radial PDEs: