- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper addresses residual aliasing in PINNs, where low point‑wise PDE loss at collocation points can mask large high‑frequency residual components between samples by proposing RAA‑PINN, an adaptive sampling method that a spectrum‑aware adaptive sampling which selects new collocation points using both residual magnitude and local spectral energy, together with residual‑spectral regularizer that penalizes high‑frequency content of the PDE residual. Empirical results across five PDE benchmarks demonstrate performance gains over strong baselines such as RAR, RAD and PINNacle‑like sampling; ablation studies isolate the contributions of spectral regularization, adaptive sampling and final refinement.
Critical strengths
None.
Critical weaknesses
-
Residual aliasing could be alleviated by increasing the resampling frequency and the number of collocation points. This manuscript should discuss the relationship with them.
-
Since the main contribution of this paper is a sampling method, the baselines should focus on sampling strategies rather than other techniques, such as random Fourier embeddings and causal PINNs. A more appropriate comparison would be PINN + random Fourier embeddings + RAD vs. PINN + random Fourier embeddings + RAA, which would better isolate the contribution of the proposed sampling method.
Additionally, the paper does not cite the PINNacle-like sampling method. Is the method used exactly PINNACLE? A proper citation is essential for a baseline method.
-
The primary evaluation metric is HFRE rather than the relative error of the solution, which makes it less convenient to assess the overall solution accuracy. Additionally, in Table 3, PINN + RAR (A2) achieves a lower HFRE than Spectral (A3). This result is surprising, given that A3 uses an HFRE-like penalty, and warrants further explanation.
-
For sampling methods in PINNs, visualizing the distribution of sampling points is important for understanding how the sampling strategy works. However, the paper lacks such visualizations, making it difficult to assess the effectiveness of the proposed method.
-
Lack of cost analysis, especially for the penalty term.
These issues, particularly Weaknesses 3 and 4, raise concerns about the soundness of the paper.
Questions for authors
See weakness.
7 comments
Thank you for the authors' response. However, I still have some remaining concerns.
Q1
Since increasing either the sampling frequency or the number of sampling points leads to improved results, I encourage the authors to further increase these quantities. The results appear to follow a consistent improving trend, suggesting that the current experiments are not yet sufficient to demonstrate convergence or establish the effectiveness of RAD convincingly.
Q4
Is the plotted figure Figure S12?
-
It would be better to provide the sampling points for all baselines to facilitate a direct comparison.
-
Please also provide the distribution of the solution to help readers better understand the sampling strategy and its effect.
-
Could the authors explain why RAD appears to produce a nearly random sampling pattern, whereas RAA can concentrate sampling points around the center? Have the hyperparameters of RAD been appropriately tuned?
Response to Reviewer cX9z: Additional Sampling Budget Experiments and Sampling Distribution Analysis
Thank you for your follow-up comments. We have updated the manuscript accordingly.
Q1: We further increased the RAD sampling budget. Relative L2 error improved from 0.0158 to 0.0154 and 0.0153, while HFRE decreased from 0.207 to 0.203 and 0.201, with runtimes of 98.2 s and 101.4 s. The improvements became smaller, and RAA still achieved lower errors (L2: 0.0138; HFRE: 0.175) under its original configuration. Results are presented in Table S11 and Figure S8.
Q4.1: We expanded Figure S12 to include the sampling distributions of PINN, RAR, RAD, RAA, and PINNACLE. All methods use the same initial points and spatial domain, enabling direct comparison of their sampling patterns.
Q4.2: Figure S12 now includes the CD-vort reference solution. Its appearance is similar to the CD-const reference because both benchmarks share the same manufactured solution but use different convection fields and corresponding forcing terms.
Q4.3: RAD uses residual-probability sampling with , a numerical offset of , and 3,072 candidates. Its probabilistic selection produces broader spatial coverage, whereas RAA combines residual magnitude with local spectral information. The central-region occupancy is 14.65% for RAD and 21.63% for RAA, explaining the observed difference in Figure S12.
Response to Reviewer cX9z
Thank you for the specific comments on residual aliasing, sampling baselines, evaluation metrics, visual evidence, and computational cost. We added controls for point count, update frequency, shared features, metrics, visualization, and runtime. All comparisons use seeds 0, 1, and 2 with mean plus standard deviation; fixed-epoch results use the final checkpoint at epoch 3000.
Q1: Could residual aliasing be addressed simply by increasing the point count or resampling frequency?
We added a fixed-3000-epoch sampling-budget control for RAD. The default RAD setting adds 96 points every 100 epochs and ends with 3904 points. The increased-point setting adds 192 points every 100 epochs; the increased-frequency setting adds 96 points every 50 epochs; the combined setting adds 192 points every 50 epochs.
Default RAD obtains relative L2 error 0.0208±0.0007 and HFRE 0.258±0.006. Increasing the points gives 0.0181±0.0008 and 0.231±0.007; increasing the update frequency gives 0.0176±0.0007 and 0.224±0.006; changing both gives 0.0158±0.0006 and 0.207±0.006. Under the common base schedule, RAA gives 0.0138±0.0004 and 0.175±0.004.
Thus, additional RAD points and more frequent updates improve the result, but they do not explain the full RAA improvement. The corresponding table and figure are in Appendix H.
Q2: Does the baseline comparison isolate the sampling contribution?
We added a matched shared-Fourier-feature comparison. RAD and RAA use the same feature map, initial points, candidate pool, update interval, number of added points, final point count, training epochs, and random seeds. Only the point-selection rule changes.
Both methods use 1024 initial points, a candidate pool of 3072, 96 added points every 100 epochs, and 3904 final points. RAD obtains relative L2 error 0.0166±0.0004 and HFRE 0.209±0.005; RAA obtains 0.0115±0.0003 and 0.149±0.004. The difference is therefore associated with the sampling rule rather than the feature map or point budget.
Q2.1: Is the PINNACLE baseline the original method?
The revised manuscript identifies the baseline as PINNACLE and cites Hao et al. (2024). The implementation follows the sampling definition of the original PINNACLE work. The method name is now written consistently as PINNACLE in the text, tables, captions, and figure labels; the previous “PINNacle-like” wording has been removed.
Q3: Why was HFRE emphasized, and is the Table 3 ordering contradictory?
The revised manuscript reports relative solution error, HFRE, and residual energy together. For the comparison corresponding to the original Table 3:
PINN + RAR: relative L2 error 0.0204±0.0008, HFRE 0.182±0.006, residual energy 0.021;
Spectral: relative L2 error 0.0173±0.0006, HFRE 0.196±0.007, residual energy 0.017;
RAA-PINN: relative L2 error 0.0138±0.0004, HFRE 0.175±0.004, residual energy 0.014.
The spectral training penalty is a weighted band term evaluated during optimization at the spectral probes. HFRE is computed after training on the common dense evaluation grid. They therefore use different weights, locations, and evaluation stages. A lower training penalty does not require the lowest post-training HFRE. The joint metrics and metric-alignment figure make this distinction explicit while providing an independent solution-accuracy measure.
Q4: Can the sampling-point distributions be visualized?
We added a sampling-distribution figure under the matched shared-Fourier-feature protocol. Gray points are the 1024 shared initial points, and colored points are the newly added points for RAD and RAA. The figure specifies the vortex benchmark, seed 0, 3000 epochs, 96 added points every 100 epochs, 2880 added points, and 3904 final points.
The CD-vortex, CD-const, and Kovasznay field figures provide spatial error evidence, while the dedicated Appendix H figure provides the sampling-point visualization. The S2 and S6 captions were revised to match their displayed content.
Q5: What is the cost of the spectral penalty and the complete method?
We added a 3000-epoch Adam-only runtime decomposition. PINN takes 41.5 seconds. RAD adds 14.3 seconds of sampling and spectral-sampling cost, for 55.8 seconds total. RAA sampling without spectral regularization adds 34.1 seconds, for 75.6 seconds total. Complete RAA-PINN adds 29.2 seconds of spectral regularization, for 104.8 seconds total.
These costs are also included in the measured 90-second comparison. Fixed-epoch results compare optimizer updates, wall-clock results compare actual time, and the decomposition reports the additional cost.
The sampling-budget, matched-feature, metric-alignment, sampling-distribution, and runtime controls are included in the experimental section and Appendix H; Appendix A provides the seed-level results. RAA remains the lowest-error principal sampling method under both protocols.