- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
PINOT is a neural operator designed to support changes in spatial patch partitioning and source discretization. It encodes local observations through scale-normalized polynomial coefficients, processes patch tokens using measure-weighted attention, and reconstructs outputs through local polynomial expansions. The paper establishes asymptotic consistency under admissible joint refinement and retention of local derivative information. Experiments across six PDE benchmarks demonstrate generalization to unseen resolutions, improved reconstruction with fewer tokens, and competitive accuracy and efficiency, particularly on nonuniform discretizations.opod data.
Critical strengths
The paper identifies a meaningful distinction between source-discretization consistency and consistency under changes in latent partitioning. The architecture addresses both local representation scaling and the influence of unequal patch measures on attention. This provides a coherent design for changing latent resolution without retraining, supported by conditional theoretical results and resolution-transfer experiments. The empirical evidence covers complementary aspects of the proposed design. Reconstruction experiments assess local information retention, while PDE prediction and runtime measurements assess practical usefulness. The strong results on nonuniform benchmarks support the value of combining local polynomial representations with adaptive token allocation.
Critical weaknesses
The consistency guarantees do not directly cover the implemented coefficient estimator. Appendix C assumes full-basis ordinary least squares with uniformly stable Gram matrices. Appendix F.2 instead uses ridge regularization and drops polynomial modes in sparsely sampled patches. These changes matter because normalization multiplies degree-m coefficient errors by h^{-m}, potentially amplifying regularization bias and observation noise as patches shrink. The paper does not establish when the implemented estimator retains the claimed limit or measure its stability in this regime.
The experiments do not sufficiently isolate the mechanisms responsible for partition robustness. The paper compares polynomial bases and adaptive partitions, but does not independently remove scale normalization or measure weighting. In particular, the quantitative patch-resolution experiment uses uniform 1D partitions, where all patch measures are equal and measure-weighted attention reduces to ordinary attention. It therefore cannot validate the proposed correction for unequal partition density. The nonuniform benchmark comparisons also combine architectural changes with adaptive partitioning available only to PINOT.
Questions for authors
-
What ridge coefficient is used in Equation (178)? Is it fixed or scale-dependent, and under what conditions does the implemented solver retain the consistency established for ordinary least squares?
-
How frequently is rank limiting activated on the nonuniform benchmarks, and which polynomial orders remain available in those patches?
-
Do existing experiments separately evaluate removal of scale normalization or measure weighting? If not, please clarify which empirical conclusions concern the complete architecture rather than individual components.
6 comments
Response to Reviewer NeME (2/4)
W1-2, Q2: Rank limiting (dropping higher-order polynomial modes)
1) Necessity and mechanism of rank limiting
Although we fixed the maximum total degree of the polynomial basis, some patches contain too few source points to uniquely estimate the coefficients of all polynomial modes. Specifically, when the number of source points is smaller than the number of basis modes, the corresponding least-squares system becomes underdetermined. For example, four cubic-polynomial coefficients cannot be uniquely determined from only two observations without imposing additional assumptions. Therefore, for patches with an insufficient number of source points, we have retained the largest set of polynomial modes that can be permitted by the available observations, dropping higher-order modes first.
2) Retained polynomial basis modes
The following table reports the distribution of the number of retained polynomial modes across patches. The full basis consists of 6 modes for NACA0012 and RAE2822, and 10 modes for DeepJEB. A small fraction of patches retain fewer modes because of insufficient source-point counts within each patch.
| Number of retained modes | NACA0012 | RAE2822 | DeepJEB |
|---|---|---|---|
| 1 | 1.2% | 1.4% | 1.1% |
| 2 | 1.0% | 1.0% | 1.0% |
| 3 | 1.2% | 1.1% | 1.0% |
| 4 | 1.8% | 1.7% | 1.0% |
| 5 | 2.5% | 2.5% | 1.2% |
| 6 | 92.3% | 92.3% | 1.5% |
| 7 | — | — | 1.8% |
| 8 | — | — | 2.0% |
| 9 | — | — | 2.1% |
| 10 | — | — | 87.2% |
3) Frequency of rank limiting activation
The fraction of patches for which rank limiting is activated corresponds to the fraction retaining fewer than the full set of polynomial modes (6 for NACA0012 and RAE2822, and 10 for DeepJEB), as summarized below.
| NACA0012 | RAE2822 | DeepJEB | |
|---|---|---|---|
| Rank limiting activated | 7.7% | 7.7% | 12.8% |
Thus, the full polynomial basis is retained for the large majority of patches in all three datasets. In practical simulation settings, since source-point density may correlate with local geometric or field complexity, sparsely sampled regions may sometimes require less higher-order local detail. This offers one possible explanation for why dropping higher-order modes in a relatively small fraction of patches may have only a limited effect on overall prediction accuracy. However, we emphasize that our conclusion regarding the practical impact of rank limiting is based on the empirical results below rather than on this interpretation. To directly assess the practical impact of rank limiting, we have further evaluated PINOT while deliberately increasing the frequency of rank-limiting activation, as described below.
4) Experimental investigation of the effect of rank limiting
Since the full polynomial basis cannot be uniquely estimated when the number of source points is insufficient, rank limiting cannot be entirely eliminated. Instead, we have investigated its effect by intentionally increasing the frequency of rank limiting, even in patches where retaining additional polynomial modes would otherwise be mathematically feasible. Specifically, we have trained PINOT on NACA0012 using the original rank-limiting strategy, which drops higher-order modes when the number of source points in a patch is smaller than the number of coefficients required to retain those modes. At inference time, we have increased this requirement from to , where is an additional parameter controlling the frequency of rank-limiting activation. We have varied from 0 (the original setting) to 4.
| Additional point requirement | Original (0) | 1 | 2 | 3 | 4 |
|---|---|---|---|---|---|
| Rank limiting activated (%) | 7.7 | 11.0 | 14.9 | 19.6 | 24.7 |
| Relative error (%) | 5.72 | 5.76 | 5.90 | 6.15 | 6.69 |
The table above shows that, even when rank limiting is deliberately activated in a substantially larger fraction of patches, the degradation in predictive accuracy remains relatively moderate over the tested range. These results indicate that the original rank-limiting strategy, which affects only 7.7% of patches on NACA0012, has a limited practical impact on overall predictive performance.
We are revising the manuscript to incorporate the experimental results and the accompanying discussion.
Response to Reviewer NeME (3/4)
W1-3: Observation noise
1) Sensitivity to observation noise
Yes, we acknowledge that PINOT trained exclusively on clean input can become sensitive to observation noise at inference time, particularly as the patch scale decreases. This tendency is evident from the substantial increase in normalized relative error (%) at as the noise level increases, as shown in the table below.
| Patch scale | Noise: 0.1% | Noise: 0.5% | Noise: 1% | Noise: 5% | Noise: 10% |
|---|---|---|---|---|---|
| 4.96 | 4.96 | 4.96 | 4.98 | 5.04 | |
| 1.25 | 1.27 | 1.30 | 2.04 | 3.32 | |
| 0.70 | 1.39 | 2.48 | 12.70 | 27.23 |
In this experiment, we have trained PINOT on clean data and evaluated it with noisy input at different patch scales using the 1D Burgers dataset with 65,536 source points. Here, denotes the patch scale, and the noise level (%) denotes the standard deviation of additive Gaussian noise relative to the standard deviation of the input data.
2) Robustness with noisy-input training
As you pointed out, this high sensitivity to observation noise is closely related to our scale normalization, which rescales polynomial coefficients by , as defined in Eq. (5) of the originally submitted manuscript. Here, is a polynomial multi-index, and is its total degree. Since higher-order coefficient estimates can be particularly sensitive to observation noise, the factor can substantially amplify noise-induced perturbations as the patch scale decreases.
Since observation noise is expected in many practical measurement settings, we have further evaluated PINOT under noise-aware training. Specifically, for each noise-level and patch-scale configuration, we have trained PINOT using noisy input and clean output, and evaluated the model under the corresponding condition. This setting also allows the learnable factor to adapt during training in the presence of noisy observations. The following table reports the resulting normalized relative error (%).
| Noise: 1% | Noise: 5% | Noise: 10% | |
|---|---|---|---|
| 0.67 | 1.00 | 1.14 | |
| 0.79 | 0.80 | 0.72 | |
| 0.34 | 0.40 | 0.50 |
Compared with clean-input training, noisy-input training substantially improves prediction accuracy under noisy observations across all tested patch scales. Notably, even at the finest patch scale of , the error remains at or below 0.50% for noise levels up to 10%. These results demonstrate that incorporating observation noise during training can effectively mitigate the noise sensitivity observed under clean-input training.
We are revising the manuscript to incorporate the experimental results and the accompanying discussion, together with our theoretical analysis of how the learnable factor can contribute to mitigating observation noise during noisy-input training.
Response to Reviewer NeME (1/4)
Dear Reviewer NeME,
We sincerely appreciate your sharp and insightful feedback. Below, we address your concerns in detail. Please note that we are currently revising the manuscript to incorporate additional results and discussions in this response, and we will upload the revised version as soon as it is complete.
W1-1, Q1: Ridge regularization
1) Ridge coefficient used in our experiments
We used a fixed ridge coefficient of λ=10−4 in the normalized least-squares system in Eq. (178), independent of the patch scale. To address your concern, we have additionally 2) derived sufficient conditions under which the ridge-regularized implementation preserves the ordinary least-squares continuum limit, and 3) experimentally quantified the prediction discrepancy induced by nonzero λ to assess the practical effect of ridge regularization under our experimental settings.
2) Sufficient conditions for retaining the ordinary least squares limit
We have analyzed a theoretical finite-resolution error bound for the discrepancy between the prediction of our ridge-regularized implementation and the corresponding unregularized continuum limit. Under the assumptions of uniform hypercubic patches, uniformly distributed source points, and Lipschitz stability of the network layers, we have derived the following bound on the spatial L∞ error:
∣∣u^fin(λ)−u^lim∣∣L∞≤Cnet(h+h nmin−1/d+λh−K).
Here, h is the patch scale; nmin is the minimum number of source points per patch; d is the spatial dimension; K is the maximum total degree of the polynomial basis; λ is a ridge regularization coefficient introduced as a numerical safeguard for coefficient fitting at finite resolutions; and Cnet is a model-dependent constant independent of h, nmin, and λ.
A detailed derivation and proof of this finite-resolution error bound will be included in the revised manuscript.
The first two terms characterize finite-resolution effects, which are discussed in our response to Reviewer A4ZX (W2). Here, we focus on the last term λh−K, which captures the contribution of ridge regularization. A sufficient condition for retaining the ordinary least squares continuum limit is λh−K→0, together with the vanishing of the other finite-resolution terms. We acknowledge that a fixed positive ridge coefficient, as used in our implementation, does not satisfy this sufficient condition under arbitrary refinement. Thus, our use of a fixed λ should be understood as a numerical stabilization for the finite resolutions considered in our experiments, rather than as a choice guaranteeing the same asymptotic limit under arbitrarily fine refinement. We next quantify its practical effect at the resolutions relevant to our experiments.
3) Experimental study on the effect of the ridge coefficient
We have trained PINOT on the 1D Burgers dataset without ridge regularization, using 256 uniform patches with 256 source points per patch. At inference time, we have varied the patch scale h and ridge coefficient λ, while keeping 256 source points per patch. Each entry in the table below reports the relative L1 difference (%) from the corresponding zero-ridge prediction under the same partition and source-point discretization.
For our chosen ridge coefficient of λ=10−4, the prediction discrepancy remains below 0.01% for all tested patch scales with h≥1/64. More than 99.5% of the patches in each of our experimental datasets fall within this scale range, supporting the conclusion that the practical effect of ridge regularization is negligible for the overwhelming majority of patches encountered in our experiments.
However, as anticipated in your concern, at finer patch scales, the effect becomes more pronounced. For example, at h=1/256, the discrepancy reaches 65.9% for λ=10−2, while it is 0.7704% for our choice of λ=10−4. This confirms that the ridge coefficient must be chosen sufficiently small relative to the patch scale. Moreover, for a fixed patch scale, the measured discrepancy scales approximately linearly with λ, consistent with the regularization-dependent term λh−K in our theoretical error bound.
We are revising the manuscript to incorporate the experimental results and the accompanying discussion.