acceptodds
Under review as a conference paper at ICLR 2027

PINOT: Partition-Invariant Neural Operator Transformer for Flexible Latent Tokens

Abstract

To reduce computational cost of global processing, several recent neural operators compress physical observations into a smaller set of latent tokens. Using fewer tokens lowers computation, but excessive compression can discard local spatial information, creating a trade-off between predictive accuracy and efficiency. It is therefore desirable to flexibly adapt the partitioning of physical observations into latent tokens according to the problem. This flexibility is particularly useful for complex geometries and nonuniform discretizations, where geometry, observation density, and local spatial variation can differ substantially across regions, making a fixed latent configuration inefficient. However, existing latent-token-based neural operators typically rely on prescribed latent configurations and do not explicitly ensure consistency under changes in latent tokenization. We introduce PINOT, a Partition-Invariant Neural Operator Transformer for flexible latent partitioning. PINOT uses patchwise scale-normalized polynomial coefficients for patch-scale-consistent latent tokens while preserving higher-order local information. Measure-weighted attention further makes global processing consistent with changes in patch density by accounting for the physical measure represented by each token. We theoretically establish partition and discretization consistency together with local expressivity of PINOT. Experiments show that PINOT generalizes to unseen patch resolutions and preserves substantially more spatial information with far fewer latent tokens. Based on these properties, PINOT demonstrates strong predictive performance against representative neural operator competitors, achieving the lowest error on all benchmarks with nonuniform discretizations.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Neural operators, Transformers, Partial differential equations, Patch-partition invariance, Discretization invariance, Adaptive patch tokenization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_NeMEOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PINOT is a neural operator designed to support changes in spatial patch partitioning and source discretization. It encodes local observations through scale-normalized polynomial coefficients, processes patch tokens using measure-weighted attention, and reconstructs outputs through local polynomial expansions. The paper establishes asymptotic consistency under admissible joint refinement and retention of local derivative information. Experiments across six PDE benchmarks demonstrate generalization to unseen resolutions, improved reconstruction with fewer tokens, and competitive accuracy and efficiency, particularly on nonuniform discretizations.opod data.

Critical strengths

The paper identifies a meaningful distinction between source-discretization consistency and consistency under changes in latent partitioning. The architecture addresses both local representation scaling and the influence of unequal patch measures on attention. This provides a coherent design for changing latent resolution without retraining, supported by conditional theoretical results and resolution-transfer experiments. The empirical evidence covers complementary aspects of the proposed design. Reconstruction experiments assess local information retention, while PDE prediction and runtime measurements assess practical usefulness. The strong results on nonuniform benchmarks support the value of combining local polynomial representations with adaptive token allocation.

Critical weaknesses

The consistency guarantees do not directly cover the implemented coefficient estimator. Appendix C assumes full-basis ordinary least squares with uniformly stable Gram matrices. Appendix F.2 instead uses ridge regularization and drops polynomial modes in sparsely sampled patches. These changes matter because normalization multiplies degree-m coefficient errors by h^{-m}, potentially amplifying regularization bias and observation noise as patches shrink. The paper does not establish when the implemented estimator retains the claimed limit or measure its stability in this regime.

The experiments do not sufficiently isolate the mechanisms responsible for partition robustness. The paper compares polynomial bases and adaptive partitions, but does not independently remove scale normalization or measure weighting. In particular, the quantitative patch-resolution experiment uses uniform 1D partitions, where all patch measures are equal and measure-weighted attention reduces to ordinary attention. It therefore cannot validate the proposed correction for unequal partition density. The nonuniform benchmark comparisons also combine architectural changes with adaptive partitioning available only to PINOT.

Questions for authors
  1. What ridge coefficient is used in Equation (178)? Is it fixed or scale-dependent, and under what conditions does the implemented solver retain the consistency established for ordinary least squares?

  2. How frequently is rank limiting activated on the nonuniform benchmarks, and which polynomial orders remain available in those patches?

  3. Do existing experiments separately evaluate removal of scale normalization or measure weighting? If not, please clarify which empirical conclusions concern the complete architecture rather than individual components.

6 comments
AuthorsOpenReview ↗

Response to Reviewer NeME (1/4)

Dear Reviewer NeME,

We sincerely appreciate your sharp and insightful feedback. Below, we address your concerns in detail. Please note that we are currently revising the manuscript to incorporate additional results and discussions in this response, and we will upload the revised version as soon as it is complete.


W1-1, Q1: Ridge regularization

1) Ridge coefficient used in our experiments

We used a fixed ridge coefficient of λ=10−4\lambda=10^{-4} in the normalized least-squares system in Eq. (178), independent of the patch scale. To address your concern, we have additionally 2) derived sufficient conditions under which the ridge-regularized implementation preserves the ordinary least-squares continuum limit, and 3) experimentally quantified the prediction discrepancy induced by nonzero λ\lambda to assess the practical effect of ridge regularization under our experimental settings.

2) Sufficient conditions for retaining the ordinary least squares limit

We have analyzed a theoretical finite-resolution error bound for the discrepancy between the prediction of our ridge-regularized implementation and the corresponding unregularized continuum limit. Under the assumptions of uniform hypercubic patches, uniformly distributed source points, and Lipschitz stability of the network layers, we have derived the following bound on the spatial L∞L^\infty error:

∣∣u^fin(λ)−u^lim∣∣L∞≤Cnet(h+h nmin⁡−1/d+λh−K).|| \hat u_{\mathrm{fin}}^{(\lambda)}-\hat u_{\mathrm{lim}} ||_ {L^\infty}\le C_{\mathrm{net}}\left(h+h \ n_{\min}^{-1/d}+\lambda h^{-K}\right).

Here, hh is the patch scale; nmin⁡n_{\min} is the minimum number of source points per patch; dd is the spatial dimension; KK is the maximum total degree of the polynomial basis; λ\lambda is a ridge regularization coefficient introduced as a numerical safeguard for coefficient fitting at finite resolutions; and CnetC_{\mathrm{net}} is a model-dependent constant independent of hh, nmin⁡n_{\min}, and λ\lambda.

A detailed derivation and proof of this finite-resolution error bound will be included in the revised manuscript.

The first two terms characterize finite-resolution effects, which are discussed in our response to Reviewer A4ZX (W2). Here, we focus on the last term λh−K\lambda h^{-K}, which captures the contribution of ridge regularization. A sufficient condition for retaining the ordinary least squares continuum limit is λh−K→0\lambda h^{-K}\to0, together with the vanishing of the other finite-resolution terms. We acknowledge that a fixed positive ridge coefficient, as used in our implementation, does not satisfy this sufficient condition under arbitrary refinement. Thus, our use of a fixed λ\lambda should be understood as a numerical stabilization for the finite resolutions considered in our experiments, rather than as a choice guaranteeing the same asymptotic limit under arbitrarily fine refinement. We next quantify its practical effect at the resolutions relevant to our experiments.

3) Experimental study on the effect of the ridge coefficient

We have trained PINOT on the 1D Burgers dataset without ridge regularization, using 256 uniform patches with 256 source points per patch. At inference time, we have varied the patch scale hh and ridge coefficient λ\lambda, while keeping 256 source points per patch. Each entry in the table below reports the relative L1L^1 difference (%) from the corresponding zero-ridge prediction under the same partition and source-point discretization.

Patch scaleλ=10−2\lambda=10^{-2}λ=10−3\lambda=10^{-3}λ=10−4\lambda=10^{-4}λ=10−5\lambda=10^{-5}λ=10−6\lambda=10^{-6}
h=1/16h=1/160.5670.05750.005760.0005760.0000576
h=1/32h=1/320.5320.05370.005380.0005380.0000538
h=1/64h=1/640.8090.08290.008320.0008320.0000832
h=1/128h=1/1286.0060.5780.05810.005810.000581
h=1/256h=1/25665.97.6560.77040.07760.00777

For our chosen ridge coefficient of λ=10−4\lambda=10^{-4}, the prediction discrepancy remains below 0.01% for all tested patch scales with h≥1/64h\ge1/64. More than 99.5% of the patches in each of our experimental datasets fall within this scale range, supporting the conclusion that the practical effect of ridge regularization is negligible for the overwhelming majority of patches encountered in our experiments.

However, as anticipated in your concern, at finer patch scales, the effect becomes more pronounced. For example, at h=1/256h=1/256, the discrepancy reaches 65.9% for λ=10−2\lambda=10^{-2}, while it is 0.7704% for our choice of λ=10−4\lambda=10^{-4}. This confirms that the ridge coefficient must be chosen sufficiently small relative to the patch scale. Moreover, for a fixed patch scale, the measured discrepancy scales approximately linearly with λ\lambda, consistent with the regularization-dependent term λh−K\lambda h^{-K} in our theoretical error bound.

We are revising the manuscript to incorporate the experimental results and the accompanying discussion.

AuthorsOpenReview ↗

Response to Reviewer NeME (2/4)

W1-2, Q2: Rank limiting (dropping higher-order polynomial modes)

1) Necessity and mechanism of rank limiting

Although we fixed the maximum total degree of the polynomial basis, some patches contain too few source points to uniquely estimate the coefficients of all polynomial modes. Specifically, when the number of source points is smaller than the number of basis modes, the corresponding least-squares system becomes underdetermined. For example, four cubic-polynomial coefficients cannot be uniquely determined from only two observations without imposing additional assumptions. Therefore, for patches with an insufficient number of source points, we have retained the largest set of polynomial modes that can be permitted by the available observations, dropping higher-order modes first.

2) Retained polynomial basis modes

The following table reports the distribution of the number of retained polynomial modes across patches. The full basis consists of 6 modes for NACA0012 and RAE2822, and 10 modes for DeepJEB. A small fraction of patches retain fewer modes because of insufficient source-point counts within each patch.

Number of retained modesNACA0012RAE2822DeepJEB
11.2%1.4%1.1%
21.0%1.0%1.0%
31.2%1.1%1.0%
41.8%1.7%1.0%
52.5%2.5%1.2%
692.3%92.3%1.5%
7——1.8%
8——2.0%
9——2.1%
10——87.2%

3) Frequency of rank limiting activation

The fraction of patches for which rank limiting is activated corresponds to the fraction retaining fewer than the full set of polynomial modes (6 for NACA0012 and RAE2822, and 10 for DeepJEB), as summarized below.

NACA0012RAE2822DeepJEB
Rank limiting activated7.7%7.7%12.8%

Thus, the full polynomial basis is retained for the large majority of patches in all three datasets. In practical simulation settings, since source-point density may correlate with local geometric or field complexity, sparsely sampled regions may sometimes require less higher-order local detail. This offers one possible explanation for why dropping higher-order modes in a relatively small fraction of patches may have only a limited effect on overall prediction accuracy. However, we emphasize that our conclusion regarding the practical impact of rank limiting is based on the empirical results below rather than on this interpretation. To directly assess the practical impact of rank limiting, we have further evaluated PINOT while deliberately increasing the frequency of rank-limiting activation, as described below.

4) Experimental investigation of the effect of rank limiting

Since the full polynomial basis cannot be uniquely estimated when the number of source points is insufficient, rank limiting cannot be entirely eliminated. Instead, we have investigated its effect by intentionally increasing the frequency of rank limiting, even in patches where retaining additional polynomial modes would otherwise be mathematically feasible. Specifically, we have trained PINOT on NACA0012 using the original rank-limiting strategy, which drops higher-order modes when the number of source points in a patch is smaller than the number of coefficients MM required to retain those modes. At inference time, we have increased this requirement from MM to M+cM+c, where cc is an additional parameter controlling the frequency of rank-limiting activation. We have varied cc from 0 (the original setting) to 4.

Additional point requirement ccOriginal (0)1234
Rank limiting activated (%)7.711.014.919.624.7
Relative L1L^1 error (%)5.725.765.906.156.69

The table above shows that, even when rank limiting is deliberately activated in a substantially larger fraction of patches, the degradation in predictive accuracy remains relatively moderate over the tested range. These results indicate that the original rank-limiting strategy, which affects only 7.7% of patches on NACA0012, has a limited practical impact on overall predictive performance.

We are revising the manuscript to incorporate the experimental results and the accompanying discussion.

AuthorsOpenReview ↗

Response to Reviewer NeME (3/4)

W1-3: Observation noise

1) Sensitivity to observation noise

Yes, we acknowledge that PINOT trained exclusively on clean input can become sensitive to observation noise at inference time, particularly as the patch scale decreases. This tendency is evident from the substantial increase in normalized relative L1L^1 error (%) at h=1/64h=1/64 as the noise level increases, as shown in the table below.

Patch scaleNoise: 0.1%Noise: 0.5%Noise: 1%Noise: 5%Noise: 10%
h=1/16h=1/164.964.964.964.985.04
h=1/32h=1/321.251.271.302.043.32
h=1/64h=1/640.701.392.4812.7027.23

In this experiment, we have trained PINOT on clean data and evaluated it with noisy input at different patch scales using the 1D Burgers dataset with 65,536 source points. Here, hh denotes the patch scale, and the noise level (%) denotes the standard deviation of additive Gaussian noise relative to the standard deviation of the input data.

2) Robustness with noisy-input training

As you pointed out, this high sensitivity to observation noise is closely related to our scale normalization, which rescales polynomial coefficients by 1/[g(∣α∣)h∣α∣]1/[g(|\alpha|)h^{|\alpha|}], as defined in Eq. (5) of the originally submitted manuscript. Here, α\alpha is a polynomial multi-index, and ∣α∣|\alpha| is its total degree. Since higher-order coefficient estimates can be particularly sensitive to observation noise, the factor h−∣α∣h^{-|\alpha|} can substantially amplify noise-induced perturbations as the patch scale decreases.

Since observation noise is expected in many practical measurement settings, we have further evaluated PINOT under noise-aware training. Specifically, for each noise-level and patch-scale configuration, we have trained PINOT using noisy input and clean output, and evaluated the model under the corresponding condition. This setting also allows the learnable factor g(∣α∣)g(|\alpha|) to adapt during training in the presence of noisy observations. The following table reports the resulting normalized relative L1L^1 error (%).

Noise: 1%Noise: 5%Noise: 10%
h=1/16h=1/160.671.001.14
h=1/32h=1/320.790.800.72
h=1/64h=1/640.340.400.50

Compared with clean-input training, noisy-input training substantially improves prediction accuracy under noisy observations across all tested patch scales. Notably, even at the finest patch scale of h=1/64h=1/64, the error remains at or below 0.50% for noise levels up to 10%. These results demonstrate that incorporating observation noise during training can effectively mitigate the noise sensitivity observed under clean-input training.

We are revising the manuscript to incorporate the experimental results and the accompanying discussion, together with our theoretical analysis of how the learnable factor g(∣α∣)g(|\alpha|) can contribute to mitigating observation noise during noisy-input training.

Reviewer_A4ZXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes PINOT, a latent-token neural operator for PDE surrogate modeling that aims to be consistent under changes in both source discretization and latent patch partitioning. Each patch is encoded via least-squares fitting of lifted features onto a shared polynomial basis (Legendre/Chebyshev), with coefficients scale-normalized by h^|α| to remove leading-order patch-size dependence. A measure-weighted transformer adds log-measure terms to attention logits so that subdividing a region does not inflate its influence, and a continuous local expansion decoder reconstructs the field at arbitrary target points. The authors prove asymptotic consistency under admissible joint refinement (Theorem 1) and local expressivity of the coefficient representation up to order K (Theorem 2, strictly more informative than GNO-style aggregations). Experiments show generalization to unseen patch resolutions (vs. GAOT), much better input reconstruction with ~8× fewer tokens, benefits of adaptive partitioning, and the lowest error on 5 of 6 PDE benchmarks, including all three with nonuniform discretizations.

Critical strengths
  1. Well-motivated and principled design. Each architectural component directly targets a specific failure mode: polynomial coefficients preserve higher-order local structure, h^|α| normalization removes patch-scale dependence, and log-measure attention corrects partition-density bias. The design–theory–experiment correspondence is unusually tight.

  2. Theoretical grounding. Consistency under joint refinement and the expressivity result (including strict dominance over scale-independent GNO aggregations) formalize properties that prior latent-token operators (GAOT, GINO, Transolver, LNO) lack or leave implicit.

Critical weaknesses
  1. Fairness of the main comparison. In Table 4, adaptive partitioning is applied only to PINOT on nonuniform benchmarks, while competitors use fixed configurations; some of the gain may come from the adaptive tokenization rather than the architecture itself. A PINOT-nonadaptive row across all benchmarks (Table 3 covers only NACA0012) would strengthen the claim.

  2. Asymptotic rather than finite guarantees. The "invariance" in Theorems 1–2 is consistency in the refinement limit; the paper gives no rates or finite-partition error bounds, so the gap between theory and the practical coarse-partition regime is unquantified. The term "partition-invariant" in the title slightly oversells an asymptotic property.

  3. Dependence on smoothness assumptions. The Taylor-expansion-based scale normalization presumes local smoothness of the lifted field; behavior near shocks/discontinuities (relevant for CE-RP, where PINOT loses to GAOT) is not analyzed, and the CE-RP result hints at this limitation without deeper discussion.

  4. Partition scheme is heuristic. The point-density-adaptive quadtree/octree partitioning adapts to sampling density, not solution complexity; error- or feature-adaptive partitioning is unexplored, as the authors acknowledge. Sensitivity to K and the point-count threshold is relegated to appendices.

Questions for authors

refer to weaknesses

5 comments
AuthorsOpenReview ↗

Response to Reviewer A4ZX (1/3)

Dear Reviewer A4ZX,

We greatly appreciate your careful and constructive feedback. Below, we address your concerns in detail. Please note that we are currently revising the manuscript to incorporate additional results and discussions in this response, and we will upload the revised version as soon as it is complete.


W1: Fairness of the main comparison

Following your suggestion, we have evaluated PINOT with nonadaptive partitioning to disentangle the contributions of its architecture and adaptive partitioning. The relative L1L^1 error (%) is summarized below.

ConfigurationNACA0012RAE2822DeepJEB
GINO (nonadaptive)9.688.6331.8
OFormer (does not use latent tokens)8.7813.5
GAOT (nonadaptive)11.912.730.9
Transolver (learnable latent space)6.445.5323.5
LNO (learnable latent space)7.616.9227.3
PINOT-adaptive5.725.3519.4
PINOT-nonadaptive6.187.4926.0
PINOT-nonadaptive (repeated encoding–decoding + subsampling)5.725.3623.7

We omit OFormer on DeepJEB because its official implementation lacks 3D support and attention over all source points is prohibitively memory-intensive.

1) Contribution of PINOT’s architecture independent of adaptive partitioning

Even without adaptive partitioning, PINOT consistently outperforms GINO and GAOT, which also employ nonadaptive partitioning, across all three datasets. This demonstrates that PINOT's performance gains are not attributed solely to adaptive partitioning and that its coefficient-based architecture provides benefits independently of adaptive partitioning.

Meanwhile, adaptive partitioning further improves PINOT's performance, indicating that the improvements reported in Table 4 (of the originally submitted manuscript) arise from the combined contributions of its architecture and adaptive partitioning.

2) Distinction between adaptive partitioning and learnable latent spaces

Although PINOT with nonadaptive partitioning does not consistently outperform Transolver and LNO, this comparison should not be interpreted as an ablation in which latent-token adaptivity is removed from all models. In particular, Transolver and LNO already incorporate adaptive token construction through learned source-to-token assignments, allowing the contribution of each source point to latent tokens to vary with the input. Thus, comparing nonadaptive PINOT against these methods is not equivalent to comparing models under uniformly nonadaptive tokenization.

The key distinction is that their adaptivity is realized through learned assignments within a latent space whose token count and overall configuration remain fixed, whereas PINOT enables adaptivity through explicit geometric partitioning whose number, size, and spatial arrangement of latent tokens can vary across input samples. In this context, the nonadaptive-PINOT comparison isolates the effect of removing PINOT’s geometric partition adaptivity, but does not remove adaptive token formation from the competing methods.

3) Further improvement through repeated encoding-decoding

Additionally, we have found that a simple modification to PINOT’s architecture, corresponding to the last row of the table above, further improves its performance, enabling PINOT even without adaptive partitioning to achieve performance higher than (or comparable to) Transolver and LNO. Specifically, we have replaced the original [encoder–processor (8 attention layers)–decoder] structure with 4 sequential [encoder–processor (2 attention layers)–decoder] blocks, keeping the total number of attention layers fixed at 8. For computational efficiency, we have also applied random subsampling to limit the maximum number of source points per patch, as nonadaptive partitioning can result in patches containing an exceptionally large number of points.

We are revising the manuscript to incorporate the table and the accompanying discussion into the manuscript.

AuthorsOpenReview ↗

Response to Reviewer A4ZX (2/3)

W2: Finite guarantees of invariance

Following your suggestion, we have newly provided 1) a theoretical error bound that quantifies the discrepancy between PINOT's finite-resolution prediction and its corresponding continuum prediction, and 2) experimental results demonstrating convergence under finite-resolution settings. We have also 3) clarified the meaning of “partition invariance” to avoid potential misunderstanding.

1) Theoretical error bound

Under the assumptions of uniform hypercubic patches, uniformly distributed source points, and Lipschitz stability of the network layers, we have derived the following bound on the discrepancy between PINOT’s finite-resolution prediction and the corresponding continuum limit:

∣∣u^fin(λ)−u^lim∣∣L∞≤Cnet(h+h nmin⁡−1/d+λh−K).|| \hat u_{\mathrm{fin}}^{(\lambda)}-\hat u_{\mathrm{lim}} ||_ {L^\infty}\le C_{\mathrm{net}}\left(h+h \ n_{\min}^{-1/d}+\lambda h^{-K}\right).

Here, hh is the patch scale; nmin⁡n_{\min} is the minimum number of source points per patch; dd is the spatial dimension; KK is the maximum total degree of the polynomial basis; λ\lambda is a ridge regularization coefficient introduced as a numerical safeguard for coefficient fitting at finite resolutions; and CnetC_{\mathrm{net}} is a model-dependent constant independent of hh, nmin⁡n_{\min}, and λ\lambda.

A detailed derivation and proof of this finite-resolution error bound will be included in the revised manuscript.

2) Convergence experiment

Although the bound characterizes how the finite-resolution discrepancy depends on hh, nmin⁡n_{\min}, and λ\lambda, it does not directly provide a numerical estimate because CnetC_{\mathrm{net}} is not explicitly quantified. Therefore, we have further evaluated the convergence behavior of PINOT on the 1D Burgers equation. Specifically, we have trained a single PINOT model with K=3K=3 using 256 patches and 256 points per patch. We have set λ=0\lambda=0 to isolate the effects of patch scale and source-point sampling (please refer to our response to Reviewer NeME (W1) for the finite-resolution effects of ridge regularization that are assessed separately). We have then evaluated the same trained model, without retraining, under different configurations of patch scale hh and the number of source points per patch nmin⁡n_{\min}. The table below reports the relative L1L^1 difference (%) with respect to the finest reference prediction (nmin⁡=256n_{\min}=256, h=1/256h=1/256).

hh / nmin⁡n_{\min}48163264128256
1/161/167.385.364.994.914.904.894.90
1/321/321.311.131.091.081.071.071.07
1/641/640.370.300.270.270.270.270.26
1/1281/1280.140.100.090.080.080.080.08
1/2561/2560.050.020.010.000.000.000.00

As shown in the table, the relative L1L^1 discrepancy generally decreases as hh decreases and nmin⁡n_{\min} increases. In particular, the discrepancies become consistently small at sufficiently fine resolutions. These results empirically support the finite-resolution convergence behavior characterized by our theoretical bound.

3) Clarification of “partition invariance”

Although our use of “partition invariance” follows the notion of “discretization invariance” commonly used in neural operators (Kovachki et al., 2023), referring to asymptotic convergence under discretization refinement (rather than exact equality at arbitrary finite discretizations), we agree that this terminology may be misinterpreted as implying strict invariance across arbitrary finite partitions. We will therefore revise our manuscript to explicitly clarify that the terminology refers to asymptotic convergence under refinement, rather than identical predictions across arbitrary finite partitions.

We are revising the manuscript to incorporate the experimental results and the accompanying discussion into the manuscript.

AuthorsOpenReview ↗

Response to Reviewer A4ZX (3/3)

W3: Dependence on smoothness assumptions

1) Effect of discontinuities

Yes, our scale-normalization analysis relies on local smoothness of the lifted input field, and discontinuities can degrade the accuracy of local polynomial approximation. To empirically investigate this effect, we have constructed a controlled 1D Darcy dataset with 4–8 discontinuities in the input coefficient field and have trained PINOT on this dataset. The table below reports the relative L1L^1 error (%) of the predicted solution, evaluated over the entire domain and in the vicinity of input discontinuities.

Partitioning strategyAverage number of patchesEntire domainVicinity of discontinuities (±1%)
Discontinuity-unaware partitioning (current)320.3460.579
Discontinuity-aware partitioning23.70.2330.411

As shown in the table, the current discontinuity-unaware partitioning exhibits higher error near discontinuities than over the entire domain, consistent with your concern.

2) Effect of discontinuity-aware partitioning

Nonetheless, it is worth noting that this limitation can be substantially mitigated by discontinuity-aware partitioning. Specifically, this strategy first places patch boundaries at discontinuities and then subdivides the resulting smooth regions. As shown in the table above, discontinuity-aware partitioning substantially reduces the error both over the entire domain and near discontinuities, while using fewer patches on average. These results indicate that accounting for input discontinuities can effectively mitigate the error associated with patches that cross nonsmooth regions.

We are revising the manuscript to incorporate the table and the accompanying discussion into the manuscript.


W4: Partitioning strategy

1) Need for advanced adaptive partitioning strategies

Yes, as acknowledged in the originally submitted manuscript, we agree that our current point-density-based partitioning does not explicitly account for field complexity. The discontinuity-aware partitioning presented above offers one potential approach for incorporating field characteristics into partitioning decisions. Developing more general error- or feature-adaptive partitioning strategies constitutes a substantial research direction in its own right, as illustrated by AMR-Transformer (Xu et al., 2025) and MeshTok (Zhao et al., 2026). As these previous methods were primarily designed for structured-grid input data, they do not directly support irregular meshes, nor do they establish theoretical output consistency under changes in partitioning.

In contrast, PINOT supports irregular meshes while providing partition consistency under refinement, offering a flexible backbone for incorporating more sophisticated adaptive partitioning strategies. However, we leave the design of advanced adaptive partitioning strategies to future work, as the present study focuses on establishing a theoretically grounded neural operator that can accommodate a broad range of externally specified partitioning strategies.

2) Sensitivity to KK and the point-count threshold

We have evaluated the sensitivity of PINOT to the maximum polynomial degree KK and the point-count threshold on the NACA0012 dataset. The relative L1L^1 error (%) is summarized below.

Relative L1L^1 error (%)Threshold 40Threshold 80Threshold 120
K=0K=018.9724.2727.57
K=1K=15.556.127.50
K=2K=25.726.156.31
K=3K=36.516.336.50

The results show moderate sensitivity to both KK and the point-count threshold, except for the substantial performance gap between K=0K=0 and K≥1K\geq 1. The significant improvement from K=0K=0 to K=1K=1 highlights the importance of retaining first-order local information; this is consistent with Corollary 13 in the manuscript, which establishes the greater informativeness of PINOT’s coefficient representation relative to the analyzed class of local GNO aggregations. However, further increasing KK does not necessarily improve performance, as higher-order coefficient estimation can become unstable in patches containing a limited number of source points, potentially offsetting the benefit of increased representational capacity. Regarding the point-count threshold, lower thresholds lead to finer partitions and generally improved prediction accuracy, although the effect interacts with the choice of KK.

We will incorporate these additional experiments, their visualizations, and a detailed analysis into the revised manuscript before the rebuttal period ends.

References

[1] Nikola Kovachki et al., Neural operator: Learning Maps Between Function Spaces With Applications to PDEs, Journal of Machine Learning Research 2023

[2] Zeyi Xu et al., AMR-Transformer: Enabling Efficient Long-range Interaction for Complex Neural Fluid Simulation, CVPR 2025

[3] Yanshun Zhao et al., MeshTok: Efficient Multi-Scale Tokenization for Scalable PDE Transformers, ICML 2026

Reviewer_BdcxOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PINOT, a latent-token neural operator that represents each patch by scale-normalized polynomial coefficients, processes tokens with measure-weighted attention, and decodes through a continuous local expansion. It proves asymptotic consistency under joint refinement and reports strong results on six PDE benchmarks, especially with adaptive partitioning on nonuniform discretizations.

Critical strengths
  1. The problem is well motivated: existing latent-token operators either fix the latent configuration or do not enforce consistency when the partition changes. The proposed encoder–processor–decoder architecture targets both limitations.

  2. The reconstruction result is promising: with the processor removed, PINOT reaches 2.92% error with about 502 tokens, vs. 15.0%/13.8% for GINO/GAOT with 4,096 tokens. This suggests that the coefficient representation preserves more local information per token.

Critical weaknesses
  1. The reconstruction comparison does not isolate the encoder. PINOT uses about 502 adaptively placed tokens and its own polynomial decoder, while GINO/GAOT use 4,096 uniformly placed tokens and their own decoders. The token-count difference cannot explain PINOT’s lower error; the concern is that token placement and decoder capacity also differ, so a lower reconstruction error could reflect either rather than the coefficient representation. The result is promising, but it does not establish that the lower error is due to the coefficient representation.

  2. The scale-normalization mechanism and measure-weighted attention are not isolated. Table 3 isolates the partition scheme, not the mechanisms, and Figure 7 changes the basis, not the normalization; neither is a direct ablation. A nonadaptive 1,024-token PINOT already reaches 6.18% on NACA0012, lower than the 1,024-token GINO/GAOT runs (9.68%/11.9%), so the comparison does not isolate these mechanisms from the rest of the architecture. Their specific contribution therefore remains unclear.

Questions for authors
  1. Could you compare the PINOT, GINO, and GAOT encoders using the same token budget and spatial locations for all three, under one shared decoder, to isolate the coefficient representation from token placement and decoder capacity?

  2. Could you ablate scale normalization and measure-weighted attention separately on NACA0012, reporting error on the training adaptive partition and error on an unseen uniform partition at the same token budget?

3 comments
AuthorsOpenReview ↗

Response to Reviewer Bdcx

Dear Reviewer Bdcx,

We sincerely appreciate your positive feedback and your recognition of the strengths of our work. Please note that we are currently revising the manuscript to incorporate additional results and discussions in this response, and we will upload the revised version as soon as it is complete.


W1, Q1: Reconstruction experiments

Following your suggestion, we have conducted additional controlled reconstruction experiments designed to isolate the effect of the encoder representation. Specifically, we have used the following common setting for PINOT, GINO, and GAOT:

  • Same token budget: 4,096 latent tokens for all three methods;
  • Same token locations: the latent tokens were placed at the same uniformly spaced spatial locations;
  • Same decoder: all encoders were paired with the same decoder architecture, namely our continuous local expansion decoder.

Under this controlled setting, the relative L1L^1 reconstruction error (%) is reported below.

PINOTGINOGAOT
Relative L1L^1 reconstruction error (%)2.1620.66.33

The results clearly demonstrate the effectiveness of PINOT’s coefficient-based encoder compared with the encoders of GINO and GAOT even when the token budget, token locations, and decoder are held fixed. In other words, this controlled comparison provides direct evidence that PINOT’s coefficient-based encoder preserves substantially more local spatial information than the compared encoder representations.

We are revising the manuscript to incorporate this controlled experiment and the corresponding discussion.


W2, Q2: Isolation of scale-normalization and measure-weighted attention

1) Experimental settings for the ablation study

Following your suggestion, we have conducted additional ablation experiments to isolate the individual contributions of scale normalization and measure-weighted attention under approximately matched token budgets.

The primary role of these two mechanisms is to improve robustness to changes in partition configurations, including changes in patch sizes, spatial arrangements, and token budgets at inference time. Therefore, we believe that each contribution is most directly assessed when the test partition differs from the training partition. In addition to the approximately token-matched setting you suggested (see the first two rows of the following table), we have further evaluated a more challenging setting in which the training and test token budgets differ substantially (see the last row of the following table). The relative L1L^1 error (%) is summarized below, where token counts for adaptive partitions denote averages across samples.

Training partitionTest partitionOriginal PINOTWithout scale normalizationWithout measure-weighted attention
1,045 adaptive1,045 adaptive5.835.925.52
1,045 adaptive1,024 nonadaptive10.9618.6213.65
501 adaptive501 adaptive5.726.295.72
501 adaptive1,024 nonadaptive13.3323.8720.67

2) Contributions of scale normalization and measure-weighted attention

First, the results in the table clearly show that both scale normalization and measure-weighted attention contribute substantially to PINOT's generalization especially when the test partition differs from the training partition (see the second and fourth rows). Their benefits become more pronounced when the training and test token budgets differ substantially (see the fourth row) than when they are approximately matched (see the second row). These controlled ablations provide empirical evidence that both mechanisms play important roles in maintaining robust predictions across changes in partition configuration.

Second, when the training and test settings use the same partitioning strategy and approximately matched token budgets (see the first and third rows), the effects of scale normalization and measure-weighted attention are comparatively small. In contrast, both mechanisms provide substantially larger benefits when the test partition differs from the training partition. These results indicate that scale normalization and measure-weighted attention are particularly important for generalization across different partition configurations, consistent with their intended roles in PINOT.

We are revising the manuscript to incorporate the experimental results and the accompanying discussion into the manuscript.

Thank you for the detailed rebuttal; it addresses most of my concerns. I will maintain my original score, which leans toward acceptance.

AuthorsOpenReview ↗

Thank You for Your Feedback and Summary of Manuscript Revisions

Dear Reviewer Bdcx,

We sincerely appreciate your positive feedback and are pleased to hear that your concerns have been addressed. We would also like to inform you that we have incorporated the reconstruction experiment results and related discussions (W1/Q1) into Section 3.2.2 of the main text. Additionally, the component ablation studies (W2/Q2) have been included in Section 3.2.1 of the main text, with further details provided in Appendix E.3.

Thank you again for your time, thoughtful feedback, and continued positive assessment of our work.

Comments on the paper

2 comments
Area_Chair_KXacOpenReview ↗

Discussion Phase Reminders

As the discussion period is rapidly coming to a close (Nov 18 AoE), I would like to remind the reviewers to please participate in discussion and acknowledge any comments posted by the authors. A secondary, important reminder: full disclosure of any AI usage is required and the reviewer must include their own human-written assessment.

AuthorsOpenReview ↗

Summary of Discussions and Remaining Concerns

Dear Reviewers and Area Chair,

We sincerely thank you all for your time and effort in reviewing our submission.

Given the limited time available during the author–reviewer discussion period, we believe there may not have been sufficient opportunity to fully discuss all concerns. We would therefore be deeply grateful if the remaining issues could receive further consideration during the AC–reviewer discussion period. As our closing remarks, we would like to briefly summarize our discussions with each reviewer.


Discussion with Reviewer Bdcx

Reviewer Bdcx’s primary concerns involved fair and controlled experiments to isolate contributions of individual component. Following our additional experiments and analyses, Reviewer Bdcx kindly acknowledged his/her assessment still leans toward acceptance.


Discussion with Reviewer A4ZX

Reviewer A4ZX raised concerns regarding a) the fairness of experimental comparison settings, b) the gap between theorem and real-world settings involving finite-resolutions and discontinuities, and c) our method’s sensitivity to hyperparameter choices along with the simplicity of our partitioning strategy.

To address these concerns, we provided a) additional experiments under controlled settings, b) the theoretical and empirical analyses of the theorem-reality gap. We also provided c-1) empirical sensitivity to hyperparameters, c-2) discontinuity-aware partitioning as an example of more advanced strategy, along with possible directions for error- or feature-adaptive partitioning.

Unfortunately, we have not yet received Reviewer A4ZX’s opinion on whether our rebuttal adequately addressed his/her concern. We would be grateful if Reviewer A4ZX and the Area Chair could further discuss these points during the AC–reviewer discussion period.


Discussion with Reviewer NeME

Reviewer NeME acknowledged that our rebuttal substantially addressed their concern regarding ablation studies, while maintaining two remaining concerns: (a) the theorem–reality gap and (b) partition-transfer performance.

To address these concerns, we responded that a) the primary role of mathematical analysis is not necessarily to capture every detail, but to provide insights to human, provided that practical deviations remain negligible, as supported by our theoretical and empirical analyses. Regarding partition-transfer performance, we emphasized that b) although finite-resolution effects lead to some increase in prediction error, meaningful spatial structures are preserved, and the degradation is substantially smaller than that observed in existing methods.

We believe this discussion point a) raises an important question about the role of mathematics in the current AI era. We therefore respectfully and earnestly ask the Area Chair to review the final exchange between Reviewer NeME and us.


Final Remark

All discussion points and additional details have been incorporated into our revised manuscript, as summarized below.

We sincerely thank all reviewers and the Area Chair once again for their time, thoughtful feedback, and dedication to the review process. Once again, we would greatly appreciate further consideration of our responses and revised manuscript during the AC–reviewer discussion period.


Summary of Discussion Points and Manuscript Revisions

ReviewerDiscussion pointsManuscript reference
Bdcx, NeMEAblation of scale normalization and measure weightingSection 3.2.1, Appendix E.3.
A4ZX, NeMEAblation of adaptive partitioningTable 4
BdcxControlled reconstruction comparisonSection 3.2.2
A4ZXFinite-resolution error analysisAppendices G.1, G.2
A4ZXClarification of "partition invariance"Section 2.3
A4ZXDependence of smoothness assumptionsAppendix H.2
A4ZXAdvanced partitioning strategiesAppendix H.2
A4ZXSensitivity to K and the point-count thresholdSection 3.2.5
NeMESufficient condition for recovering the ordinary least-squares limitAppendix G.1
NeMEEffect of ridge coefficientAppendix G.3
NeMEEffect of rank limitingAppendix G.4
NeMEEffect of observation noiseAppendix H.1
A4ZX, NeMEArchitectural refinements under nonadaptive partitioningAppendix H.3

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.