acceptodds
Under review as a conference paper at ICLR 2027

Compact Conditional Neural Fields for PDE Surrogates with Depth-Adapted Shared Operators

Abstract

Learning PDE surrogates requires compact models that retain both global sample information and spatially localized context. We introduce a compact conditional neural field architecture in which a task encoder produces distributed query context together with a pooled sample condition, and a coordinate-conditioned decoder reconstructs the physical field. To reduce parameter storage without removing iterative refinement, repeated transformations share a base operator while using lightweight depth-specific low-rank adaptations. This design preserves node- or grid-level information paths while allowing a compact global condition to modulate the field decoder. Across standard PDE benchmarks, the resulting models achieve competitive accuracy with substantially fewer parameters: relative on Elasticity using M parameters, and , , and on Airfoil, Pipe, and Darcy using fewer than M parameters each. Relative to published Transolver-profile parameter counts, our models reduce parameter storage by approximately –, while maintaining errors close to strong published baselines under the reported protocols. Ablations show that compact global conditioning is most valuable when distributed communication is restricted, whereas full token or convolutional context can already carry substantial sample-specific information. The results support a parameter-efficient conditional-field route for PDE surrogate modeling that avoids forcing the solution through a single global bottleneck.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
PDE surrogate modeling, neural operators, coordinate neural fields, conditional neural fields, scientific machine learning, parameter-efficient learning, mesh-based learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_DfCiOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a compact conditional neural-field architecture for PDE surrogate modelling. A task-specific encoder (a graph message-passing encoder with node-to-token attention for meshes, or a multiscale CNN for structured grids) produces a per-node context H and a pooled global code z. A coordinate-conditioned iterative decoder uses both, with FiLM modulation from [z, h_i]. Repeated affine maps (message, token and decoder maps) share one base matrix across depth and add a per-step rank-4 LoRA-style update (as in Relaxed Recursive Transformers), to reduce parameter count.

Critical strengths
  1. The models are competitive at under 1M parameters. Reaching errors in the same range as Transolver on Elasticity/Airfoil/Pipe/Darcy with roughly a third to half of the parameters is a useful data point for practitioners who care about storage.
Critical weaknesses
  1. The evaluation protocol does not support the accuracy-size claims. The headline numbers are validation errors of checkpoints selected on that same validation set. They are then placed next to published test-set results of baselines in Table 1. This is optimistically biased. According to the appendix, for Darcy a test split exists (rows [0,200) of the second source) but is not reported. Where true holdouts are used (ShapeNetCar, AirfRANS), the model is worse than Transolver: velocity +21%, drag +56% on ShapeNetCar; AirfRANS drag error 2.65 with Spearman 0.50.

  2. The contribution is thin, and the paper's own ablations undercut the main design. (a) Pn Elasticity the independent-map model is 24.8% more accurate. The adapted model saves parameters but costs accuracy, and there is no parameter-matched comparison to show that sharing+LoRA is a better use of a parameter budget than simply shrinking the network. (b) Replacing the sample-dependent z with a learned constant even slightly helps, in every configuration that is actually used (token-enabled Elasticity, Airfoil CNN, ShapeNetCar). z only matters in a token-disabled configuration that is never deployed. So the "conditional" part of the title is not supported as a useful design choice.

Questions for authors

Q1. Can the authors report test-split results for all static tasks, at least Darcy where a test split exists, with checkpoints selected only on validation data?

Q2. Can the authors run Transolver with the same reference-geometry / correspondence inputs used by your Elasticity and Airfoil adapters, so the benefit of the architecture can be separated from the benefit of the extra inputs?

Reviewer_B8RtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a compact conditional neural field architecture for PDE surrogate modeling. The method combines distributed spatial context with global conditioning and introduces depth-adapted shared operators to reduce parameter storage while preserving iterative refinement. Experiments across multiple PDE benchmarks demonstrate competitive accuracy with substantially fewer parameters. The ablations further investigate the tradeoffs between parameter sharing, depth-specific adaptation, and different information pathways.

Critical strengths
  1. The proposed architecture achieves competitive performance with fewer than 1M parameters on several PDE benchmarks, demonstrating a promising accuracy–storage tradeoff.
  2. The comparisons between shared, depth-adapted, and independent operators, together with the conditioning ablations, provide useful insights into the relationship between parameter sharing and spatial information flow.
Critical weaknesses
  1. The depth-adapted shared operators closely resemble the combination of recursive weight sharing and layer-specific LoRA explored in Relaxed Recursive Transformers. Although the paper acknowledges this connection, the methodological distinction and PDE-specific contribution beyond adapting existing components remain insufficiently established.
  2. The depth-adapted model reduces parameter count but performs noticeably worse than the independent-operator variant. Without a parameter-matched independent baseline, it remains unclear whether the proposed sharing strategy offers advantages over simply reducing model width. Moreover, comparisons with published methods involve differences in evaluation protocols, limiting conclusions about relative competitiveness.
Questions for authors
  1. Beyond applying recursive weight sharing and depth-specific low-rank adaptation to PDE surrogate modeling, what distinguishes the proposed operators from those in Relaxed Recursive Transformers? Are there any PDE-specific design choices that are essential to the observed performance?
  2. How does the depth-adapted model compare with a narrower independent-operator model under a matched parameter budget? This would help clarify whether the proposed parameterization provides a meaningful advantage beyond parameter reduction.
  3. The ablation results show that sample-dependent conditioning provides little improvement when token communication or CNN context is available, and sometimes constant conditioning performs slightly better. Could the authors clarify the intended role of the pooled condition z? Is z expected to provide complementary information, or mainly compensate for insufficient distributed communication?
  4. The proposed sharing mechanism substantially reduces parameter storage, but the shared operators are still repeatedly executed at each refinement step. Could the authors clarify whether the method provides practical inference-time advantages compared with independent operators, beyond reduced model size?
Reviewer_xWTpOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper introduces an encoder-decoder method for solving PDEs. The encoder can either be based on a CNN for regular grids or a message passing framework for irregular ones. The encoder produces a global conditioning and a local conditioning vector for the decoder. Given the local and global context, the decoder predicts the solution field. The core question of the paper is whether it makes sense to share parameters between layers. The experiments show that the overall model performs close to a larger Transsolver++ architecture. The parameter sharing leads to fewer parameters but also reduces the accuracy.

Critical strengths
  1. The method is evaluated on a diverse set of PDEs and even larger car datasets, and seems to perform similarly to Transsolver++, at least on the simpler settings.
Critical weaknesses

The biggest downside of the paper is that it is very hard to follow, especially the method section. Some examples:

  • The method section goes into too much detail regarding hyperparameters and special cases for the elasticity PDE (the same goes for other sections too, for example, it is unnecessary to list all concrete PDE errors in the abstract, or mention that you don't use AMP or the concrete seed value in the beginning of the experiment section)
  • The depth-adapted shared linear maps are introduced without context or connections to the previous section

This also makes it hard to understand what the methodological novelty is. In general, the paper should be rewritten more in a top-down way, first specifying the overall structure and then going into details. A lot of hyperparameter choices should be omitted from the method section. For a lot of statements, such as the POD teacher or the special case for the elasticity, the reader needs more context.

As a second downside, the number of parameters of a model is not really that interesting. Rather, inference time and memory usage compared to the baselines would be more interesting, especially since Transsolver++ outperforms the proposed model across most experiments

Questions for authors
  1. No POD supervision was mentioned multiple times as an advantage. Which methods actually use this? I am not aware of POD teachers in the context of neural PDE solvers, and the paper did not cite any references for this.
  2. Did you also compare the computational effort (memory, inference speed) to Transsolver++?
  3. Why would we disable communication between tokens/positions? It seems absolutely necessary to do so in order to have a working model.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.