acceptodds
Under review as a conference paper at ICLR 2027

Geometry Conditioning in Neural Operators through Representation Alignment

Abstract

Neural operators provide efficient surrogates for physical simulation by learning mappings from problem specifications to solution fields. However, effectively incorporating geometric information into these models often relies on specialized architectural mechanisms, tightly coupling geometry handling with the underlying operator design. In this work, we explore a different perspective: geometry can guide operator learning independently of the operator architecture. We introduce GEometry-aware Alignment of Representations (GEAR), a lightweight regularization framework that treats geometry as a relational training prior. Rather than matching representations to predefined geometric embeddings, GEAR aligns their local relations with those defined by geometry, enabling lightweight geometric guidance directly during training. To balance geometric guidance with the physical prediction objective, GEAR suppresses alignment updates that conflict with the physical prediction objective, allowing geometric structure to guide learning without over-constraining task-specific features. This decouples geometry-aware learning from architectural design and enables the same training strategy to be applied across neural operator backbones. Experiments across multiple operator architectures and simulation datasets involving varying geometries demonstrate consistent improvements in predictive performance.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Neural Operators, Representation Learning, Surrogate, Scientific Machine Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_UCcjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces GEAR (Geometry-aware Alignment of Representations), an architecture-agnostic training framework for point-wise neural operators.

Critical strengths
  1. Comprehensive Ablation Studies: The authors perform extensive ablation experiments isolating each core design choice.

  2. Grounded Mathematical Formulations: The paper provides rigorous derivations in Appendix B supporting its core algorithmic steps.

Critical weaknesses
  1. While the paper claims improved optimization step efficiency (7.4%–27.4% fewer steps), it omits absolute training wall-clock times (CPU+GPU) in the primary text. A neural operator training pipeline includes data loading, data device transfer, candidate neighborhood searching, local pG/pHp^G/p^H similarity calculations, KL divergence evaluation, and intermediate gradient projection. Reporting normalized ratios only in Appendix Table 5 (which shows shallow models actually take 28%–38% longer wall-clock time) obscures the true computational overhead. To make efficiency claims reliable, the authors should report full wall-clock execution profiles broken down by individual computational components. In particular this algorithm adds a huge computation outside of the vanilla model, the number reported seems little bit questionable.

  2. The paper restricts its experimental comparisons to a fixed 200-epoch budget across all models. In engineering simulation, inference speed and accuracy is paramount, while offline training time is a secondary consideration. If vanilla Transolver can reach or exceed GEAR's accuracy by simply training for 400 or 600 epochs (allowing its unconstrained representations to naturally find optimal physical modes), the benefit of GEAR becomes questionable. An ablation study comparing the well-trained model of both methods is necessary to determine whether GEAR genuinely pushes the asymptotic accuracy limit.

  3. The authors argue that vanilla Transolver fails to maintain geometric structure because its KL divergence with respect to pGp^G increases over training epochs (Figure 5). However, pGp^G is constructed using hand-designed heuristics (combining spatial Euclidean distance, boundary distance, and normal orientation vectors). For instance, in volume meshes in fluid mechanics / structure mechanics, Euclidean spatial proximity is often sufficient to capture local physics. Forcing intermediate states to align with arbitrarily weighted boundary or orientation metrics (DijGD_{ij}^G) imposes an artificial prior. An increasing KL divergence in vanilla models does not necessarily imply that the latent representations are "degrading" or "bad".

  4. The paper lacks a sensitivity analysis or clear practitioner guidelines on how to tune these parameters for a novel geometry or PDE dataset. Without a standardized tuning protocol, applying GEAR to new engineering domains requires extensive trial-and-error hyperparameter search, diminishing its practical usability.

Questions for authors

Please see my Critical Weaknesses for questions.

Reviewer_QJLaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes an alignment mechanism called GEAR to enforce geometric constraints in a neural operator training process. Specifically, in this mechanism one calculates two distributions for each query point. One of them is the similarity distribution of hidden states for nearby points at each query point, the other one is the discrepancy distribution of geometric properties for the same nearby points at the same query point. One then calculates the KL divergence between these two distributions, and define it as the alignment loss. The alignment loss is combined with the target loss to give out the final loss. In order to make sure the alignment loss helps but not hurts the training progress, this work also suggests an adaptive geometric guidance. In adaptive geometric guidance, one can dynamically change the weight of gradients coming from alignment loss. In extensive experiments, the mechanisms introduced in this work are combined with recent neural operators and show some improvement in their performance.

Critical strengths
  1. Clarity. Despite some of the unexplained terms (see critical weaknesses), the paper is well-written and easy to follow.
  2. Extensive experiments. This work includes experimental results on a wide range of datasets, with neural operators from different categories in comparison.
Critical weaknesses
  1. Only small subsets of datasets are used in experiments. In appendix A1, the authors mention that only 100 samples are used for training models, which means the results are not convincing to the reviewer.
  2. Some of the key technical terms are not explained clearly. For example, in lines 188-189 there is one term G denoting the geometric discrepancy between two points, but this term is not defined. In line 257 there is one term s_e suddenly appearing without any definition.
Questions for authors
  1. One of the key concerns the reviewer has is that, does this constraint of matching the two distributions really make sense? The intermediate latent vectors in the model already fuse the information other than geometry, like boundary conditions and working conditions. In backbones like transolver, these latent vectors even exchange information with each other, and make the latent vectors not limited to the representation of the query point any more. All of these concerns could hurt the necessity of this mechanism.
  2. Another concern is the inclusion of FFNO in the benchmark. FFNO is proposed to solve problems on regular grids. The reviewer is curious how this method can be applied to solve the problems with irregular meshes mentioned in this paper, and questions the fairness of the inclusion of this method.
  3. Related to point 1 in critical weaknesses, why do you just use a very small part of all datasets for training and testing?
  4. Related to point 2 in critical weaknesses, although the reviewer finds the definition of s_e in the appendix, the reviewer does not find how G is defined in this manuscript.
Reviewer_FsQ3OpenReview ↗
Rating
3
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

GEAR uses geometric relations to guide intermediate features in neural operators during training. It filters gradients that conflict with the prediction task and gradually removes geometric guidance. The inference architecture remains unchanged. Experiments cover five engineering datasets and five main backbones. Most comparisons show improvements, including a reduction in Transolver's AirCraft error. The paper also studies alignment depth, guidance schedules, and training cost.

Critical strengths
  1. The method improves several backbones across different engineering tasks without adding inference cost. The comparisons are extensive and the results are encouraging.

  2. Table 2 shows why geometric guidance should be reduced during training. Appendix D.1 also clearly reports the difference between training-step savings and actual runtime savings.

Critical weaknesses
  1. The contribution of each training component is not clear. Table 2 studies guidance schedules, but does not isolate gradient filtering or compare the controller with a simple decaying alignment weight. The related work should also discuss MASIV: Toward Material-Agnostic System Identification from Videos, which uses geometric guidance for learning physical models, and Relational Knowledge Distillation and Similarity-Preserving Knowledge Distillation, which transfer relations between feature representations. Please clarify how GEAR's feature alignment and adaptive guidance differ.

  2. Some implementation details and uncertainty estimates are missing. The geometric discrepancy, cue weights and scales, candidate sampling, and controller settings are not fully specified. The main comparisons use around 100 training cases and 20 to 50 test cases, but do not report variation across runs. This makes the method harder to reproduce and leaves the reliability of smaller gains unclear.

Questions for authors
  1. Are there existing ablations without gradient filtering or with a simple decaying alignment weight?
  2. Please specify the geometric discrepancy, candidate sampling, alignment budget, temperature, and controller settings. Were these fixed across datasets and backbones?
  3. Were the main comparisons repeated across seeds or splits? Can you report variation from the existing runs or test cases?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.