- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces warm-starting PDE solvers with any-dimensional machine learning—a mathematical framework for expressing PDEs across different dimensions. Under this framework, the authors describe explicit conditions under which an dimensional PDE can be learned from an -dimensional machine learning model. For general higher-dimensional initial data that do not satisfy exact-lifting conditions, the paper demonstrates that lower-dimensional pretraining still provides a useful warm start.
Critical strengths
The paper is well written and introduces several abstract mathematical concepts progressively, making it accessible to a broader audience. The overall structure is logical and the narrative is engaging. Furthermore, the method provides a clear mathematical rationale for how parameters should transfer across dimensions.
The proposed framework addresses an important computational challenge: training 3D simulations is computationally demanding, so leveraging easier-to-obtain 2D counterparts is a promising direction. The paper uses numerical experiments (e.g., Navier-Stokes results) to show savings in both training budget and data storage.
Critical weaknesses
-
Exact compatibility is not directly validated experimentally. Although the paper illustrates why lifting should be exact under the stated constraints, it does not report an experiment verifying this property. While mathematically straightforward, an empirical check would confirm that filter conversion and the complete neural network preserve the claimed compatibility. Given its central role, this inexpensive validation would strengthen the link between theory and implementation.
-
Benefits are demonstrated on a single architecture family. All experiments rely on a geometric U-Net implemented in ginjax, leaving it unclear whether the observed improvements extend to other neural network or neural operator architectures. Testing lower-dimensional pretraining on additional models—including architectures with and without explicitly enforced any-dimensional compatibility—would provide stronger evidence for the approach's generality.
Questions for authors
Do the warm-start benefits extend beyond ginjax to other architectures, including those with and without explicitly enforced any-dimensional compatibility?
2 comments
Thank you
Thank you for the response and the additional compatibility experiment, which helps connect the theoretical result to the implementation. I also appreciate the clarification regarding the architectural requirements and scope. I will maintain my score of 3.
Rebuttal
W1 (Exact compatibility): Thank you for this suggestion. We have tested this empirically by randomly generating 2D input scalar and vector input data from a normal distribution and embedding this data in 3D using the phi embedding. We then initialized a random 2D equivariant UNet, and used the Nepo warm-start coefficient conversion to convert this to a 3D model. With these we can test the compatibility condition by mapping the 2D input through the 2D model, then embedding it in 3D, and compare it to mapping the embedded 3D data through the 3D model. When we do this, we get a mean-squared error of 9.654e-05. For comparison, a randomly initialized 3D model gets an error of 0.146.
The reason it is 9.654e-05 and not 0 is that small numerical errors in compatibility can build up over multiple layers. However, we used the large UNet (Equiv) model used for the Navier--Stokes experiments, so despite many layers these errors are small.
W2/Q1 (Extensions to other architectures): Two important points prevent any-dimensional machine learning methods from being applied to any general neural network. First, as we explain in the paper, some form of symmetries are necessary to use the framework of representational stability that allows us to perform any-dimensional machine learning. Second, even after choosing a model that implements suitable symmetries, the conditions on the model architecture must be proven for each model individually, as we do in Theorem 4.1. With further significant work, more equivariant models could be adapted to be any-dimensional, but that is outside the scope of this work.