- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
The paper trains a contrastive encoder on local features of 1D PDE solutions from three equation families, then applies it to twelve families to build a shared latent space ("Operator Atlas"). The authors analyze the structure of this space and use it, without retraining, for reconstruction, equation identification, parameter inference, and anomaly detection.
Critical strengths
The question of whether one representation can capture relations between different PDE families is interesting and relevant. The paper also examines the learned space from many angles (structure analysis, ablations, extensions to 2D and time, and downstream tasks), and it openly reports some negative results, such as the weaker transfer to 2D and time-dependent settings.
Critical weaknesses
W1. Key parts of the method are not clearly defined, which makes the results hard to interpret.
-
W1.1 It is unclear what a single training sample is. The model works on individual grid points, but the paper also refers to "patches" (30 per solution) without defining them, and q, k and sim(·,·) in the contrastive loss (Eq. 4) are not defined either.
-
W1.2 The design choices are not justified. Why does each point use exactly these six features (one of which, the second difference, can be computed from the other two differences)? Why is the atlas 16-dimensional, and how do the findings change with this dimension?
-
W1.3 It is unclear what each point in the t-SNE/UMAP plots represents (a single grid point or a whole solution). If each point is a single grid point, structures such as the Allen–Cahn energy regions or the Helmholtz u > 0 / u < 0 split may simply reflect the input value u rather than learned geometry.
W2. The claim that the model generalizes from three families to twelve is not supported.
-
W2.1 The paper does not explain how the three training families were chosen or how sensitive the results are to this choice.
-
W2.2 Three of the twelve evaluated families are the training families themselves, and some of the others are nearly the same equations in the 1D steady setting (e.g., steady heat vs. Poisson/Laplace).
-
W2.3 Results are always averaged over all twelve families. The paper never shows how the model performs on the nine unseen families alone.
-
W2.4 The paper does not show whether the pretrained representation actually helps when learning a new family.
Questions for authors
Q1. (W1.1, W1.3) Please define precisely what a "patch" and a contrastive sample are, and what q, k and sim(·,·) are in Eq. 4. Also, what does each point in the t-SNE/UMAP plots represent?
Q2. (W1.2) Why were these six input features and a 16-dimensional atlas chosen? How do the main findings change with a different atlas dimension (e.g., 8 or 32)?
Q3. (W2.1) How were the three training families selected, and how do the results change if the model is trained on a different set of three families?
Q4. (W2.2, W2.3) Could you report the results separately for the three training families and the nine unseen families?
Q5. (W2.4) To test whether the learned representation really transfers, could you compare validation curves on each unseen family for (a) the model trained from scratch and (b) the model fine-tuned from the three-family checkpoint, with the same data and budget? If (b) gives no clear advantage, the claim of a shared, transferable representation would be weakened.
We thank the reviewer.
W1.1 / Q1 — Samples, patches, and loss
A sample is one interior grid point, not a field. With N=64, an instance has 62 candidates. Input si∈R6 contains ui, differences (ui−ui−1)/Δx, (ui+1−ui)/Δx, second difference (ui+1−2ui+ui−1)/Δx2, xi, and Δx. A patch is this stencil, not a window. Drawing 30 points per instance with replacement, weighted by ∣∂xu∣, yields 3×1000×30=90,000 samples. Testing uses all 62 points (or the first Q in few-shot).
For an anchor, q=oi=gop(zi)∈R16; candidates are other batch embeddings (B=512), cosine similarity, τ=0.1. All same-family patches are positives; the loss groups embeddings by family. The single-positive shorthand is removed; retrieval budget q becomes Q.
W1.2 / Q2 — Features and atlas dimension
The second difference is linearly dependent; explicit curvature spares MLP reconstruction—feature engineering, not new information.
We retrained with opdim∈{4,8,16,32,64}, two seeds each; seed ranges (to 0.143), not mean±std, define noise. Dimension 16 gives separation 0.586 on both seeds (untrained: 0.496); 32 is weaker, while 4 and 8 lie within seed variability. Dimension 64 exceeds 16 on both seeds only for unseen families (0.706 vs 0.575); trained-family gain is absent: possible headroom. All 10 cross-dimension comparisons correlate positively (Spearman 0.40–0.61, p=0.002).
W1.3 — Do structures merely reflect u?
We probed Allen–Cahn bands using u, raw features, the atlas, and 2-D t-SNE. Since labels are deterministic under E=(u2−1)2/4, 1-NN on u reaches 0.994, above the atlas (0.927) and t-SNE (0.893). We removed the structural claim, retaining continuity.
Band direction is linearly readable from the atlas (logistic regression: 0.744), less from raw features (0.587): linearization only. Field-grouped ε ridge R2 is 0.971 (raw), 0.911 (u), 0.796 (atlas): no information beyond u—a stated limitation.
W2.1 / Q3 — Training-family selection
Two seeds each: A is the original; B removes Helmholtz, C substitutes convection–diffusion, D adds Burgers. All exceed chance (0.083) and the untrained control (0.496), preserving angular organization. Only B consistently degrades separation (0.489/0.519 vs A: 0.586/0.586); C does not substitute (0.519/0.519). Thus Helmholtz is load-bearing, though the triple is not essential. D may help (0.579/0.692), but the 0.113 seed gap makes this suggestive. Cross-variant angles remain positively correlated (Spearman 0.21–0.53, p=0.002).
W2.2 — Near-duplicate families
Steady source-free heat is Laplace-like; Poisson adds a source, and the atlas should quantify that proximity. Near-duplicates do not drive results: all three heat fields map to trained advection–diffusion, sharing random sources with heat. Eight unseen families achieve 3/3 identification; the sole error is Fisher–KPP→Burgers. Leakage would favor trained families, contrary to the split.
W2.3 / Q4 — Trained versus unseen families
Plausibly, independent random sources make these fields irregular under geodesic interpolation and nearly coincident. Anomaly AUROC is pooled (97+36 nominal vs 192 anomalous), precluding a family split. This argues against a lookup table.
W2.4 / Q5 — Fine-tuning versus random initialization
For each unseen family, we compare the checkpoint with two random initializations under identical data, optimizers, and 100 epochs; the two largest parameters are held out. Only readout heads (18,641 parameters) adapt; the frozen trunk complements zero-shot evaluation.
The checkpoint gives the lowest held-out reconstruction loss for 8/9 families, except Biharmonic (p=200/500 extrapolation). After fine-tuning, mean identification is 0.886 vs 0.701/0.747 from random initialization, winning 9/9 against both seeds (median gap 0.167; range 0.056–0.278). Its zero-shot score (0.889) is unchanged.
Initialization gives no parameter-inference gain (0.084 vs 0.054/0.052), as in the untrained control. Reconstruction-only fine-tuning reduces anomaly AUROC by ~0.03, near the noise boundary. We limit transfer claims to identification and reconstruction geometry; parameter inference depends on readout and data.