- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes UniJEPA, a pretraining scheme for few-shot forecasting of physical systems. Each field is routed by type (scalar, vector, tensor) into a shared Transformer, a predictor forecasts the next frame’s latent against an EMA target, and lightweight decoders expose the prediction to reconstruction, physics-residual, rotation-consistency and realizability losses during pretraining. The pretrained encoder and predictor are then fine-tuned inside an FNO on 16 trajectories. On Active Matter from The Well, this lowers full-state VRMSE by about 10% over a parameter-matched FNO trained from scratch; with some gains on PDEBench scalar systems and CFD2D.
Critical strengths
The following include the strengths of the manuscript: (S1) The structure of the paper is one of its strengths.
(S2) Multi-seed experiments are another positive point, commonly neglected in this area and present here.
Critical weaknesses
The following include the weaknesses of the manuscript (conceptual, technical and presentation):
(w1) The overall motivation of the work is not clear from the introduction or abstract. From the method it seems the method is acting a surrogate given all the predictive losses and hence the introduction must clarify this for the reader. Under the acceleration assumption it is expected so see plots comparing inference speed when comparing to baselines and even potentially the simulation itself.
(w2) If all the benchmarks are deterministic it is difficult for me to understand how JEPA or any representation encoder would facilitate the downstream objective which is forecasting in this work. The primary area where representation encoders shine are where observations are noisy and not deterministic to remove the noises not needed to model the state of a stochastic system.
(w3) The usage of term JEPA also fully seems out of place. Joint-Embedding Predictive Architecture generally refers to a case where we have joint embedding of the target and context and all predictions take place in this space. Once you add the pixel space (or physical space in this work) loss the JEPA disappears. What the work uses is some form of latent regularizer more like BYOL but with next frame as the target instead of an augmented view of the same frame.
(w4) The JEPA term is never shown to matter and in Active Matter JEPA-only pretraining equals random initialisation (0.1430 for both, Table 3), and its frozen probes are worse than a random encoder (Table 5). Table 4 credits the rotation and physics losses on decoded fields (rotation alone: 0.1427 -> 0.1306). Also in CFD2D and scalars the UniJEPA and JEPA are not distinguishable. Overall the improvements seem marginal to none compared to JEPA.
(w5) There are numerous number of terminologies used without proper definition and are not standard to the field. This is not an exhaustive list but mostly the ones that stand out and significantly reduce the readability of the paper and weakens the claims in general:
-
Use of “semantics” for physical variables (Secs. 1, 3.1). The term is non-standard and conflates two independent notions. First a field’s tensor rank, which fixes its transformation law, and a physical relation between fields. Stress and strain are rank-2 tensors related through a stiffness tensor that, for an isotropic material, reduces to a scalar Young’s modulus; the three share one relation but not one type.
-
Use of “pretraining” implies a general pretraining stage, but it is benchmark-specific: one checkpoint per family (Sec. 3.1), so the representations are never used across benchmarks or problems.
-
“Labeled trajectory” / “scarce labels” / “limited labeled trajectories must support learning the target dynamics” (Sec. 1). In forecasting every future frame is its own label, so pretraining and fine-tuning use the same supervision; the phrasing hides this. “Target dynamics” (Sec. 1). Undefined; the system forecast downstream is the system pretrained on.
(w6) The losses the main result depends on are not defined. L_eq never appears as a loss: App. A defers to App. D, where Eq. 37 is a test metric, and the model, inputs and group it is applied to during pretraining are unstated. L_phys for Active Matter is implied to be a kinetic residual (Eq. 9), declared non-evaluable in App. B, and reduced to strain consistency plus realizability in App. F.2, with the strain-consistency loss never given. Eq. 15 has five weights, Table 19 reports two, and the downstream fine-tuning objective is never written at all.
(w7) Main pipeline figures are not clear. For instance in figure 1 what is rho, u, or Q how are they different and what is a unified typed encoder? What is the stop gradient acting on and which loss is not backpropagated through which block? Figure 2 contains similar issues and is additionally too busy to understand what are the contributions.
Questions for authors
Following questions are raised to clarify the objective of the work and help to resolve the weaknesses. It would also help to provide any clarifications on the weaknesses raised directly.
Q1. Can authors describe the need for a JEPA like framework in deterministic environments where we can simply do pixels space predictions with masked modelling or next token prediction to extract rich features? The datasets evaluated here never seem to suffer from noisy observations. See w2.
Q2. Can authors provide some description on the stability of the framework? This framework contains and encoder-predictor-decoder all trained end to end. Decoder tries to spread information for reconstruction purposes and the predictor tries to collapse these into lower spectrums.
Q3. Does removing the JEPA from the pipeline actually affect the results provided here? The difference to baselines are not significant as presented, consequently it is expected the sources of gains to be ablated in detail.