- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper presents JEPACast, a generative precipitation-nowcasting framework which combines a V-JEPA-style encoder that has been trained from scratch on radar sequences, a latent flow-matching Transformer, and a ViT decoder that was trained independently. It uses two complementary criteria to characterise the forecasting representations: decodability, defined as the capacity to faithfully reconstruct radar observations, and predictability, which is assessed by how effectively lightweight probes can extract future-relevant information from frozen past representations. The key contributions are a representation-centric diagnostic framework for nowcasting latents and a specific model that carries out generative forecasting in a representation space that has been learned for predictability. The evaluation involves round-trip reconstruction diagnostics; the use of linear and MLP probes to identify future motion, high-intensity precipitation, and source–sink dynamics; controls regarding latent dimensionality; architectural ablations; and benchmark comparisons on SEVIR and MeteoNet. The results show that future motion is considerably more accessible to the probes examined—especially the linear readouts—and that JEPACast generally leads to an improvement in threshold-based forecast skill, with more evident gains in the case of intense precipitation and at longer lead times. Nevertheless, the method does not outperform every metric or evaluation scale, and its SEVIR CRPS is still worse than that of FlowCast.
Critical strengths
-
The study goes beyond comparing end-to-end forecast predictions by looking separately at reconstruction fidelity and the accessibility of future-relevant information in the frozen representations. Evidence is provided by means of round-trip reconstruction diagnostics, linear and MLP probes, controls involving latent dimensionality, and lead-time analyses. The most convincing of these are the results from the linear motion probes, which indicate that, with the proposed probe protocol, the JEPA representation makes future motion considerably more accessible while still maintaining competitive—although not always the best—reconstruction quality. Furthermore, the equal-width analysis implies that latent dimensionality by itself is not sufficient to account for this advantage in motion, although the corresponding advantages regarding intensity and source–sink dynamics diminish under strong compression.
-
JEPACast shows excellent threshold-based forecasting results on both SEVIR and MeteoNet, especially when it comes to predicting intense precipitation and at longer lead times for SEVIR. The benchmark results, the component ablations, and the qualitative comparisons all support the idea that the entire pipeline is effective. The fact that these downstream improvements are in line with the representation-level findings offers encouraging evidence that predictive representation learning is a useful approach for generative nowcasting.
Critical weaknesses
- The JEPA training objective's contribution is not entirely separate from other factors. As shown in Table 4, the latent-representation ablation affects the representation-learning objective, the encoder architecture, the native latent dimensionality, and the decoder at the same time. Even though the different variants in this ablation employ a matched but shorter forecaster-training schedule, it is still not clear whether the observed advantage would continue when full training is carried out or at convergence. The equal-width VideoVAE and PCA analyses do provide some useful partial controls, but these are mainly carried out at the diagnostic-probe level and not through matched end-to-end forecasting experiments. A comparison in which the encoder architecture, the latent size, the decoder capacity and the training procedure, the forecaster configuration, and the optimization budget are kept constant while only the representation-learning objective is varied would reinforce the claim that the improvements in the downstream task are due to predictive representation learning rather than to the wider JEPA-based representation pipeline.
- The connection between diagnostic predictability and downstream forecasting performance is still indirect. Instead of directly assessing the difficulty of modelling the full conditional distribution of the future latents, the probes look at how easily constrained readouts are able to obtain the selected future quantities—specifically, motion, intensity, and source–sink dynamics. Following the equal-width compression shown in Table 5, the JEPA representation still has a clear advantage when it comes to motion, but its advantages in terms of intensity and source–sink either decrease or reverse compared to some of the representations learned through reconstruction. The evidence thus provides stronger support for improved accessibility of future motion than for a general improvement in the predictability of the future. In order to more convincingly show that the diagnostic is an indicator of downstream nowcasting skill, the paper should set up a systematic relationship between the probe scores and the corresponding end-to-end forecasting performance—for instance, across different representation-learning objectives, latent widths, and forecaster capacities.
Questions for authors
-
Beyond the reported controls, is there existing end-to-end evidence comparing predictive and reconstruction-based pretraining under matched encoder architecture, latent dimensionality, decoder architecture and training, and forecaster settings? Both variants in Table 4 use the same shortened training schedule, preserving within-group fairness; are full-schedule results available to determine whether the advantage persists? These results would affect how strongly I attribute the downstream gains to the JEPA objective rather than the overall representation pipeline.
-
Do existing results show whether representations with stronger probe scores also achieve better downstream forecasting performance under matched forecaster settings? In particular, do the motion advantages retained at equal latent width correspond to forecasting gains despite the weaker intensity and source–sink results? This would affect my assessment of the diagnostic’s utility for evaluating forecasting representations beyond the specific JEPACast configuration.
3 comments
Follow-up to Reviewer Jeua: equal latent width
As promised in our first response, we report the end-to-end experiment at equal latent width (Q2).
[W2, Q2] Equal latent width
We trained our encoder at a latent width of 16, the width of the FlowCast VAE latent, and paired it with the same forecaster (20k steps) as in Table C1. At equal width of 16, the motion advantage of the JEPA latent carries over to forecasting (Table C3). The JEPA latent has the higher motion probe and the higher CSI-M. Compared with our 512-wide latent, reducing the width to 16 lowers CSI-M by 0.0055.
Table C3. End-to-end comparison at equal latent width on SEVIR, with JEPACast (width 512) for reference. Width is the number of channels per 16 km cell, and the motion probe reports R² (linear / MLP) from Tables 1 and 5 of our paper.
| Latent | Width | Motion probe ↑ | CSI-M ↑ |
|---|---|---|---|
| JEPACast | 512 | +0.757 / +0.806 | 0.4646 |
| V-JEPA (16-dim) | 16 | +0.231 / +0.666 | 0.4591 |
| FlowCast VAE | 16 | +0.183 / +0.619 | 0.4548 |
Revision. We will add the equal-width comparison at width 16 to the revised paper, together with the comparison at width 512 (Table C1).
[Update] We have uploaded the revised paper, which includes all the revisions promised in our responses, with the changes marked in blue. We also corrected a typo in Table C2. We would greatly appreciate it if you could take a look at the revised paper and our responses.
Gentle reminder
Dear Reviewer Jeua,
Thank you again for the time and care you have put into reviewing our paper. We understand that this is a very busy period, and we truly appreciate your effort.
We would like to gently follow up on our responses and the revised paper. As the discussion period will close soon, we would be grateful if you could let us know before then whether they address your concerns, and we would be glad to answer any further questions in the remaining time.
Response to Reviewer Jeua
We thank Reviewer Jeua for the careful and constructive review. We are encouraged that the reviewer found (i) the representation-level analysis valuable, (ii) the linear motion probes convincing, and (iii) the downstream gains consistent with the representation-level findings.
Following the reviewer's suggestions, we added a matched end-to-end comparison of latents (Table C1) and full-schedule learning curves (Table C2). The end-to-end experiment with equal latent width (Q2) is in progress, and we will report the results here as soon as it is complete.
[W1, Q1] Isolating the pre-training objective, and the full schedule
As the reviewer requested, Table C1 compares latents end-to-end with the same forecaster (20k steps) and sampler. V-JEPA 2.1, temporal masking, and VideoMAE additionally share our encoder architecture and size, training data, pre-training budget, and decoder recipe, so that for them only the pre-training recipe changes, and VideoVAE (128-ch) matches our latent size exactly with a reconstruction objective. With these factors fixed, masked latent prediction forecasts better than reconstruction. JEPACast reaches a CSI-M of 0.4646, compared with 0.4475 for masked pixel reconstruction (VideoMAE) and 0.4302 for autoencoding at the same latent size (VideoVAE (128-ch)), and it also improves on the original V-JEPA 2.1 recipe and on the causal temporal-masking variant.
Table C1. Latent comparison on SEVIR. Probes report R² (linear / MLP) as in Table 1.
Full schedule (Q1). As the reviewer asked, we continued training to the full 45k steps (Table C2). The advantage of the JEPA latent remains over the whole schedule, and it leads at every evaluated step. On the JEPA latent, the forecaster is still improving at 45k (0.4715). With the FlowCast VAE latent, which is used in the latent ablation of Table 4 in our paper, the same forecaster peaks at 15k (0.4551, close to FlowCast's own model) and then overfits, and on VideoVAE (128-ch) it saturates at 0.4401, a level that the JEPA latent reaches at 5k steps.
Table C2. Test CSI-M of the same forecaster during training.
Revision. We will add the matched comparison and the learning curves to the revised paper.
[W2, Q2] Probe scores and forecasting performance
Thank you for this important question. With the same forecaster, probe scores and forecasting performance follow the same order for four of the five latents at the native width of 512. Among the latents of Table C1, JEPACast, V-JEPA 2.1, VideoMAE and VideoVAE (128-ch) are ranked in the same order by the MLP probes of all three targets, by the linear motion probe (R² +0.757, +0.721, +0.677 and +0.263) and by CSI-M (0.4646, 0.4558, 0.4475 and 0.4302). The one exception is the causal temporal-masking variant, whose probes are comparable to ours but whose forecasts are lower. It is the only encoder trained directly to predict the future from the past frames, which is what the probes test, so its probe scores follow from its objective rather than from the overall quality of the latent. Its latent is less decodable (round-trip CSI-M 0.854 vs. 0.877), and the gap is already present at the first lead (+5 min, 0.660 vs. 0.694), where little has to be predicted. This is why our diagnostic measures both predictability and decodability.
What the probes measure. The direct measure of how hard the future latents are to model is the forecaster's own training loss, but each latent has its own dimensionality and scale, so this loss cannot be compared across latents. The probes instead predict the same physical quantities from every latent, which makes them comparable. They can thus serve as an early, complementary indicator of how well a forecaster will learn on a latent, which the learning curves then show directly (Table C2).
Revision. We will add the comparison of probe scores with forecasting performance to the revised paper and narrow the statement at L72 to future motion.