- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes VFM4ST, which uses a domain encoder (not learnable), a Video ViT model (pre-trained), and a decoder (jointly optimized with the Video ViT model. The prediction for a domain is obtained by using a three-level processing representing Anchor, Dynamics, and Readout. The paper shows that this process leads to good results on four small-scale datasets, both when training for each downstream task independently and when training jointly for all tasks.
Critical strengths
- The method is evaluated on four heterogeneous domains, including both dense grid-based prediction and sparse station-based forecasting.
- The ablation comparing video-pretrained initialization with training from scratch shows consistent performance improvements.
- The experiments include useful studies on pretraining modality, adaptation strategy, low-data performance, domain-specialized training, and shared multi-domain training.
Critical weaknesses
- The work overlaps considerably with ST-VFM (Chen et al. 2025). ST-VFM is cited but not evaluated as a baseline.
- ST-VFM uses image backbone and performs experiments with video backbones showing that these underperform. The speculated reason is that image datasets are much more diverse and large compared to video datasets, thus image backbones have better pre-training. This contradicts with the main narrative of this paper, and it is not proven otherwise by the paper.
- The proposed “three-port decoder” is just expressing established mechanisms of the downstream tasks (persistence or climatological anchoring, warping/advection, residual correction, and geometry-specific readout).
- Multi-domain training with domain embeddings and domain-specific heads is standard multi-task or multi-domain learning.
- Experiments do not account for overall computational complexity.
- It is not clear whether the performance improvements are due to the method or just the use of the specific backbone.
Questions for authors
- How does the method compare with ST-VFM?
- What is technically novel in the proposed “three-port decoder” beyond the established use of persistence or climatological anchors, warping or advection, residual correction, and task-specific readout mechanisms?
- What is the methodological contribution of the shared model, beyond standard multi-domain learning?
- What is the computational cost of the method and how does it compare with the rest methods in the comparisons?