- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies continual learning (CL) for weather and climate foundation models across three task-expansion scenarios: variable CL (new atmospheric targets), regional CL (new geographical domains), and climate-projection CL (new response objectives). Through an empirical investigation, the authors find that forgetting is strongly scenario-dependent and that cross-task transfer is uneven and directional. They propose GUST, which stores task knowledge in frozen low-rank adapters and selectively composes historical updates via a directed task graph integrating structured domain relations, learned task geometry, and online transfer feedback. Experiments on WeatherBench (ClimaX backbone) and ClimateBench report improvements in the acquisition–retention trade-off relative to compared CL baselines.
Critical strengths
S1. Well-motivated problem formulation. The paper addresses a genuine gap: existing weather/climate foundation models operate under a static lifecycle, yet deployment demands sequential capability expansion. Formulating CL across three complementary axes (variables, regions, climate responses) provides representative scenarios that are broadly relevant to the AI4Science community.
S2. Thorough empirical investigation before method design. Section 3.2 is a standout contribution. The finding that forgetting severity is associated with task-structural compatibility (rather than sequential optimization per se), and that parameter isolation alone leaves a substantial gap to joint training, provides clear and actionable guidance. The stability–plasticity analysis in Fig. 1(b) is informative.
S3. Reasonable method design with interpretable structure. GUST cleanly decouples stable knowledge storage (frozen LoRA adapters) from adaptive knowledge reuse (graph-guided composition). The integration of structured domain priors, learned task geometry, and online utility/interference signals into a unified routing score (Eq. 4) is principled. The temporal DAG structure enforces causal ordering and prevents access to future tasks.
Critical weaknesses
W1. Task-order robustness is not evaluated. All experiments use fixed task sequences. Since GUST freezes routing rows after task acquisition, changing the order affects available historical adapters and graph propagation paths. Results under at least one alternative ordering would help establish whether the improvements generalize beyond the chosen sequences.
W2. Scalability characterization is insufficient. GUST stores one adapter per task and maintains a K×K routing matrix. The paper lacks measurements of memory growth, training overhead, and inference latency. Although the current propagated routing row can be computed in O(K²) per routing update, the practical costs of adapter composition and deployment remain unclear, particularly for longer task sequences.
W3. Baseline adaptation and tuning require clarification. Sec. 5.1 uses an identical learning rate and training schedule across methods with substantially different parameterizations. In particular, adapting prompt-based classification methods to dense weather regression requires non-trivial design choices. The paper does not sufficiently specify prompt injection, trainable parameter counts, or per-method hyperparameter tuning. These omissions make it difficult to assess the fairness of the comparisons.
Questions for authors
Q1. How robust is GUST to different task orderings? Could the authors report results for a reversed or shuffled variable CL sequence?
Q2. What does two-hop propagation provide beyond directly learning adapter composition weights? Since the final update remains a convex combination of historical updates, two-hop propagation does not expand the set of possible mixtures relative to an unconstrained simplex router. Is its benefit primarily an inductive bias or improved optimization?
Q3. Were the baselines tuned individually? Could the authors clarify the prompt-based implementations and report the memory and inference overhead of GUST?
I thank the authors for their clarifications and additional experiments, which substantially address several of my concerns. I maintain my positive score.