acceptodds
Under review as a conference paper at ICLR 2027

Counterfactual Predictions in Scientific Emulators Without Controlled Experiments

Abstract

Many scientific questions require reasoning about what was never observed: *What if the conditions, interventions, or history had been different?* Models can predict accurately on observed data yet fail on such what-if queries when correlated inputs are varied independently. A common remedy is to add controlled simulation data in which these factors are explicitly disentangled, but this requires access to a simulator, can be computationally expensive, and inherits the simulator's modeling assumptions. We introduce *ReRoute*, a framework for targeted scientific what-if prediction that combines factual data with partial mechanistic knowledge, without requiring controlled intervention data for adaptation. *ReRoute* fixes the queried input of a pretrained backbone to a reference value, reintroduces its variation through a known mechanistic pathway, and fine-tunes on the original factual data, while leaving downstream effects to the learned dynamics. We provide a causal identification result for this construction under explicit structural assumptions, with the core argument machine-checked in Lean. After showing that *ReRoute* achieves highly accurate counterfactual predictions in a controlled advection-diffusion system where exact responses are available, we turn to state-of-the-art climate emulation. On held-out coupled-climate interventions, *ReRoute* reduces aggregate climate error by 18.2-31.8% under severe CO distribution shifts while preserving skill under standard conditions, at a small fraction of the cost of retraining on additional controlled simulations, without even accounting for the substantial expense of generating such data. Finally, on an emulator trained from historical ERA5 reanalysis, where no counterfactual reference exists, *ReRoute* preserves substantially more of the surface warming implied by the observed boundary conditions under a fixed-CO counterfactual.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
AI for Science, Counterfactual Reasoning

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_FnapOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors design a hybrid modeling mechanism that can enhance a data-driven emulator with a mechanistic pathway to enable counterfactual scenarios, even if, for training, only factual data is available. The proposed mechanism offers a middle path between the two commonly available ways to assess counterfactual scenarios: having counterfactual data to train the emulator or having the full mechanistic model.

The mechanism takes the pretrained emulator and fixes the variable of interest to a reference value. The pathway then describes the change from the reference value to any other value mechanistically, which the pretrained model is then fine-tuned to adapt to. They evaluate their method on an advection-diffusion transport example, showing that it substantially improves the error, and on a realistic climate emulation task asking counterfactual questions under CO2CO_2 shifts. Formally, the authors derive causal identification guarantees that they machine-checked in Lean.

Critical strengths

The authors present a simple but compelling idea with convincing results in a well-digestible way. I especially appreciate the "intuition" and "example" bits.

  1. The approach is a genuinely new and effective way to use hybrid modeling, where usually the ML component is used to correct or improve a mechanistic model. Here, however, the mechanistic part is used to specifically correct the inability of the ML component to extrapolate beyond its training domain into counterfactual regimes.

  2. The downstream task of counterfactual scenarios in climate emulation is well chosen and highly compelling. The task of disentangling CO2CO_2 and SST when no controlled counterfactual data exists is an important challenge in the Earth sciences. The results prove the effectiveness of their approach, and I do believe it holds merit for many other applications.

Critical weaknesses

I do not have strong issues with this work. There are two small things I would highlight.

  1. The advection-diffusion experiment appears slightly tautological, as the pathway seems to need the exact full mechanics, which is, in principle, what this approach tries to bypass. I understand that here it serves more like a proof of concept, while the climate emulation is the realistic setup.

  2. I am missing an ablation on the reference value. It would be important to know how sensitive the approach is to that choice.

Questions for authors

I am currently leaning toward a clear accept, as the idea is compelling and the results are convincing. Nonetheless, a sensitivity analysis of the reference point would further strengthen my confidence in the robustness of the method.

4 comments
AuthorsOpenReview ↗

Response to Reviewer Fnap

Thank you very much for the positive assessment. We added the reference-value sensitivity study as suggested: we observe that the gains hold up to 2× CO$_2$, and the default 1× is the best practical choice.

[W1] Advection–diffusion is the proof of concept; climate is the realistic test.

  • We agree: AD is the controlled setting in which the assumptions of the identifiability theorem hold exactly, including target determination.
  • In climate, where the pathway is only approximate, the improvement persists across pathway scalings, any scale from ×0.75 to ×2 still beats the baseline on both extreme CO₂-shift tests (T2 in our response to Reviewer rK5t).

Manuscript update: §§ 4.1, B.4, and F.1.3; Table 5.

 

[W2, Q] Reference value: gains hold up to 2×, and the default 1× is the best practical choice.

  • We sweep the reference from 1× to 5× CO₂ in climate. Zero-shot ReRoute already works well, so we run the sweep without weight updates, which cleanly separates the effect of the reference from adaptation.

T5: Reference value s0 from 1× to 5× CO₂ (ReRoute zero-shot, physical pathway. Baseline (no-RC): released model trained without random-CO₂ data. AMIP hist. is the factual test; the other columns are CO₂ interventions. Training-mix mean: average CO₂ over the baseline's training data. At s0 = 4×, the abrupt-4×CO₂ query equals the reference, so ReRoute reduces to the baseline there. α, lower is better: channel-mean RMSE of normalized time-mean biases; mean ± std over 5 realizations for AMIP, scored 2012–2020, and random-CO₂, 5 years; over 36 initial conditions for abrupt-4×CO₂, 90 days. Bold: best per column and every value whose gap to it is within one standard deviation.)

ModelAMIP hist.AMIP const-CO₂Abrupt 4×CO₂Rand-CO₂
Baseline no-RC (no rerouting)0.0350 ± 0.00130.0369 ± 0.00040.1595 ± 0.01100.1130 ± 0.0029
s0 = 1× (363 ppm)0.0327 ± 0.00140.0394 ± 0.00210.1324 ± 0.01530.0814 ± 0.0075
s0 = training-mix mean (604 ppm)0.0589 ± 0.00080.0658 ± 0.00160.1275 ± 0.01060.0657 ± 0.0028
s0 = 2× (727 ppm)0.0680 ± 0.00080.0750 ± 0.00110.1305 ± 0.01260.0556 ± 0.0023
s0 = 4× (1454 ppm)0.1240 ± 0.00090.1293 ± 0.00090.1595 ± 0.01090.0750 ± 0.0015
s0 = 5× (1817 ppm, outside training range)0.1495 ± 0.00200.1548 ± 0.00100.1867 ± 0.01010.1057 ± 0.0016
  • Every reference up to 2× improves both extreme CO₂-shift tests (abrupt-4×CO₂, random-CO₂) over the baseline model.
  • Higher references favour high-CO₂ queries but cost factual skill; a reference outside the training support is worst.
  • The default 1× is the only reference that keeps factual skill at the baseline level while improving both extreme CO₂-shift tests.

Manuscript update: Table 6, § F.1.4.

Thank you again for the encouraging review and the helpful suggestions!

Dear reviewer Fnap:

Thank you again for the positive assessment and your constructive feedback. As the discussion period concludes soon, we kindly ask if the rebuttal has addressed your concerns.

Should any further questions or concerns arise, we are happy to provide clarification during the remaining discussion time.

Reviewer_FnapOpenReview ↗

Response to authors

Dear authors,

Thank you very much for your response and the additional experiments (including the ones in the responses to the other reviewers). The added pathway-scaling and reference-value analysis address both of my raised concerns. It is interesting to see that there seems to be a trade-off between catching extreme tests and maintaining factual skill, and it does make sense to me. All the additions strengthen my confidence in my assessment.

Dear reviewer Fnap:

Thank you so much again for your valuable suggestions and continued positive feedback! We are really glad that the additional analyses addressed your concerns and strengthened your confidence in our work.

We truly appreciate the time and effort you and the other reviewers have put into reviewing our work. Thank you again!

Reviewer_WFFnOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces ReRoute, a framework for improving counterfactual prediction in scientific emulators without requiring additional controlled intervention data during adaptation.

Scientific emulators may learn correlations between inputs rather than the correct underlying mechanisms. As a result, when correlated variables are changed independently, the models may produce inaccurate predictions. Controlled experiments can help disentangle these effects, but generating such data can be computationally expensive, require simulator access, and introduce additional modeling assumptions.

To address this problem, ReRoute combines a pretrained neural network with a known mechanistic pathway. The method fixes the targeted input to a reference value in the pretrained backbone, reintroduces its variation through the known mechanistic pathway, and fine-tunes the resulting model using the original factual data. The authors also provide a theoretical causal identifiability result under explicit structural assumptions, supported by Lean-verified proofs.

The authors evaluate ReRoute on a controlled advection–diffusion system, coupled climate simulations, and historical ERA5 reanalysis data. In the advection–diffusion experiment, ReRoute reduces counterfactual-response error by up to 89.6% over 100-step rollouts, despite being fine-tuned using only 10-step rollouts. In the coupled climate experiments, it reduces aggregate prediction error by 18.2–31.8% under severe CO2 distribution shifts. The ERA5 experiment provides a qualitative evaluation where no observed counterfactual reference is available. The authors also report approximately 18× lower training compute for ReRoute adaptation compared with training a controlled-data counterpart, excluding pretraining and data-generation costs.

Critical strengths

1. Simple and computationally efficient framework

The proposed ReRoute framework is relatively simple and computationally efficient. It combines existing pretrained scientific emulators with partial mechanistic knowledge without requiring additional controlled intervention data during adaptation. The method can be applied through fine-tuning rather than retraining from scratch, making it potentially useful for scientific applications where generating new simulations is expensive or impractical. The reported improvements across controlled advection–diffusion and climate experiments demonstrate the practical potential of this approach.

2. Theoretical foundation for counterfactual prediction

A strength of the paper is its attempt to provide a theoretical foundation for counterfactual prediction through a causal identifiability result. The authors explicitly state the structural assumptions under which interventions can be identified from observational data and known mechanistic pathways. The inclusion of Lean-verified proofs further strengthens the mathematical rigor and transparency of the theoretical contribution. Although these guarantees depend on restrictive assumptions and do not directly cover all experimental settings, the theoretical analysis provides a useful foundation for understanding when the proposed approach may be applicable.

Critical weaknesses

1. Lack of ablation studies to isolate the contribution of ReRoute

The authors introduce ReRoute by making three changes to the pretrained model: (1) fixing the targeted input to a reference value, (2) introducing a known mechanistic correction, and (3) fine-tuning the model on the original factual data. However, the main experiments compare the complete ReRoute-adapted model against the original pretrained baseline, without adequately isolating the contributions of these changes. In particular, there is no comparison against the original backbone fine-tuned on the same factual data with an equivalent training budget (Figure 1, page 3; Section 4.2.1, page 7). Therefore, although the results demonstrate that the complete adapted model performs better under the evaluated conditions, they do not establish how much of the improvement is specifically attributable to the proposed ReRoute mechanism rather than fine-tuning or the individual modifications. This is my primary concern, as the missing ablations directly affect the experimental evidence supporting the paper's central contribution.

2. Limited evidence separating the benefit of the mechanistic correction from the proposed learning procedure

In the advection–diffusion experiment, the authors use the exact one-step evolution operator to construct the mechanistic correction (Equation 4.2, page 6). Since the intervention-dependent response is already analytically available, it is unclear how much of the reported improvement comes from the exact correction itself rather than the proposed ReRoute adaptation procedure. A comparison using the analytical correction without ReRoute fine-tuning would help clarify this distinction. Furthermore, the main counterfactual-response metric measures the error in the difference between two trajectories (Appendix B.3), rather than the absolute prediction errors of the individual trajectories. Reporting both absolute state errors would provide a more complete assessment of predictive accuracy. These limitations make it difficult to fully assess the effectiveness of the proposed method beyond the supplied mechanistic correction.

Questions for authors

1. Contribution of individual ReRoute components (Page 3, Figure 1; Page 7, Section 4.2.1)

ReRoute introduces three changes to the pretrained model: reference conditioning, a known mechanistic correction, and fine-tuning on factual data. However, the main experiments compare the complete ReRoute model against the original pretrained baseline.

Can the authors provide a comparison against the original backbone fine-tuned on the same factual data with an equivalent training budget? Additionally, can the authors provide ablation results isolating the contributions of reference conditioning, mechanistic correction, and fine-tuning? These comparisons are important for determining whether the reported improvements are specifically attributable to the proposed ReRoute mechanism.

2. Contribution of the exact analytical correction (Page 6, Equation 4.2)

In the advection–diffusion experiment, the authors construct the mechanistic correction using the exact one-step evolution operator. Since the intervention-dependent response is already analytically known, how much of the reported improvement can be attributed to ReRoute rather than the exact correction itself?

Can the authors report the performance of the pretrained model augmented with the same analytical correction but without ReRoute fine-tuning? This comparison would help establish the additional benefit of the proposed adaptation procedure.

3. Interpretation of counterfactual-response error (Page 7, Figure 2; Appendix B.3)

My understanding is that the reported counterfactual-response RMSE measures the error in the difference between the intervention and reference trajectories, rather than the absolute error of either predicted state. Is this interpretation correct?

If so, can the authors also report the absolute state RMSE for both the reference and intervention trajectories over the 100-step rollout? Since errors shared by both trajectories can cancel in the current metric, these additional results would help determine whether ReRoute improves the accuracy of the predicted physical states in addition to the intervention-induced response.

6 comments
AuthorsOpenReview ↗

Response to Reviewer WFFn (Part 1/2): ablations and protocol

Thank you for your careful review and valuable feedback. We ran additional ablations following your suggestions for both climate and advection-diffusion systems. They show that the gains come from rerouting, not from fine-tuning or clamping alone.

 

[W1, Q1] Each component isolated: rerouting is the key component.

  • Rerouting alone, without fine-tuning, already gives most of the counterfactual gain in both systems (zero-shot ReRoute, C+P).
  • Clamping the backbone’s source input to its reference value is not sufficient, as expected: it simply evaluates the pretrained model at that value, without adding the physical correction from the reference to the queried value (C).
  • Fine-tuning without rerouting on the same data and budget does not improve counterfactual skill (F).
  • Fine-tuning the rerouted model improves it further, on factual and counterfactual tests alike (C+P+F).

T1a: Component ablations with reference comparison (bold: best per column and every value whose gap to it is within one standard deviation; protocol below. Reference not ranked)

ModelComp.AMIP hist.AMIP const-CO₂4×CO₂Rand-CO₂
Trained with additional random-CO₂ data
Reference—0.0418 ± 0.00750.0356 ± 0.00100.1181 ± 0.01280.0452 ± 0.0037
Trained without random-CO₂ data
Baseline—0.0350 ± 0.00130.0369 ± 0.00040.1595 ± 0.01100.1130 ± 0.0029
Clamp onlyC0.0353 ± 0.00140.0401 ± 0.00130.1439 ± 0.00940.1291 ± 0.0050
Fine-tune onlyF0.0383 ± 0.00080.0395 ± 0.00140.1578 ± 0.01040.1194 ± 0.0022
ReRoute (zero-shot)C+P0.0327 ± 0.00140.0394 ± 0.00210.1324 ± 0.01530.0814 ± 0.0075
ReRoute (fine-tuned)C+P+F0.0320 ± 0.00110.0378 ± 0.00160.1283 ± 0.01120.0670 ± 0.0011

C = Clamp, P = Pathway, F = Fine-tune.

Training and evaluation protocol.

  • Models: all ablations and ReRoute variants start from the baseline ACE2S-SHiELD+ checkpoint trained without random-CO₂ data (no-RC). In the zero-shot setting, we use the pathway with the pretrained model without additional training, keeping all model weights fixed. ReRoute uses s₀ = 1× (363.43 ppm) and the physical RRTMG pathway.
  • Fine-tuning: all fine-tuned variants use the same factual training data, objective and learning rate (2e-6), each trained for 30 epochs. For both, we use the earliest saved checkpoint (epoch 2), since longer training did not improve held-out factual AMIP skill (2009–2011). Rerouting moves the backbone away from its pretrained operating point; for ReRoute we therefore interpolate between the fine-tuned and baseline weights (WiSE-FT; Wortsman et al., 2022), with the mixing weight (0.25) chosen on the same held-out years. Fine-tune only worsened both validation climate metrics (held-out AMIP years and held-out equilibrium climates) at every checkpoint, so there was no gain for interpolation to preserve. This is expected: the released checkpoint was already selected on these validation metrics using the same training data (Clark et al., 2026), so further training without rerouting likely only moves it away from that optimum.
  • Objective (ACE2S): 0.9 CRPS + 0.1 energy score on a two-member ensemble, evaluated at the last step of 1-, 2-, 4-, 12- or 20-step rollouts, following Clark et al. (2026).
  • Metric: α is the channel-mean spatial RMSE of normalized time-mean biases (lower is better), reported as mean ± std over: AMIP historical/constant-CO₂ (rollouts from 1979-01-01, following Clark et al. (2026), scored on 2012–2020; 5 stochastic realizations); abrupt-4×CO₂ (90 days, 36 initial conditions); and random-CO₂ (5 years, 5 stochastic realizations). Values therefore differ slightly from the submitted manuscript, and we updated the manuscript accordingly.
  • Random-CO₂ setting: following Clark et al. (2026), CO₂(t) = 2n(t)2^{n(t)} · X₀ with n(t)∼U[−2,2]n(t) \sim U[-2, 2] redrawn every 30 days. Our test member is centred at X₀ = 4× (1453.7 ppm), so CO₂ spans 1×–16× (≈363–5800 ppm).
  • Fixed-CO₂ AMIP: CO₂ is held at 336.6 ppm, versus the historical 391–415 ppm over 2012–2020.

Manuscript update: Table 1, § F.1.2; § F.1.1.

References

  • Clark, Spencer K., et al. "Disentangling the effects of sea surface temperature and CO₂ in global machine learned weather-climate emulators." arXiv preprint arXiv:2606.07928 (2026).
  • Wortsman, Mitchell, et al. "Robust fine-tuning of zero-shot models." 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022.
AuthorsOpenReview ↗

Response to Reviewer WFFn (Part 2/2): advection–diffusion and absolute errors

[W2, Q2] Zero-shot ReRoute, the pretrained model with the analytical correction, already gives most of the gain; fine-tuning adds a large further step.

T1b: Advection–diffusion component ablations (step-100 RMSE, lower is better. Difference RMSE: error of the intervention-minus-reference change; absolute RMSE: error of the intervention forecast. Averaged over 8 initial conditions and all parameter pairs of the region: 1,681 for the full grid, 380 for far-OOD. Both fine-tuned variants use the same factual data and 15-epoch budget, with the checkpoint of lowest factual validation loss; mean ± std over five fine-tuning seeds.)

ModelComp.Diff. full gridDiff. far-OODAbs. full gridAbs. far-OOD
Baseline (pretrained FNO)—0.03330.07840.03500.0779
Fine-tune onlyF0.0302 ± 0.00060.0757 ± 0.00140.0312 ± 0.00070.0754 ± 0.0017
Rerouting D
Clamp only (D)C0.05040.06900.05140.0695
ReRoute-D (zero-shot)C+P0.02350.03560.02650.0360
ReRoute-D (fine-tuned)C+P+F0.0090 ± 0.00030.0128 ± 0.00040.0110 ± 0.00030.0126 ± 0.0004
Rerouting (c, D)
Clamp only (c, D)C0.44650.60020.44630.5997
ReRoute-(c, D) (zero-shot)C+P0.00880.01140.01330.0131
ReRoute-(c, D) (fine-tuned)C+P+F0.0064 ± 0.00010.0081 ± 0.00020.0088 ± 0.00010.0086 ± 0.0001

C = Clamp, P = Pathway, F = Fine-tune. Rows without ± are deterministic (no weight update).

  • The zero-shot rows of T1b are the requested comparison: the pretrained model with the same analytical correction, without ReRoute fine-tuning.
  • ReRoute fine-tuning improves every metric beyond it. The correction supplies only the difference between one-step operators; the backbone still has to learn the reference dynamics from factual data.
  • Fixing the input to the reference value without the physical pathway fails (clamp-only rows).

Manuscript update: Table 2, § B.4.

 

[Q3] ReRoute is more accurate on the absolute states too, not only on the response difference.

  • Your interpretation is correct: the response metric measures the error of the intervention-induced change, $E_{\mathrm{diff}} = \mathrm{RMSE}[(\hat u_t^{s} - \hat u_t^{s_0}) - (u_t^{s} - u_t^{s_0})]$, so errors shared by both trajectories can cancel.
  • We now also report the absolute RMSE (T1b). The same conclusions hold: both fine-tuned ReRoute variants outperform ordinary fine-tuning and their zero-shot counterparts, on the full grid and far-OOD.
  • For the reference forecast, both ReRoute variants improve on the pretrained baseline and are on par with ordinary fine-tuning.

Manuscript update: the rollout plots now use absolute RMSE. Figure 2 and Table 2, § B.4

We hope these ablations resolve your primary concern, and we would be grateful if you would consider raising your score. We are happy to answer any additional questions.

Dear reviewer WFFn:

We greatly appreciate your constructive feedback. As the discussion period concludes soon, we kindly ask if the rebuttal has addressed your concerns.

Should any further questions or concerns arise, we are happy to provide clarification during the remaining discussion time.

Reviewer_rK5tOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper addresses what-if queries on a pretrained emulator when the input of interest, here CO2 in ACE2 climate emulators, moves together with other inputs such as sea surface temperature in the training data. ReRoute fixes the CO2 value the backbone sees at a reference and adds the real CO2 effect through a known radiative heating term computed with RRTMG. The model is fine-tuned on the original data, and a what-if query changes only the known term. An appendix gives an identification result for a structural causal model, partly checked in Lean. Experiments cover an advection-diffusion system with an exact pathway, the SHiELD benchmark (18.2 percent lower error under abrupt four times CO2 and 31.8 percent under random CO2), and ACE2 trained on ERA5, where the fixed CO2 response moves closer to zero, and the tropical cyclone count moves closer to ERA5. The authors also claim 18 times lower cost than training with random CO2 simulations.

Critical strengths
  • The problem is real, and the method is cheap. It needs a short fine-tuning run and a physical term that scientists already have, and the SHiELD gains are measured against reference simulations, which is the right test.
  • The appendix is candid that extrapolation to new pathway values rests on an assumption the data cannot check, and that some experiments fall outside the theory.
Critical weaknesses
  • The identification result does not cover the paper’s setting. It assumes the targeted variable is independent of all downstream noise, so it cannot be entangled with other inputs, and within the observed range the plain conditional distribution already equals the interventional one. If sea surface temperature is a consequence of CO2, the theorem gives a different quantity from the climate query, which holds it fixed. Outside the observed range, the result rests on the target determination assumption, and the paper does not show it is weaker for a known pathway. The Lean check only verifies the derivation from the assumptions. If the pathway is only partly right, the backbone can still pass the missing effect through an input that tracks CO2, so clamping the input closes just one proxy route.
  • The experiments do not show that the gains come from rerouting. In advection-diffusion, the pathway is the exact solver difference, so the result holds by construction and the baseline gets no physics. In SHiELD, there is no control that fine-tunes without rerouting, no run that clamps CO2 without the pathway, and no test of sensitivity to the pathway size. The abrupt four-times-CO2 test lasts 90 days, so it mostly measures the fast radiative response that the pathway supplies. On the fixed CO2 AMIP run, the closest match to the ERA5 query, ReRoute is slightly worse. The model trained with random CO2 data is never reported. The ERA5 result has no reference and treats near zero as correct, although the true land response is not zero, and a model that ignores CO2 would also give near zero.
Questions for authors
  • How does the theorem apply when other inputs, such as sea surface temperature, are correlated with CO2?
  • What were the random CO2 evaluation settings (range and length), the CO2 value held fixed in the SHiELD fixed-CO2 run, the SHiELD fine-tuning settings, and the loss used for the stochastic ACE2S model?
  • Please report fine-tuning without rerouting, clamping CO2 without the pathway, and results with the pathway scaled up and down.
8 comments
AuthorsOpenReview ↗

Response to Reviewer rK5t (Part 1/4): theory and pathway misspecification

Thank you very much for your careful review. We ran your suggested experiments and added them to the revision. They show that (1) the gains come from rerouting, (2) hold over a 10-year horizon, and (3) are closest to a SHiELD ground truth; the identifiability theory covers both climate queries.

 

[W1, Q1] The theorem covers the climate queries in the paper, and the correlation with SST does not violate its assumptions.

  • Both types of climate query are covered:
    1. Prescribed SST (AMIP constant-CO₂, random-CO₂, ERA5 what-if): SST is in the fixed context Z, so $P(X \mid \mathrm{do}(S = s'))$ is exactly the climate query, which holds SST fixed.
    2. Slab ocean (abrupt-4×CO₂): SST is computed from the surface energy budget, so it is in X and responds to CO₂ through the learned dynamics. Z contains only q-flux, mixed-layer depth and insolation, none of which depend on CO₂.
  • Correlation does not violate source exogeneity. Assumption A.4 concerns the noise terms $U_S$ and $U_X$, not the variables: for example, a child $X_1 = f_1(S, U_1)$ with $U_1 \perp U_S$ still depends on $S$.

Manuscript update: clarified in Remark A.3.

Outside the training range, only the physical pathway has to extrapolate, not the learned model, and this works in practice. Random-CO₂ reaches ≈5800 ppm, four times the largest training value. With a known pathway, target determination only requires the pathway values to hold beyond the training range, not the backbone to directly extrapolate in inputs such as CO₂ and SST. This is a much easier requirement: the pathway comes from a radiative-transfer model (RRTMG), and the near-logarithmic dependence of CO₂ radiative effects over this range is well-established physics that can be checked offline, whereas the behaviour of a learned backbone outside its training data cannot. Although this assumption (Asm. A.5) cannot be guaranteed in general, empirically, we observe that ReRoute still strongly reduces the random-CO₂ error (T1a from Response part 2). Manuscript update: Remark A.2(3); Table 1

ReRoute is robust to substantial pathway misspecification (T2 from Response part 3). We agree that if the pathway captures only part of the CO₂ effect, the backbone can still route the rest through inputs that track CO₂, so clamping closes only the direct route; in practice, any scale from ×0.75 to ×2 still beats the baseline model on both extreme CO₂-shift tests, and even ×0.5 helps on abrupt-4×CO₂. We also agree that Lean formalization verifies that the result follows from the stated assumptions, but does not validate the assumptions themselves. Researchers must therefore assess whether these assumptions are appropriate for each application. Manuscript update: § F.1.3 and Table 5.

AuthorsOpenReview ↗

Response to Reviewer rK5t (Part 2/4): climate ablations and protocol

[W2, Q3] Summarizing all ablations, in both systems: rerouting is the key component.

  • Rerouting, without fine-tuning (zero-shot), already provides most of the improvement in counterfactual prediction over the pretrained baseline in both systems.
  • Clamping the backbone’s source input to its reference value is not sufficient, as expected: it simply evaluates the pretrained model at that value, without adding the physical correction from the reference to the queried value. Fine-tuning without rerouting does not improve counterfactual skill.
  • Fine-tuning the rerouted model improves it further, on factual and counterfactual tests alike.
  • Fixed-CO₂ AMIP: ReRoute's mean error is indeed slightly higher than the baseline model's (T1a). For fine-tuned ReRoute, the difference is within one standard deviation.
  • The model trained with random-CO₂ data is now reported as a reference model (T1a).

 

T1a: Component ablations with reference comparison (bold: best per column and every value whose gap to it is within one standard deviation; protocol below. Reference not ranked.)

ModelComp.AMIP hist.AMIP const-CO₂4×CO₂Rand-CO₂
Trained with additional random-CO₂ data
Reference—0.0418 ± 0.00750.0356 ± 0.00100.1181 ± 0.01280.0452 ± 0.0037
Trained without random-CO₂ data
Baseline—0.0350 ± 0.00130.0369 ± 0.00040.1595 ± 0.01100.1130 ± 0.0029
Clamp onlyC0.0353 ± 0.00140.0401 ± 0.00130.1439 ± 0.00940.1291 ± 0.0050
Fine-tune onlyF0.0383 ± 0.00080.0395 ± 0.00140.1578 ± 0.01040.1194 ± 0.0022
ReRoute (zero-shot)C+P0.0327 ± 0.00140.0394 ± 0.00210.1324 ± 0.01530.0814 ± 0.0075
ReRoute (fine-tuned)C+P+F0.0320 ± 0.00110.0378 ± 0.00160.1283 ± 0.01120.0670 ± 0.0011

C = Clamp, P = Pathway, F = Fine-tune.

 

[Q2] Training and evaluation protocol.

  • Models: all ablations and ReRoute variants start from the baseline ACE2S-SHiELD+ checkpoint trained without random-CO₂ data (no-RC). In the zero-shot setting, we use the pathway with the pretrained model without additional training, keeping all model weights fixed. ReRoute uses s₀ = 1× (363.43 ppm) and the physical RRTMG pathway.
  • Fine-tuning: all fine-tuned variants use the same factual training data, objective and learning rate (2e-6), each trained for 30 epochs. For both, we use the earliest saved checkpoint (epoch 2), since longer training did not improve held-out factual AMIP skill (2009–2011). Rerouting moves the backbone away from its pretrained operating point; for ReRoute we therefore interpolate between the fine-tuned and baseline weights (WiSE-FT; Wortsman et al., 2022), with the mixing weight (0.25) chosen on the same held-out years. Fine-tune without rerouting worsened both validation climate metrics (held-out AMIP years and held-out equilibrium climates) at every checkpoint, so there was no gain for interpolation to preserve. This is expected: the released checkpoint was already selected on these validation metrics using the same training data (Clark et al., 2026), so further training without rerouting likely only moves it away from that optimum.
  • Objective (ACE2S): 0.9 CRPS + 0.1 energy score on a two-member ensemble, evaluated at the last step of 1-, 2-, 4-, 12- or 20-step rollouts, following Clark et al. (2026).
  • Metric: α is the channel-mean spatial RMSE of normalized time-mean biases (lower is better), reported as mean ± std over: AMIP historical/constant-CO₂ (rollouts from 1979-01-01, following Clark et al. (2026), scored on 2012–2020; 5 stochastic realizations); abrupt-4×CO₂ (90 days, 36 initial conditions); and random-CO₂ (5 years, 5 stochastic realizations). Values therefore differ slightly from the submitted manuscript, and we updated the manuscript accordingly.
  • Random-CO₂ setting: following Clark et al. (2026), CO₂(t) = 2n(t)2^{n(t)} · X₀ with n(t)∼U[−2,2]n(t) \sim U[-2, 2] redrawn every 30 days. Our test member is centred at X₀ = 4× (1453.7 ppm), so CO₂ spans 1×–16× (≈363–5800 ppm).
  • Fixed-CO₂ AMIP: CO₂ is held at 336.6 ppm, versus the historical 391–415 ppm over 2012–2020.

Manuscript update: Table 1, § F.1.2; § F.1.1.

References:

  • Clark, Spencer K., et al. "Disentangling the effects of sea surface temperature and CO₂ in global machine learned weather-climate emulators." arXiv preprint arXiv:2606.07928 (2026).
  • Wortsman, Mitchell, et al. "Robust fine-tuning of zero-shot models." 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022.
AuthorsOpenReview ↗

Response to Reviewer rK5t (Part 3/4): advection–diffusion ablations and climate pathway scaling

T1b: Advection–diffusion component ablations (step-100 RMSE, lower is better. Difference RMSE: error of the intervention-minus-reference change; absolute RMSE: error of the intervention forecast. Averaged over 8 initial conditions and all parameter pairs of the region: 1,681 for the full grid, 380 for far-OOD. Both fine-tuned variants use the same factual data and 15-epoch budget, with the checkpoint of lowest factual validation loss; mean ± std over five fine-tuning seeds.)

ModelComp.Diff. full gridDiff. far-OODAbs. full gridAbs. far-OOD
Baseline (pretrained FNO)—0.03330.07840.03500.0779
Fine-tune onlyF0.0302 ± 0.00060.0757 ± 0.00140.0312 ± 0.00070.0754 ± 0.0017
Rerouting D
Clamp only (D)C0.05040.06900.05140.0695
ReRoute-D (zero-shot)C+P0.02350.03560.02650.0360
ReRoute-D (fine-tuned)C+P+F0.0090 ± 0.00030.0128 ± 0.00040.0110 ± 0.00030.0126 ± 0.0004
Rerouting (c, D)
Clamp only (c, D)C0.44650.60020.44630.5997
ReRoute-(c, D) (zero-shot)C+P0.00880.01140.01330.0131
ReRoute-(c, D) (fine-tuned)C+P+F0.0064 ± 0.00010.0081 ± 0.00020.0088 ± 0.00010.0086 ± 0.0001

C = Clamp, P = Pathway, F = Fine-tune. Rows without ± are deterministic (no weight update).

Advection–diffusion: rerouting is key, and the backbone still learns the dynamics. ReRoute indeed uses additional physical information unavailable to the purely data-driven pretrained emulator. Our ablations show that rerouting is key to using this information effectively: clamping the source inputs alone is insufficient, while combining clamping with the pathway substantially improves counterfactual predictions, with further gains from fine-tuning (T1b). In this controlled setting, the pathway supplies only the one-step operator difference. The backbone must still learn the dynamics at the reference input values, including their dependence on inputs left unclamped, such as the varying advection speed (c) in the (D)-only variant.

Manuscript update: Table 2, § B.4.

 

T2: Pathway scaled from × 0.5 to × 2 (ReRoute zero-shot, s0 = 1×. α, lower is better: channel-mean RMSE of normalized time-mean biases; mean ± std over 5 realizations for AMIP, scored 2012–2020, and random-CO₂, 5 years; over 36 initial conditions for abrupt-4×CO₂, 90 days. Full protocol in Part 2. Bold: best per column and every value whose gap to it is within one standard deviation.)

ModelAMIP hist.AMIP const-CO₂4×CO₂Rand-CO₂
Baseline no-RC0.0350 ± 0.00130.0369 ± 0.00040.1595 ± 0.01100.1130 ± 0.0029
Pathway×0.50.0351 ± 0.00150.0384 ± 0.00160.1334 ± 0.01030.1083 ± 0.0059
Pathway×0.750.0324 ± 0.00230.0398 ± 0.00080.1316 ± 0.01130.0956 ± 0.0068
Pathway×1 (physical)0.0327 ± 0.00140.0394 ± 0.00210.1324 ± 0.01530.0814 ± 0.0075
Pathway×1.250.0335 ± 0.00370.0377 ± 0.00180.1326 ± 0.01340.0764 ± 0.0037
Pathway×1.50.0334 ± 0.00180.0392 ± 0.00220.1343 ± 0.01360.0756 ± 0.0024
Pathway×20.0330 ± 0.00090.0396 ± 0.00280.1432 ± 0.01240.0888 ± 0.0023

ReRoute's gain does not hinge on the exact pathway scale. Zero-shot ReRoute already works well, so we use the physical RRTMG pathway as is and without backbone fine-tuning, which cleanly separates the effect of the pathway from learning. Halving or doubling it still beats the baseline model on abrupt-4×CO₂ with factual skill unchanged, and the physical scale is within one standard deviation of the best on every extreme CO₂-shift test. We deliberately do not tune the scale: selecting it would require the controlled counterfactual data that ReRoute is designed to do without.

Manuscript update: Table 5, § F.1.3.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers and AC

Dear AC and Reviewers,

We sincerely thank all reviewers for their thoughtful and constructive feedback. We appreciate their positive assessment of ReRoute's simple and practical approach (rK5t, WFFn, Fnap), computational efficiency (rK5t, WFFn), clear presentation (rK5t, WFFn, Fnap), and encouraging results (WFFn, Fnap). Reviewers also highlighted the novel use of hybrid modeling (Fnap), the theoretical foundation with explicit, Lean-verified assumptions (rK5t, WFFn), and the reference-based evaluation on SHiELD (rK5t). In particular, we are grateful for their recognition of the importance of counterfactual prediction when controlled intervention data are unavailable (rK5t, WFFn, Fnap).

The suggestions helped strengthen the manuscript and we are very grateful. Below we group related questions and summarize the experiments and clarifications that address them. The individual replies to each reviewer provide the complete point-by-point responses.

Main result of the rebuttal: rerouting is the key component. We ran the suggested ablations, in both climate and Advection-Diffusion systems. Without any fine-tuning, zero-shot ReRoute already gives most of the counterfactual gain, and fine-tuning the rerouted model adds a large further step; clamping alone and ordinary fine-tuning on the same data do not consistently help. Fine-tuned ReRoute closes most of the gap to the model trained on expensive controlled simulations.

New experiments

  • Component ablations, climate and advection–diffusion (rK5t, WFFn): rerouting is the key component (T1a, T1b; manuscript update: Table 1, § F.1.2; Table 2, § B.4).
  • Reference model trained with random-CO₂ data (rK5t): ReRoute closes most of the gap without such data (T1a; manuscript update: Table 1, § F.1.2).
  • Pathway scaling, 0.5× to 2× (rK5t): robust to misspecification. Zero-shot ReRoute already works well, so we use the physical pathway as is and without weight updates, which cleanly separates the effect of the pathway from learning (T2; manuscript update: Table 5, § F.1.3).
  • Reference-value sweep, 1× to 5×, without weight updates (Fnap): the default 1× is the best practical choice, and gains hold for every reference up to 2× (T5; manuscript update: Table 6, § F.1.4).
  • 10-year abrupt-4×CO₂ (rK5t): ReRoute removes the baseline model's warming drift, while ordinary fine-tuning makes it worse (T3; manuscript update: Table 7, § F.1.5).
  • SHiELD ground truth for the fixed-CO₂ response (rK5t): both ReRoute variants have less than half the baseline model's error (T4; manuscript update: Table 8, § F.1.6).
  • Absolute state errors, advection–diffusion (WFFn): ReRoute is more accurate on the states, not only on the response (T1b; manuscript update: Table 2, § B.4). Climate experiments already reported absolute errors.

Clarifications

  • Theory (rK5t): it covers both climate queries; the correlation between CO₂ and SST does not violate source exogeneity (manuscript update: Remark A.3).
  • Advection–diffusion (rK5t, WFFn, Fnap): a controlled proof of concept, not solved by construction; the exact pathway alone is not enough (manuscript update: §§ 4.1 and B.2–B.4).
  • Protocols (rK5t, WFFn): full training and evaluation settings (manuscript update: § F.1.1).

Manuscript update: all new results and clarifications are included in the uploaded revision under §§ A, B.4, and F.1.1–F.1.6.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.