acceptodds
Under review as a conference paper at ICLR 2027

From Local Accuracy to Global Forecasting: Expert Selection for Multistep Multivariable Tokamak Fusion Dynamics

Abstract

Amid growing concerns over the global energy crisis and climate change, fusion energy has emerged as a promising route toward sustainable power, while artificial intelligence is increasingly being used to model and control tokamak plasmas. Yet its dynamics are high-dimensional, strongly coupled, and only partially observed, while practical forecasting requires multistep recursive prediction across heterogeneous physical variables. Existing deep learning predictors are difficult to achieve consistent and accurate predictions across all physical variables and time steps using a single model. Moreover, when using predictions as recursive inputs for future models, local accuracy may not necessarily determine the performance of multi-step predictions. Based on the above discovery, we propose that different models should be adaptively used for different time steps and physical variables, and systematically compared the four Mixture-of-Experts (MoE) frameworks in this work, including the behavior-cloning baseline (Baseline-MoE), DAgger-style MoE (DaD-MoE), reinforcement-learning-based MoE (PPO-MoE), and full-information rollout-aware MoE (FIRE) over a frozen bank of heterogeneous physical predictors. We first evaluate the proposed MoE framework through simulation and further extend them to real-world device measurements. The results show that PPO-MoE achieves the lowest complete-rollout error on simulated MAST-U vertical-displacement events, reducing it by relative to Baseline-MoE. On experimental measurement from the real-world MAST device, FIRE reduces complete-rollout error by relative to the best fixed-expert reference and by relative to Baseline-MoE. These results demonstrate that adaptive expert routing can improve recursive multivariable plasma forecasting and offer a new perspective on forecasting in fusion-energy systems.Code is available at https://anonymous.4open.science/r/test0924-16A6.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
AI for PDE, Nuclear Fusion, Mix-of-Experts, Long-horizon Prediction

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_3FezOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses multistep recursive forecasting of tokamak plasma dynamics, motivated by the finding that no single model is best across all physical variables and horizons, and that one-step accuracy does not predict full-rollout performance. The authors route among a frozen bank of heterogeneous experts, selecting one per physical head at each step, and compare four frameworks: behavior-cloning Baseline-MoE, DAgger-style DaD-MoE, PPO-MoE, and their proposed FIRE, which combines soft full-information supervision with KL-anchored rollout refinement. PPO-MoE performs best on simulated MAST-U data (6.38% below Baseline-MoE), while FIRE performs best on real MAST measurements (12.29% below the strongest fixed expert, 7.81% below Baseline-MoE). The results show that adaptive expert routing, evaluated on complete-rollout error, improves long-horizon multivariable forecasting in fusion-energy systems.

Critical strengths

The paper is well motivated, showing with evidence that no single predictor dominates across variables and rollout steps and that one-step rankings disagree with full-rollout rankings, which justifies both the routing approach and the full-rollout evaluation criterion. The evaluation is rigorous, spanning simulation and real MAST device data with strictly disjoint expert-training, router-training, validation, and test splits. The systematic comparison of four routing frameworks, all outperforming the strongest fixed expert on real data, is complemented by informative appendix analyses, including ablations, routing-entropy studies, and an honest negative transfer result.

Critical weaknesses

One center claim is that per-step adaptive routing improves recursive forecasting, but it never compares against the natural cheap alternatives: a static per-head expert assignment selected on validation rollouts, or simple output averaging/stacking of the experts. The “strongest fixed expert” reference uses one expert for all heads, which is a weak strawman given the paper’s own premise that experts specialize per variable. Without these baselines, it is impossible to tell whether the gains come from step-wise adaptivity (the contribution) or merely from per-head specialization that a trivial static policy would capture.

Questions for authors

The related-work discussion focuses on magnetic-confinement tokamak prediction. Fusion surrogate modeling also spans other regimes with similar challenges of data scarcity and timing-sensitive targets, such as ICF waveform prediction (e.g., "Decomposition-Guided Diffusion Language Models for Inertial Confinement Fusion Prediction" EMNLP), where physics-driven PPO rewards have likewise been used. Could the authors comment on whether the proposed routing framework, or the observation that one-step accuracy does not determine rollout quality, is expected to carry over to non-recursive fusion-prediction settings?

2 comments
AuthorsOpenReview ↗

Response to Reviewer 3Fez 1

1. Add per-variable fixed-expert and expert-averaging baselines to assess whether dynamic routing provides additional gains beyond fixed per-variable specialization.

Main weakness

Thank you for this suggestion. The single fixed-expert baseline in the original manuscript used the same expert for all physical quantities. In the revised manuscript, we added two baselines, Static per-head and Uniform average, to compare fixed per-variable expert specialization and simple output averaging with dynamic routing.

For Static per-head, we first select the best-performing expert for each physical quantity based on each expert’s full-rollout validation error. We then keep this assignment fixed throughout recursive forecasting on the test set. At each step, the designated experts generate predictions for their respective quantities. We assemble these predictions into a complete state and feed it back into the next step. This allows different physical quantities to use different experts while keeping the assignment fixed across forecast steps. For Uniform average, all experts receive the same current state at each step. We average their predictions with equal weights for each physical quantity, assemble the resulting state, and feed it back into the next step.

These results have been added to Tables 1–6 of the revised manuscript. Appendix A.4 of the revised manuscript describes the experimental settings.

Response Table 1. New baseline results on simulated MAST-U (relative ℓ2\ell_2 error)

EvaluationMethodActive coilsPassive coilsIpI_pψ\psiJtorJ_{\mathrm{tor}}Macro
One step (Table 5)Static per-head0.0000378820.0144740.0000359020.000103300.000386020.0030074
One step (Table 5)Uniform average0.0000370560.0141190.0000356680.000108360.000417830.0029436
10 ms endpoint (Table 1)Static per-head0.00416630.242830.00293570.00765360.0179830.055114
10 ms endpoint (Table 1)Uniform average0.00410150.228570.00292150.00770770.0180150.052263
Full rollout (Table 6)Static per-head0.00119270.170740.00100380.00227960.00578840.036201
Full rollout (Table 6)Uniform average0.00117730.169460.00100710.00233180.00595270.035985

Response Table 2. New baseline results on experimental MAST (pooled NRMSE)

EvaluationMethodCoil currentSolenoid currentIpI_pψ\psiMacro
One step (Table 3)Static per-head0.010542840.007477830.010725190.022678180.01285601
One step (Table 3)Uniform average0.010782820.006667330.010930620.022790120.01279272
100 ms endpoint (Table 4)Static per-head0.116081130.072232630.067665030.078387210.08359150
100 ms endpoint (Table 4)Uniform average0.118761990.076650640.065217480.073191190.08345533
Full rollout (Table 2)Static per-head0.088939220.061689020.056936200.074929490.07062348
Full rollout (Table 2)Uniform average0.088843990.064882890.053808740.071472900.06975213

Over complete rollouts on experimental MAST, FIRE reduced Macro NRMSE by 7.88% and 6.72% relative to Static per-head and Uniform average, respectively. Over complete rollouts on MAST-U, PPO reduced the Macro relative ℓ2\ell_2 error by 3.05% and 2.47% relative to those two baselines, respectively. These comparisons show that, under the settings evaluated in this study, dynamic routing provides further gains in recursive forecasting beyond fixed per-variable expert specialization and simple output averaging.

AuthorsOpenReview ↗

Response to Reviewer 3Fez 2

2. Discuss how the routing framework and the insight that local accuracy may not reflect final task quality could extend to non-recursive fusion prediction, such as ICF waveform prediction.

Question for the authors

Thank you for this suggestion, which helps us broaden the discussion to a wider range of fusion prediction tasks. In Appendix A.11, “Limitations and Future Directions,” of the revised manuscript, we added a discussion of how the routing framework could extend to non-recursive prediction tasks such as inertial confinement fusion (ICF), and we cite the ICF-DLM paper there [1]. The revised manuscript identifies this extension as a direction for future work and notes that prediction quality could be evaluated using pointwise error, neutron yield, and peak timing.

Predicting ICF outcomes likewise involves limited experimental data and sensitivity to the timing of key events. The ICF-DLM study you mentioned directly predicts neutron production rate waveforms from laser pulses and target capsule parameters. It uses a physics-driven PPO reward that accounts for neutron yield, peak timing, and waveform fidelity, providing a useful reference for our discussion.

More generally, our study suggests that local prediction metrics may not adequately reflect performance on the physical task of interest. This insight also applies to how we evaluate non-recursive predictions. For example, when predicting a complete ICF waveform directly, average pointwise error alone may not capture how accurately the model predicts the timing and local shape of a brief emission peak or the integrated neutron yield. In both types of tasks, this motivates aligning model optimization and evaluation more closely with the physical objectives.

If complementary experts are available for a non-recursive task, the router could select experts based on inputs such as laser pulses and target capsule parameters. The router could then be optimized using physical metrics computed from the final combined prediction. This would require adapting the routing inputs, output structure, and training objectives to direct waveform prediction so that expert selection serves the goals of predicting waveform shape, peak timing, and neutron yield.

References

[1] Xiang Zhang, Varchas Gopalaswamy, Rahman Ejaz, Riccardo Betti, and Dongfang Liu. 2026. Decomposition-Guided Diffusion Language Models for Inertial Confinement Fusion Prediction. In Findings of the Association for Computational Linguistics: EMNLP 2026, pp. 13567–13582, Budapest, Hungary. Association for Computational Linguistics. DOI: 10.18653/2026.findings-emnlp.646.

Reviewer_bwFVOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper is about tokamak forecasting in an autoregressive setting, focusing on the MAST device. The authors observe that different models are best for different variables and horizons. The paper asks whether choosing between a fixed set of (frozen) surrogates at each step gives better rollouts than any single one

To tackle this, the authors propose what I see as an autoregressive ensemble with time-dependent model selection. At each step a router picks one frozen surrogate per variable group, and that choice changes the next input. Routers are learned RL-style models.

Experiments on simulated MAST-U and real (?) MAST data show lower rollout error.

Critical strengths

Error accumulation in long autoregressive rollouts is a real and long-standing issue for surrogate models. This is even more important in high-stakes physical systems such as tokamaks. The task is hard and the paper evaluates on data from a real device.

Learning when to trust which of several available surrogates is a simple and useful idea for settings where models cannot be replaced or retrained easily.

Critical weaknesses
  1. Data origin. I think the dataset used is FAIR-MAST. It is however never clearly named. The simulated MAST-U data is also of unknown origin, but information on the simulator is provided.
  • Real data is only "the MAST device", a "Level-2 all-shot archive" (lines 723-727). No shot list or split manifest is given.
  • Psi on real MAST seems to be an equilibrium reconstruction (figure 5), but is called "measured" (figure 6).
  1. Related work. RL-based dynamic ensembling and model selection for forecasting is not cited. With a few quick searches I could find several papers that learn an RL policy to weight or select forecasting models at each step [1, 2, 3]. [2] also feeds the ensemble back to the base models to reduce autoregressive error accumulation.
  • "MoE" is misleading in my opinion. This is selection over frozen, hand-built experts. I would call it dynamic ensemble selection, or an autoregressive ensemble with a learned selection policy.
  • The "discovery" (lines 019-022) is well known in surrogate modeling and time series. Error accumulation leads to OOD and some good one-step models are not robust.
  1. Writing. The abstract and other parts are hard to follow for me, and there are english mistakes.
  • line 017-019 is not correct english. suggested: "Existing deep learning predictors struggle to give accurate and consistent predictions for all variables...."
  • line 019-021 is unclear. suggested: "When predictions are fed back as inputs, one-step accuracy does not necessarily determine long term accuracy."
  • lines 343-344 calls MAST-U, a real device, "a simulated fusion-energy system". I understand simulations are used, but the wording is wrong.
  • experts have different names in Fig. 1 and in the text.
  • The framing is overly dramatic and generic, in my opinion (e.g. global energy crisis).
  • the naming of FIRE is inconsistent (024-027 vs 101)
  1. Results and inconsistencies. The comparisons use a weak reference and mix seed ensembles with single models (captions of Tables 1, 2 and 11). The code is incomplete, and some numbers are inconsistent (e.g. the Macro definition). NOTE: I had Claude go through the code and the paper to find these, and I did not check the code myself.
  • Main tables are three-seed ensembles of routers, Baseline-MoE included (l.365-366, l.419-420). Fixed experts are single models. That it is a seed ensemble is only said in the appendix (l.948-951). Per-seed FIRE on MAST is 0.0665 against 0.0651 for the ensemble (Table 10 vs Table 2).
  • On MAST-U, gains are quoted over Baseline-MoE, which is worse than the best fixed expert. Against it, the routers gain 0.7-3.0% over the full rollout (Table 6) and 8.7-11.6% at the 10 ms endpoint (Table 1). A per-variable best fixed expert is not compared. It explains about a third of the 12.29% on MAST (my computation from Table 2: 3.6 of 12.3 points).
  • On MAST, FIRE's components have no significant effect (Table 10, CIs include zero), and soft targets are worse than hard ones (single seed). On MAST-U, No-KL is also not significant, yet the text says the KL term contributes (l.1234-1235).
  • The MAST FIRE Stage B training loop is missing from the code (train_stage_b.py, l.100-101). There is no code to extract the data or to build the MAST splits, and no simulator code. The PPO config flags the test set as previously accessed ("formal_test_accessed": true).
  • Checkpoint selection is not equal. Baseline-MoE is selected by teacher-forced validation cross-entropy only. DaD-MoE, PPO-MoE and FIRE are selected by complete-rollout validation error, and FIRE's beta is tuned on validation (l.957-959, l.966-968, l.974-975, l.980-982). This may explain part of the gap to the main reference, Baseline-MoE.
  • 6.38% (abstract) and 15.46% (Sec. 4.2.1) are different metrics (full rollout vs 10 ms endpoint). Macro in Tables 5 and 6 is not the mean of the columns, unlike its stated definition (l.991-992).

References: [1] Fu et al. Reinforcement Learning Based Dynamic Model Combination for Time Series Forecasting. https://ojs.aaai.org/index.php/AAAI/article/view/20618 [2] Ding et al. Dynamic ensemble for probabilistic time-series forecasting via deep reinforcement learning. https://amazon.science/publications/dynamic-ensemble-for-probabilistic-time-series-forecasting-via-deep-reinforcement-learning [3] Feng and Zhang. Reinforcement Learning based Dynamic Model Selection for Short-Term Load Forecasting. https://arxiv.org/abs/1811.01846

Questions for authors
  1. Is the real MAST data FAIR-MAST, or something else? Is the psi field a reconstruction? Is any real MAST-U measurement used, or is MAST-U only simulated? TokaMind is cited. Is it the same data?
  2. What does Fig. 6 show? The text calls it "agreement" (l.1045-1049), but P3L and P3U are not tracked and Ip drifts after about 100 ms
  3. What are the results for per-seed (non-ensembled) routers with standard deviations, the best fixed expert per variable, and an average of the experts?
  4. How does the method differ from RL-based dynamic ensembling [1, 2, 3]? Would they be applicable baselines?
  5. Can you release the missing code (Stage B for MAST FIRE, data extraction and split construction) and confirm that the reported numbers come from it? Was the formal test set used before the final choices were made?
6 comments
AuthorsOpenReview ↗

Response to Reviewer bwFV 1

1. Clarify the provenance and splits of both datasets, and the meaning of the magnetic flux field in the real MAST dataset.

Weakness 1 and Question 1

Thank you for this suggestion. The original manuscript cited related work such as TokaMind and described the MAST Level-2 archive and data splits in Appendix A.1, but the information on data provenance was spread across the manuscript. Appendix A.1 of the revised manuscript now explicitly states that the experimental MAST data come from a local snapshot of the MAST Level-2 archive. It also provides the public FAIR-MAST Level-2 access guide, an example of data preprocessing, and the shot IDs of all 664 discharges in the formal test set.

The magnetic flux field ψ\psi in the experimental MAST data comes from equilibrium reconstructions in the public archive. To clarify this point, Figures 6 and 7 of the revised manuscript explicitly label the reference flux field as an equilibrium reconstruction. As explained in the original manuscript, we generated the MAST-U data using the FreeGSNKE simulator to study vertical displacement events. Appendix A.1 of the revised manuscript retains the description of the simulation procedure and adds a link to simulation examples in the anonymous repository.

2. Clarify the focus of this work on fusion forecasting and its relationship to reinforcement-learning-based dynamic ensemble methods.

Related-work comment in Weakness 2 and Question 4

We developed a forecasting framework that combines an expert bank with a router to predict coupled physical quantities over multiple time steps in magnetic confinement fusion. The expert bank is designed around the characteristics of different physical quantities in experimental device data, and the router selects an expert for each quantity at each forecast step. Our focus is on whether this combination improves recursive forecasting in fusion systems and on comparing the overall performance of three routing approaches: DaD, PPO, and FIRE.

Thank you for pointing out these related studies. In the revised manuscript, we cite Fu et al. [1], Ding et al. [2], and Feng and Zhang [3] in the related work discussion and Appendix A.9, and explain how their work relates to ours in dynamic weighting, model selection, and recursive feedback. We further examine how selecting experts separately for each physical quantity from a frozen expert bank affects the subsequent evolution of the coupled state.

3. Explain the meaning and rationale of the term “MoE” in this work.

MoE terminology comment in Weakness 2

Your suggested term “dynamic ensemble selection” describes the expert-selection mechanism in our framework. We used the term “MoE” in the original manuscript because the framework uses a learnable router to select experts for different physical quantities based on the current state. MoE implementations vary across research areas. In our setting, the experts are pretrained and frozen, and the router is trained separately. To make this setting explicit, we added references including Lédée et al. [4] and Ismail et al. [5] where the Introduction of the revised manuscript first introduces MoE. We also clarified that the router selects among frozen predictors.

4. Revise unclear or overly strong wording and make the terminology consistent across the text and figures.

Comment on the use of “discovery” and Weakness 3

Thank you for pointing out these wording issues. In the revised manuscript, we rewrote the sentences in the Abstract about multivariate forecasting and recursive feedback and now describe the difference between one-step accuracy and recursive forecasting performance as an empirical observation. The revised manuscript also clarifies that the MAST-U experiments are numerical simulations of the device.

Figure 1 is where the experts are first introduced. The original manuscript used descriptive names in this figure to help readers distinguish the experts. The revised figure pairs these names with the abbreviations used in the main text, such as Fast physics (ODE0.25) and Dual head fusion (Dual), and refers readers to Appendix A.2 for their definitions. We also made the full name of FIRE consistent across the Abstract and the main text.

The energy background follows a common motivation in fusion research. The Introduction of the original manuscript cites a Nature Physics perspective on disruption prediction [6] and a Nature Communications study of TCV experiments [7] to explain the motivation for pursuing magnetic confinement fusion as a potential source of low-carbon electricity and illustrate how plasma forecasting can support tokamak operation.

The Introduction then describes the specific challenges addressed in this work: coupling among multiple physical quantities, partial observability, and recursive forecasting.

AuthorsOpenReview ↗

Response to Reviewer bwFV 2

5. Clarify how results across random seeds are aggregated and add per-variable fixed experts, uniform expert averaging, and a post hoc error envelope.

Ensembling and baseline comments in Weakness 4 and Question 3

Table 2 of the original manuscript uses probability ensembling. It averages the routing probabilities from three random seeds before selecting experts. Table 10 of the original manuscript evaluates each seed independently and reports the mean and population standard deviation of the errors. Thus, the FIRE results of 0.06649 and 0.06506 on experimental MAST are the mean error across independently evaluated seeds and the probability-ensemble error, respectively. The revised manuscript clarifies these two reporting conventions in the table notes and adds independent-seed results for DaD, PPO, and FIRE in Table A.2.

Response Table 1. Full-rollout Macro error across three independent seeds (mean ± population standard deviation, Table A.2)

DatasetDaDPPOFIRE
Simulated MAST-U (relative ℓ2\ell_2 error)0.036102 ± 0.0003160.035299 ± 0.0003030.036495 ± 0.000337
Experimental MAST (pooled NRMSE)0.06722859 ± 0.001473750.06934725 ± 0.000710530.06648753 ± 0.00037548

Following your suggestion, we added two baselines: per-variable fixed experts (Static per-head) and uniform expert averaging (Uniform average). The first selects one fixed expert for each physical quantity based on full-rollout validation error. The second averages the experts’ predictions at each step and feeds the averaged prediction into the next step. The revised manuscript also includes RPNN as an independently trained recurrent forecasting baseline. Appendix A.4 of the revised manuscript describes the experimental settings. The new baseline results from Tables 1–6 are given below.

Response Table 2. Baseline results on simulated MAST-U (relative ℓ2\ell_2 error)

EvaluationMethodActive coilsPassive coilsIpI_pψ\psiJtorJ_{\mathrm{tor}}Macro
One step (Table 5)Static per-head0.0000378820.0144740.0000359020.000103300.000386020.0030074
One step (Table 5)Uniform average0.0000370560.0141190.0000356680.000108360.000417830.0029436
One step (Table 5)RPNN0.0000925480.0526570.0002029980.002986630.002658470.0117195
10 ms endpoint (Table 1)Static per-head0.00416630.242830.00293570.00765360.0179830.055114
10 ms endpoint (Table 1)Uniform average0.00410150.228570.00292150.00770770.0180150.052263
10 ms endpoint (Table 1)RPNN0.00357830.272190.00286970.00914240.0169460.060946
Full rollout (Table 6)Static per-head0.00119270.170740.00100380.00227960.00578840.036201
Full rollout (Table 6)Uniform average0.00117730.169460.00100710.00233180.00595270.035985
Full rollout (Table 6)RPNN0.00105800.196830.00103380.00445900.00646950.041970

Response Table 3. Baseline results on experimental MAST (pooled NRMSE)

EvaluationMethodCoil currentSolenoid currentIpI_pψ\psiMacro
One step (Table 3)Static per-head0.010542840.007477830.010725190.022678180.01285601
One step (Table 3)Uniform average0.010782820.006667330.010930620.022790120.01279272
One step (Table 3)RPNN0.009145000.007073930.010288880.024074520.01264558
100 ms endpoint (Table 4)Static per-head0.116081130.072232630.067665030.078387210.08359150
100 ms endpoint (Table 4)Uniform average0.118761990.076650640.065217480.073191190.08345533
100 ms endpoint (Table 4)RPNN0.063363260.057664000.060944620.068463020.06260872
Full rollout (Table 2)Static per-head0.088939220.061689020.056936200.074929490.07062348
Full rollout (Table 2)Uniform average0.088843990.064882890.053808740.071472900.06975213
Full rollout (Table 2)RPNN0.086346160.063832650.050737180.072708620.06840615

In the full-rollout evaluation on experimental MAST, FIRE achieved a Macro NRMSE of 0.06506, below the three new baselines at 0.07062, 0.06975, and 0.06841. On simulated MAST-U, PPO achieved a Macro relative ℓ2\ell_2 error of 0.035095, also below the three new baselines at 0.036201, 0.035985, and 0.041970.

Appendix A.4 of the revised manuscript also adds a post hoc lower error envelope. We first let each expert complete its own recursive rollout. Using ground truth, we then take the lowest error across experts for each forecast step and physical quantity in each test trajectory. This analysis provides an idealized reference for expert selection and suggests room for further improvement using the existing expert bank.

Response Table 4. Best router and post hoc lower error envelope over full rollouts (Table A.1)

DatasetBest routerRouter Macro errorPost hoc lower-envelope Macro errorRelative error reduction
Simulated MAST-UPPO0.035095430.032561837.22%
Experimental MASTFIRE0.065061540.0554674214.75%
AuthorsOpenReview ↗

Response to Reviewer bwFV 3

6. Align claims about the contributions of FIRE’s components with the uncertainty in the ablation results.

Ablation comment in Weakness 4

Table 10 of the original manuscript reported the ablation results for FIRE’s design choices and their associated uncertainty. Following your comment, we revised the interpretation of these results in Appendix A.6 of the revised manuscript. On MAST-U, the confidence interval supports the error reduction from adding Stage B to Stage A. The paired confidence interval for the error difference associated with the KL constraint includes zero. On experimental MAST, the confidence intervals for the error differences associated with Stage B and the KL constraint both include zero. The soft-label model also performed slightly worse than the hard-label model in the single-seed comparison.

The revised manuscript therefore describes these findings as possible effects of individual components under different data conditions and identifies where uncertainty remains. Comparisons between the complete routing systems and fixed experts support our broader conclusion that dynamic expert routing can improve recursive forecasting.

7. Explain the training and checkpoint-selection settings of the MoE methods and the additional parameter comparisons for FIRE.

Checkpoint-selection comment in Weakness 4

As described in Appendix A.3.3 of the original manuscript, each method selects the best checkpoint according to its specified validation metric. For DaD, PPO, and FIRE, final model selection uses the candidate checkpoint with the lowest full-rollout validation error. Baseline-MoE is a behavior-cloning baseline trained on one-step expert labels, so we select the checkpoint with the lowest teacher-forced validation cross-entropy. Appendix A.3.3 of the revised manuscript retains this description.

We also select FIRE’s anchoring coefficient on the validation set. The original manuscript compared β=0.05\beta=0.05 and β=0.01\beta=0.01 on experimental MAST, and Appendix A.6 of the revised manuscript retains these results. With β=0.05\beta=0.05 and β=0.01\beta=0.01, respectively, the Macro NRMSE values were 0.01263718 and 0.01263737 at one step, 0.07762687 and 0.07760262 at the 100 ms endpoint, and 0.06506154 and 0.06530286 over the full rollout. The two settings produced similar results. Response Tables 2 and 3 in Response 5 report the new comparisons in the revised manuscript with per-variable fixed experts, uniform expert averaging, and RPNN.

8. Address the missing code and the test-access flag, and clarify how the reported experimental results were obtained.

Code comment in Weakness 4 and Question 5

Thank you for pointing out the missing components in the public code. We added the FIRE Stage B training implementation for experimental MAST to the code accompanying the revised manuscript and updated the supporting materials. FreeGSNKE is open source, and we provide a link to the simulator and examples of the MAST-U simulations. Appendix A.1 of the revised manuscript details the data sources, preprocessing, and splitting procedures.

The configuration flag formal_test_accessed: true records access to the test set during the initial demo. For the formal experiments, we used the validation set for checkpoint selection and parameter tuning. We did not use the test set for training or model selection. The reported results come from evaluating the selected models on the predefined formal test split.

9. Explain the improvement percentages for different prediction horizons and clarify the numerical presentation of Macro.

Comment on 6.38%, 15.46%, and Macro in Weakness 4

Thank you for checking the numerical consistency. In the original manuscript, 6.38% is the reduction in Macro error for PPO relative to Baseline-MoE over the complete 20-step recursive rollout, from 0.037486 to 0.035095. The 15.46% reduction is for the same comparison at the 10 ms endpoint, from 0.057533 to 0.048638. The revised manuscript specifies the evaluation range for each result.

Tables 5 and 6 of the original manuscript did not clearly indicate the display scales used for different variables, which made it ambiguous how to average the displayed values. The underlying error values were correct. The revised manuscript now displays the error values directly without additional scaling. Macro remains the equally weighted average of the errors across the five physical quantities.

Reviewer_QeZnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies adaptive expert selection for multistep, multivariable tokamak forecasting. A router selects predictions for different physical quantities from a frozen bank of expert models, and the resulting predictions are fed back into subsequent forecast steps. The paper compares behavior cloning, learner-state data aggregation, PPO-based routing, and FIRE, which combines full-information soft supervision with trajectory-level refinement. Experiments on simulated MAST-U trajectories and experimental MAST data show improvements over individual experts and the behavior-cloning router.

Critical strengths
  1. Paper addresses a real world fusion problem with evaluation on simulated and real experiment data measurements.
  2. Using router to select expert models is well motivated, since different models - ML or physics based solvers can be accurate in different regions of plasma dynamics.
  3. The distinction between one-step accuracy and recursive forecasting performance is interesting, and the results provide evidence that their rankings can differ.
Critical weaknesses
  1. Evidence for long horizon prediction is limited : The simulation experiment covers only 20 prediction steps, or 10 ms. This is a limiting factor, since even plasma trajectories can be longer at different reactors. Longer forecasts are needed to show whether the improvements continue and whether errors eventually grow rapidly. If longer fusion trajectories are unavailable, testing on another dynamical-system dataset would be a useful extension, since the method does not require fusion-specific data.

  2. The paper needs to show prediction error at every forecast step. The main comparisons report one-step error, average error over the forecast, and error at the final step. These summaries do not show when the methods start to differ or how quickly their errors grow. Figure 1 shows expert performance for one example, but a comparison of all routing methods across the test set is needed. Please report error against forecast step for each physical quantity, with uncertainty estimates.

  3. Important baselines are missing. The paper should compare against strong recurrent and Transformer models trained to predict multiple future steps. All methods use use the same input space. The authors do cite (Char, 2024) but do not show its method RPNN as a baseline.

  4. Several things change between methods, making the gains difficult to explain. Appendix A.3 describes extra inputs and network components for PPO compared with the baseline and DaD. FIRE also represents expert choices differently. The methods use different rules for selecting their final checkpoints. Therefore, the results do not clearly show whether improvements come from the training method, the extra information, or the network design. Can this be clarified ? What is the basis for these differences?

  5. The overall benefit of FIRE and MoE-PPO seems very low compared to the much simple DaD framework. Are there specific cases where it performs better ?

Questions for authors
  1. How were the evaluation horizons selected relative to the physical timescales, and are longer trajectories available? Either in simulation or from real data ?

  2. Since the method is not fusion-specific and does make any physics based assumptions in modeling, can authors show performance on other datasets (possibly ones with longer horizons) ?

  3. What is the prediction error at each horizon step ?

  4. How do the methods compare when inputs, router architecture, action factorization, and recursive checkpoint selection are matched?

  5. How does it compare against baselines such as recurrent networks, transformer based models, RPNN (char 2024) ?

3 comments
AuthorsOpenReview ↗

Response to Reviewer QeZn 1

1. Forecast horizon and evaluation beyond fusion

Weakness 1 and Questions 1–2

As you noted, the MAST-U experiment in the original manuscript recursively forecasts 20 steps at 0.5 ms per step, for a total of 10 ms. We chose this horizon to capture the rapid evolution of the vertical displacement events (VDEs) simulated in this study. The experimental MAST data include longer discharge trajectories. Appendix A.5.4 of the revised manuscript reports per-step forecast errors through 200 ms, as detailed in Response 2 below.

Following your suggestion, we added a RealPDEBench cylinder-flow experiment in Appendix A.10 of the revised manuscript. The table below reproduces the mean relative ℓ2\ell_2 errors over the full 30-step rollout from Table A.3 of the revised manuscript. Macro is the equally weighted average of the errors for the three physical quantities.

MethodVelocity uuVelocity vvPressure ppMacro
FNO0.0124320.0473500.0372190.032333
FNO+P0.0123620.0471150.0354450.031641
FNO+UV0.0241040.3819980.0390910.148398
FNO+P+UV0.0122810.0472970.0360780.031885
FIRE0.0119830.0450040.0346670.030551

Compared with FNO+P, the best-performing fixed expert, FIRE reduced the Macro error by 3.44% over 30 steps and by 2.76% over 40 steps. We deliberately retained a candidate expert with relatively high error for velocity component vv to make routing for that variable more challenging.

The cylinder-flow experiment predicts two velocity components and pressure within a single flow field. Tokamak forecasting also involves interactions among externally controlled coil currents, plasma current, and magnetic flux. During a recursive rollout, these quantities jointly form the input to the next step, so an error in one quantity can affect subsequent forecasts of the others. This makes it important both to select suitable experts for each physical quantity and to account for how those choices affect later forecast steps. Our router addresses this challenge by selecting experts for each physical quantity at each step. These differences in coupling may partly explain the smaller gains in the cylinder-flow experiment.

2. Per-step forecast errors and uncertainty

Weakness 2 and Question 3

We added per-step error curves for each physical quantity and for Macro in Figures 9–11 of Appendix A.5.4 of the revised manuscript. The shaded bands show pointwise 95% confidence intervals obtained by paired bootstrap resampling. On experimental MAST, the Macro error generally increased with the forecast horizon. At 100 ms, FIRE and DaD had similar Macro NRMSE values of 0.07763 and 0.07780, respectively, both below PPO’s 0.08121. In the 200 ms cohort, DaD had the lowest Macro error point estimate at every step from step 12 (60 ms) through the endpoint, where the errors for DaD, FIRE, and PPO were 0.08755, 0.09192, and 0.09449, respectively. On simulated MAST-U, the Macro relative ℓ2\ell_2 error of each method reached its minimum at step 4 (2 ms) and then increased. From around 4.5 ms, the gap between PPO’s passive-coil error and those of the other two methods widened. At 10 ms, PPO had the lowest error point estimate for all five physical quantities, and the Macro errors for PPO, FIRE, and DaD were 0.04864, 0.04926, and 0.05027, respectively.

AuthorsOpenReview ↗

Response to Reviewer QeZn 2

3. Comparison with the RPNN baseline

Weakness 3 and Question 5

Thank you for suggesting RPNN as a baseline. We cited Char et al. [1] in the original manuscript. Following your suggestion, we adapted RPNN to both the simulated MAST-U and experimental MAST datasets and reported the baseline results in Tables 1 and 2 of the revised manuscript. Appendix A.4 of the revised manuscript describes the adaptation. We trained RPNN separately on each dataset using an encoder–GRU–decoder architecture that predicts a Gaussian distribution over state increments.

The tables below summarize the new per-variable results for RPNN from Tables 1–6 of the revised manuscript, along with the results for the three MoE methods over complete rollouts. Percentages in parentheses give the relative error reduction against full-rollout RPNN for the same dataset and physical quantity. Positive values indicate lower errors, and negative values indicate higher errors.

Simulated MAST-U (relative ℓ2\ell_2 error)

EvaluationMethodActive coilsPassive coilsIpI_pψ\psiJtorJ_{\mathrm{tor}}Macro
One step (Table 5)RPNN0.0000925480.0526570.0002029980.002986630.002658470.0117195
10 ms endpoint (Table 1)RPNN0.00357830.272190.00286970.00914240.0169460.060946
Full rollout (Table 6)RPNN0.00105800.196830.00103380.00445900.00646950.041970
Full rollout (Table 6)DaD0.0011738 (−10.95%)0.16941 (+13.93%)0.0009912 (+4.12%)0.0023084 (+48.23%)0.0058273 (+9.93%)0.035941 (+14.37%)
Full rollout (Table 6)PPO0.0011370 (−7.47%)0.16546 (+15.94%)0.0009788 (+5.32%)0.0022233 (+50.14%)0.0056783 (+12.23%)0.035095 (+16.38%)
Full rollout (Table 6)FIRE0.0011736 (−10.93%)0.16902 (+14.13%)0.0009935 (+3.90%)0.0022622 (+49.27%)0.0057353 (+11.35%)0.035837 (+14.61%)

Experimental MAST (pooled NRMSE)

EvaluationMethodCoil currentSolenoid currentIpI_pψ\psiMacro
One step (Table 3)RPNN0.009145000.007073930.010288880.024074520.01264558
100 ms endpoint (Table 4)RPNN0.063363260.057664000.060944620.068463020.06260872
Full rollout (Table 2)RPNN0.086346160.063832650.050737180.072708620.06840615
Full rollout (Table 2)DaD0.08833368 (−2.30%)0.05493872 (+13.93%)0.04789438 (+5.60%)0.07086322 (+2.54%)0.06550750 (+4.24%)
Full rollout (Table 2)PPO0.08751399 (−1.35%)0.05751838 (+9.89%)0.05283465 (−4.13%)0.07418335 (−2.03%)0.06801259 (+0.58%)
Full rollout (Table 2)FIRE0.08422290 (+2.46%)0.05676223 (+11.08%)0.04713915 (+7.09%)0.07212188 (+0.81%)0.06506154 (+4.89%)

Over complete recursive rollouts, all three MoE methods achieved lower Macro errors than RPNN on both datasets. Relative to RPNN, DaD, PPO, and FIRE reduced the Macro error by 14.37%, 16.38%, and 14.61% on simulated MAST-U and by 4.24%, 0.58%, and 4.89% on experimental MAST, respectively.

AuthorsOpenReview ↗

Response to Reviewer QeZn 3

4. Router design and the effect of expanded observations

Weakness 4 and Question 4

In the main experiments of the original manuscript, we compared the recursive forecasting performance of three complete routing systems using the same frozen expert bank and test protocol within each dataset. We retained each method’s router architecture and action representation because they are part of its design. Each method follows a checkpoint screening procedure suited to its training stages, and all methods use full-rollout validation error for final model selection. In the complete recursive rollout evaluations on MAST and MAST-U, the original DaD, PPO, and FIRE systems each achieved a lower Macro error than the best single fixed expert on the corresponding dataset. This supports our conclusion that dynamic expert selection can improve recursive forecasting.

As you pointed out, PPO and FIRE use richer observations than DaD. To test whether this additional information improves DaD’s full-rollout forecasts, we added an experiment in Appendix A.5.6 of the revised manuscript. We gave DaD the expanded observations used by PPO and FIRE, together with the associated encoder branch, while retaining DaD’s original training procedure, joint action representation, and checkpoint selection rules. Relative to the original DaD, the expanded version reduced full-rollout Macro error by 1.51% on experimental MAST and increased it by 2.05% on MAST-U. Thus, matching the inputs by providing additional information does not necessarily reduce forecast error.

DatasetPhysical quantityError change relative to original DaD
Experimental MASTCoil current−4.41%
Experimental MASTSolenoid current+0.90%
Experimental MASTPlasma current IpI_p−4.47%
Experimental MASTMagnetic flux ψ\psi+2.25%
Experimental MASTMacro−1.51%
Simulated MAST-UActive coils+2.53%
Simulated MAST-UPassive coils+2.12%
Simulated MAST-UPlasma current IpI_p+2.02%
Simulated MAST-UMagnetic flux ψ\psi+0.85%
Simulated MAST-UToroidal current density JtorJ_{\mathrm{tor}}+0.53%
Simulated MAST-UMacro+2.05%

5. Examples of gains over DaD on individual discharges

Weakness 5

Following your suggestion, we added three examples from experimental MAST discharges in Figure 12 of Appendix A.5.5 of the revised manuscript. Relative to DaD, FIRE reduced the coil-current RMSE by 46% over the 200 ms window in shot 21881. PPO reduced the plasma-current RMSE by 47% over the 200 ms window in shot 19880 and by 54% over the 100 ms window in shot 20452.

To explore which discharge characteristics were associated with larger error reductions, we grouped discharges in the 100 ms and 200 ms MAST test cohorts into the highest and lowest quartiles for features such as changes in coil control voltage and magnetic flux. We then compared the mean error reductions of FIRE and PPO relative to DaD between the two groups. After correction for multiple comparisons, none of the examined features showed a reliable difference between the groups. The analysis therefore leaves open which types of discharges benefit most from FIRE or PPO. We report this finding in Appendix A.5.5 of the revised manuscript. In future work, we will investigate how these data characteristics can inform the design of MoE routers.

References

[1] Ian Char, Youngseog Chung, Joseph Abbate, Egemen Kolemen, and Jeff Schneider. 2024. Full Shot Predictions for the DIII-D Tokamak via Deep Recurrent Networks. arXiv preprint arXiv:2404.12416. DOI: 10.48550/arXiv.2404.12416.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank the reviewers for their thorough reviews and constructive suggestions. We appreciate their positive comments on our work, including its focus on a real-world fusion forecasting problem and evaluation on both simulated and experimental device data (Reviewers QeZn, bwFV, and 3Fez); its focus on the important and challenging problem of autoregressive error accumulation, with practical relevance to physical systems such as tokamaks (Reviewer bwFV); its strong motivation (Reviewers QeZn and 3Fez); its insightful observations supported by experimental results (Reviewers QeZn and 3Fez); its rigorous evaluation with strictly disjoint expert-training, router-training, validation, and test splits (Reviewer 3Fez); and its clear presentation that helps deepen understanding of the problem (Reviewer 3Fez). We also believe that our proposed adaptive expert routing framework can make a valuable contribution to research on multistep, multivariable forecasting in fusion systems. Based on the reviewers’ valuable feedback, we have conducted additional experiments and revised the manuscript, which we hope will address their concerns. The main additional experiments and improvements are as follows:

  1. We added per-step prediction error curves and uncertainty analysis. Figures 9–11 in Appendix A.5.4 report per-variable error curves and 95% confidence intervals for experimental MAST through 100 ms and 200 ms, and simulated MAST-U through 10 ms. Appendix A.5.5 also presents discharge examples illustrating the forecasting gains of FIRE and PPO over DaD. See our response to Reviewer QeZn for details.
  2. We added a cylinder-flow experiment in a non-fusion setting. Appendix A.10 presents the new RealPDEBench cylinder-flow forecasting experiment. Relative to the best fixed expert, FIRE reduced the Macro error by 3.44% and 2.76% over 30-step and 40-step rollouts, respectively. The 30-step results are reported in Table A.3. See our response to Reviewer QeZn for details.
  3. We added three baselines: per-variable fixed experts, uniform expert averaging, and RPNN. The new results are included in Tables 1–6, with experimental settings described in Appendix A.4. We also added a post hoc lower error envelope analysis to assess the potential for further improvement with the existing expert bank. See our responses to Reviewers QeZn, bwFV, and 3Fez for details.
  4. We added a comparison using expanded observations and clarified router design and model selection. Appendix A.5.6 presents a new comparison in which DaD uses expanded observations. We also clarified each method’s router design and validation-based model selection procedure in Appendix A.3. See our responses to Reviewers QeZn and bwFV for details.
  5. We improved statistical reporting and clarified error metrics. Table A.2 now reports the mean and standard deviation of errors across independently evaluated random seeds, and we clarify how these statistics differ from probability-ensemble results. We also clarified how error metrics and percentage improvements are calculated, and revised the relevant tables and interpretation of ablation results. See our response to Reviewer bwFV for details.
  6. We expanded the data documentation and accompanying code. Appendix A.1 provides additional information on data sources, preprocessing, and splits, and we clarify that the magnetic flux field in experimental MAST comes from equilibrium reconstruction. The accompanying materials also include the FIRE Stage B training implementation and MAST-U simulation examples. See our response to Reviewer bwFV for details.
  7. We expanded the related-work discussion and improved the manuscript’s presentation. We added a discussion of reinforcement-learning-based dynamic ensembling and model selection, clarified the setting of routing among frozen experts, revised the Abstract, and standardized terminology and names across the text and figures. Appendix A.11 also discusses extensions to non-recursive prediction tasks such as inertial confinement fusion (ICF) forecasting. See our responses to Reviewers bwFV and 3Fez for details.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.