- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper introduces TokaReconBench, a benchmark that targets transferability of tokamak equilibrium reconstruction. Compared to existing benchmarks, TokaReconBench takes magnetics as reconstruction main inputs, supporting simulation-to-real transferring on the same device, cross-device evaluation, and target-device supervision. For cross-domain evaluation (e.g., from ITER-like simulation to EXL-50U simulation), fine-tuning on target domain is utilized to establish transferability.
Critical strengths
- The authors promised to release their datasets, which can be valuable for tokamak fusion research.
- This work is easy to follow, avoiding overly technical details in fusion science.
Critical weaknesses
- Insufficient ablation. For tasks T4 and T5, the usefulness of pre-training is not justified. The transferability may come solely from the 5% in-domain fine-tuning.
- The significance of this work is not fully justified. The fundamental improvement from the three earlier benchmarks in Table 1 should be discussed.
- This work relies on existing models and does not include model design specifically for the task. Despite the loss functions are designed for the task, they do not show convincing improvement in Table 5.
Questions for authors
- For T4 and T5, could you remove the pre-training phase while keeping other settings the same?
- Could you justify the fundamental differences between TokaReconBench and the three previous benchmarks? Can these benchmarks be easily (like simply asking an AI agent to do the job) modified or extended to support magnetics inputs, same-device sim2real, cross-device evaluation, and target-device supervision?
7 comments
Response to Reviewer 77kf (2/3)
(Continued from Part 1.)
Concern 2
Weakness2: The significance of this work is not fully justified. The fundamental improvement from the three earlier benchmarks in Table 1 should be discussed.
Question2: Could you justify the fundamental differences between TokaReconBench and the three previous benchmarks? Can these benchmarks be easily (like simply asking an AI agent to do the job) modified or extended to support magnetics inputs, same-device sim2real, cross-device evaluation, and target-device supervision?
Reply:
We thank the reviewer for raising this question.
In the Related Work section of the original manuscript, lines 730 to 740 in Appendix A, we provided an overview of the distinctions between the three existing related benchmarks and our work.
In general, a scientific benchmark generally comprises three core elements:
-
Dataset: In TokaReconBench, this encompasses public MAST experimental data, ITER like simulation data, and our newly contributed EXL-50U simulation and experimental datasets (Section 2.3).
-
Research problem: Predicting the internal poloidal flux distribution and magnetic geometry directly from external magnetic diagnostic signals (Section 2.1).
-
Evaluation tasks: Five train and test domain pairs designed to systematically evaluate same device generalization, simulation to real transfer, and cross-device transfer (Section 2.2).
Among these three elements, paired multi regime data serves as the essential foundation that enables the problem formulation and evaluation tasks. By contributing configuration aligned simulation and experimental datasets for EXL-50U and pairing them with MAST experimental data and ITER like simulation data, TokaReconBench supports all five evaluation tasks, including same device simulation to real transfer and cross-device transfer under limited target supervision. Conversely, the absence of certain data modalities in earlier benchmarks imposes fundamental limitations that cannot be resolved by simple code modifications or model development:
-
TokaMark [Rousseau et al., 2026] contains only experimental data from a single device (MAST). Because it covers only one device, cross-device equilibrium reconstruction cannot be evaluated. Moreover, even if combined with external ITER-like simulation data, it lacks paired simulation data on MAST and therefore cannot support same device simulation to real evaluation (task in our benchmark).
-
FusionEQ [Nakkina et al., 2026] includes experimental data only for DIII-D and MAST without corresponding simulation datasets, preventing simulation to real evaluation. Furthermore, FusionEQ uses non magnetic signals, such as coil currents and Thomson scattering profiles, as inputs to infer equilibria for future reactor scenarios. Because of these differences in input modalities and problem formulation, FusionEQ addresses a fundamentally different conditional inference task. In contrast, TokaReconBench targets the primary operational requirement in existing tokamak facilities: real time equilibrium reconstruction driven by external magnetic diagnostics.
-
CrossGS [Yoo et al., 2026] differs from our work even more fundamentally. First, its inputs are not magnetic diagnostics or experimental measurements, but rather four equilibrium parameters together with the two spatial coordinates , forming six input channels. The model learns to evaluate the forward mapping from known parameters to , rather than solving the inverse equilibrium reconstruction problem from external sensors. Second, CrossGS relies exclusively on analytic equilibria without experimental discharges, making simulation to real evaluation impossible. Third, all devices in CrossGS share the exact same six dimensional parameter space, reducing cross-device transfer to degenerated covariate shift problem within a homogeneous input space. In contrast, cross-device tasks in TokaReconBench reflect realistic deployment conditions where sensor counts, spatial locations, orientations, coil layouts, and output grids differ substantially across devices, requiring transfer across heterogeneous domains.
In summary, TokaReconBench differs fundamentally from these three earlier benchmarks in data availability, inverse problem formulation, and cross-domain evaluation scope. Because existing benchmarks lack paired simulation and experimental datasets on the same device (such as EXL-50U) along with standardized external magnetic diagnostics across multiple machines, they cannot support this comprehensive evaluation of equilibrium reconstruction generalization. This is an intrinsic data and problem formulation barrier that cannot be overcome simply by using an AI agent.
The detailed comparative analysis is located in Appendix A.
(Continued in Part 3.)
Response to Reviewer 77kf (3/3)
(Continued from Part 2.)
Concern 3
Weakness3: This work relies on existing models and does not include model design specifically for the task. Despite the loss functions are designed for the task, they do not show convincing improvement in Table 5.
Reply:
We thank the reviewer for sharing this perspective. Consistent with established benchmark papers in AI for Science, the primary goal of TokaReconBench is not to propose a new model architecture or training loss, but to release standardized, configuration aligned datasets (including paired simulation and experimental data on EXL-50U), define realistic multi regime evaluation protocols, and establish rigorous baselines across representative architectures and losses. Through this systematic evaluation, we uncover critical bottlenecks, such as the lack of consistent cross-device pretraining gains and the task- and architecture-dependent effects of physics-informed auxiliary losses, that would otherwise remain hidden. We believe exposing these gaps provides essential guidance for the field, and designing specialized architectures and transfer algorithms to overcome them is an important direction for future work.
References
[Casper et al., 2014] Development of the ITER baseline inductive scenario. Nuclear Fusion 54, 013005. https://doi.org/10.1088/0029-5515/54/1/013005
[Shi et al., 2025] Strategy and experimental progress of the EXL-50U spherical torus in support of the EHL-2 project. Plasma Science and Technology 27, 024003. https://doi.org/10.1088/2058-6272/ad9e8f
[Rousseau et al., 2026] TokaMark: A Comprehensive Benchmark for MAST Tokamak Plasma Models. arXiv:2602.10132. https://arxiv.org/abs/2602.10132
[Nakkina et al., 2026] The Fusion Equilibrium Challenge: Inferring Magnetic Geometry Without Magnetic Diagnostics. arXiv:2609.01750. https://arxiv.org/abs/2609.01750
[Yoo et al., 2026] Towards Data-Efficient Cross-Device Generalization of Grad-Shafranov Equilibria via Transfer Learning Neural Operator. arXiv:2606.15512. https://arxiv.org/abs/2606.15512
Response to Reviewer 77kf (1/3)
We thank the reviewer for the thoughtful review. We appreciate the reviewer’s positive assessment of the manuscript’s clear and accessible presentation, and the recognition that the release of our datasets could provide a valuable resource for the tokamak fusion research community.
Reply:
We thank the reviewer for this constructive suggestion. We agree that directly comparing source domain pretraining followed by fine-tuning against training from scratch on the same target domain data helps clarify the exact contribution of pretraining versus target domain supervision.
To clarify this, we conducted a new control experiment during the rebuttal period, covering task T4 (ITER simulation to EXL-50U simulation) and task T5 (MAST experiment to EXL-50U experiment).
Taking the U-Net architecture described in Appendix E.1 as an example, we compared source pretrained models fine-tuned on 5% of the target domain data against randomly initialized models trained from scratch on the exact same 5% target domain subset. The results are reported in following Table 9.
Table 9. Comparison between source pretrained fine-tuning and training from scratch, both using 5% target domain data, on cross-device task T4 and task T5 with U-Net. Both strategies use the same target-training subset and preprocessing. Bold values indicate the better result within each task.
The experimental results show that in task T4, the field reconstruction errors of the two strategies are very close, differing by less than 1%. In task T5, training from scratch reduces the relative flux error (Eψrel) from 0.44637 to 0.42113 (a 5.65% reduction) and reduces the conditional mean LCFS 95th percentile Hausdorff distance (ELCFS95) from 23.85 cm to 19.73 cm (a 17.27% reduction). Thus, in this U-Net control experiment, source-domain pretraining followed by target-domain fine-tuning did not provide a consistent advantage over training from scratch with the same target-data budget.
This behavior may be related to the substantial differences in physical scales and internal electromagnetic environments across tokamak devices [Casper et al., 2014; Shi et al., 2025]. For example, in sampled training data for task T4, the median reference plasma current is 12.41 MA for ITER-like simulation and 0.417 MA for EXL-50U simulation; the corresponding median centered flux RMS values are approximately 4.25 and 0.02149 Wb/rad, respectively, differing by more than two orders of magnitude.
Although we applied dimensionless normalization and data standardization during preprocessing to remove unit inconsistencies, fundamental differences in vessel geometry, poloidal field coil layouts, and magnetic topology make it difficult for priors learned on the source device to generalize directly.
This finding is also consistent with our discussion in Section 3.3 (lines 430 to 431 and lines 463 to 466) of the original manuscript regarding simulation to real and cross-device transfer: the tested two-stage pretraining and fine-tuning recipe remains insufficient to bridge the physical distribution gap across devices.
We have included these experimental results and the physical mechanism analysis in Appendix G.
We will extend these experiments to additional architectures (such as FNO) and include the results in the final version of the paper.
(Continued in Part 2.)