acceptodds
Under review as a conference paper at ICLR 2027

TokamaCon: An Experiment-Derived Parallel Environment Suite for Tokamak Plasma Control

Abstract

Real-time tokamak plasma control is a prerequisite for commercial magnetic-confinement fusion. Operating high-performance tokamak plasmas constitutes an extreme control challenge: controllers rely on sparsely indirect diagnostics, face distribution shift both within and across plasma discharges, adapt to multi-time-scale transport coupling and the effects of magnetohydrodynamic (MHD) instabilities, and obey hard safety limits where violations can terminate the discharge or damage hardware. Yet standard reinforcement-learning environments rarely expose this fusion-specific combination as a simultaneous, interacting control problem, while fusion-control studies lack reusable scenarios and metrics for studying it reproducibly. We introduce TokamaCon, which decouples tokamak real-time control into two representative environment slices, both grounded in EAST data and checked against reference simulators, thereby enabling the development and evaluation of reinforcement-learning algorithms for fusion-control problems. IsoFlux regulates magnetic equilibrium at 1 kHz on a high-frequency switching plant with input delay; Profile regulates kinetic profiles at 100 Hz on a stiff 1D transport PDE with instability-driven events. Both environments provide native PyTorch GPU-vectorised rollouts, reaching roughly IsoFlux and Profile environment steps per second at the recommended batch sizes on a single GPU. We release standardised scenarios, per-episode metrics, provenance metadata, and reference controllers for reproducible comparison of control approaches. Code and scenarios are released at https://anonymous.4open.science/r/TokamaCon_anonymous-CED5.

Primary area:
datasets and benchmarks
Keywords:
Tokamak Plasma Control, Plasma Simulation, GPU-Native Environments, Fusion Energy

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_yaB2OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents TokamaCon, which provides two environments for developing reinforcement learning (RL) algorithms based on the EAST tokamak. IsoFlux addresses the control of the plasma current and boundary shape (i.e., magnetic equilibrium control), while Profile addresses kinetic profile control. Each environment is built on a specific EAST shot, with the dynamics model and initial conditions derived from the measurements and reconstructions of that shot. The paper mainly describes the two environments (state, action, reward, and related physical background) and reports the results of representative RL methods such as proximal policy optimization (PPO) trained on them. In the experiments, the PPO variants appear to learn the task reasonably well on IsoFlux, with fully-observed PPO (PPO-FO) achieving a reward close to the maximum. In contrast, on Profile, all variants remain at a similar and considerably lower reward, suggesting that the task is not effectively learned. Moreover, even on IsoFlux, PPO-FO degrades sharply under distribution shift, such as an unseen plasma equilibrium.

Critical strengths

The proposed environments allow machine learning (ML) researchers to experience tokamak control problems that approximate real-world operation, which could attract interest from the ML community in nuclear fusion.

Critical weaknesses

It remains unclear whether the proposed environments would meaningfully facilitate the development of RL algorithms for tokamak control. The benchmark provides only one main scenario per environment, each built on a single operating point of a single shot from a single machine (EAST), which limits its use for general algorithm comparison.

Questions for authors

I have the following questions for the authors:

  • The paper does not explain why shots 152444 and 137220 were selected, or why the two environments are built on different shots. Could the authors clarify the selection criteria?
  • The plasma control system of EAST already operates with PID-based controllers, and in IsoFlux, the PID baseline outperforms the deployable RL baselines (PPO and PPO-LSTM) while remaining far more robust on an unseen equilibrium. Could the authors clarify in which situations RL is expected to be necessary for tokamak control, and whether the proposed scenarios represent such situations?
  • Since each environment provides only one main scenario, it is difficult to assess whether a policy learns generalizable control behavior or simply fits a specific operating point. Could the authors discuss how the benchmark could support the evaluation of generalization, and how policies developed on it could eventually be transferred to real tokamak operation?
  • It is unclear whether the paper is primarily intended for ML researchers or for fusion researchers. What are the key takeaways of this benchmark, and for which community are they intended?

In addition, I would like to offer the following suggestions:

  • Although ICLR welcomes submissions from diverse application domains, it is fundamentally an ML conference. The current manuscript, however, assumes considerable background in plasma physics, making it difficult to follow for most ML researchers.
  • The font sizes in Figures 1-4 should be increased for better readability.

Overall, this work could serve as a good entry point for ML researchers to tokamak control and attract their interest in this domain. However, I still have concerns about the overall completeness and practical utility of the benchmark, and I therefore recommend rejection in its current form.

Reviewer_uZ4GOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces TokamaCon, a pair of RL environments of tokamak plasmas control, grounded in EAST data. IsoFlux regulates plasma current and boundary flux at 1 kHz using a linearised plant model. Profile regulates the electron temperature profile at 100 Hz with a 1D transport model. The authors compare the GPU environments with reference simulators (EAST PCSVP and RAPTOR) and report throughput. They train a few PPO variants, and include a PID controller for IsoFlux.

Critical strengths

The paper targets a valuable and high impact scientific challenge. Learned control for tokamak plasmas is hard (partial observation, stiff dynamics, safety limits) and open environments are not common. A verifiable, accurate and fast suite would help the community.

Critical weaknesses

TL;DR: I am not in favor of acceptance. In my opinion, the paper is better placed in a nuclear fusion venue rather than ICLR. The core ML-specific contribution is very limited, and validation is too small for a new environment or benchmark paper. Paper, appendix and code disagree in some places. The released code is incomplete.

  1. Scope and ML contribution. The paper releases two reduced simulators. It offers little ML analysis beyond three PPO variants, not enough for a benchmark in an ML venue.
  • Degrave et al. (2022, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8850200/) used a nonlinear simulator. IsoFlux is a linear system around one equilibrium per instance. On a linear plant classical control is strong, and so PID beats PPO. This might not work in favor of benchmarking learned control.
  • The Profile NTM model "has not been experimentally validated" (l.190). It is not clear why a more complete and established model is not used.
  • Gym-TORAX is named as the closest work. There is no comparison or discussion of the transport model differences.
  • IsoFlux parity of 9.8e-5 is called floating-point noise (l.363). IsoFlux runs in fp64 (Table 9), where round-off is much lower at 1e-16 per operation. C.4 (l.864) itself reports 1e-9 against Simulink.
  1. Baselines and evaluation are insufficient. The RL controllers are PPO variants with one shared hyperparameter set. There are no other algorithm families. Table 3 reports single numbers without uncertainty.
  • on the Profile environment rewards lie within 0.009 of each other (0.6329 to 0.6420). l.376 mentions a "multi-seed" protocol, but no seeds or intervals are reported.
  • Etrack, RMHD and TTTF are defined (Sec. 4.1, C.8) but never reported. The main OOD sweep (Fig. 6b) covers IsoFlux only.
  1. Inconsistencies in paper and code. Values in the text disagree with the appendix and with the released code. The release is also incomplete. I could not run the code, so these points come from having an agent go through it. I am reporting the main point from the agent pass.
  • Missing code: the IsoFlux environment (envs/PCSVP/), the IsoFlux PID, setup.py and environment.yml are not in the archive. IsoFlux training imports envs.utils_plot, which exists in neither branch. The Profile eval script imports eight modules that exist in neither branch (PID, Zero, Random, SAC, TD3, Asymmetric_SAC, Priv_PPO_Conv, Priv_PPO_Conv_A1). The mirror's top level has only the two branch folders, so they are not elsewhere.
  • Profile scenario: the text gives seed 0.15 m (l.206, 729, 822) and disruption threshold 0.15 m (l.307, 1007, 1037, 1108). Table 8 gives seed 0.015 m (l.956).
  • Equations and metrics: the bootstrap term of Eq. 3 is largest at w=0 and has no threshold, which contradicts l.185-186. The code uses the standard w/(w²+wd²) form, and a GGJ term that Eq. 3 omits. RMHD (Eq. 10) can only be 0 or 1/T, since its threshold equals the termination threshold (l.1037). I found no code that computes it. The saturation formula (Eq. 12) counts (step, channel) pairs, but l.411-412 and l.1053 say "any actuator". The code matches the formula. The two definitions can differ by up to 12×, so the "roughly 20×" comparison (l.461) holds only for the formula.
Questions for authors

Requests:

  1. Release the complete code, including the IsoFlux environment, the PID and the Profile evaluation entry points, with one command per table.
  2. Report Table 3 and Fig. 6 over several seeds with intervals.
  3. Compare against Gym-TORAX or explain why TORAX was not used for the NTM scenario.

Questions: 4. What are the throughput and speedup for the NTM scenario of Profile? 5. How can a 9.8e-5 error be the noise floor for an fp64 port? Is there accumulation? 6. can you expand and justify on how your work is different from Degrave et al. (2022, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8850200/) or Tracey et al. (2024)? It is my understanding those use nonlinear simulators, making them higher fidelity and advantaging learned control as opposed to simple methods like PID. What is the point of the linear environments?

Reviewer_ZBHSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TokamaCon provides two GPU-based simulation environments for studying reinforcement learning in tokamak control. IsoFlux controls plasma current and shape. Profile controls plasma temperature and aims to suppress neoclassical tearing modes (NTMs). The paper provides example tasks, evaluation measures, and baseline controllers.

Critical strengths

1.The paper addresses a useful problem. Shared simulation environments could make fusion-control research easier to reproduce and compare.

  1. Providing GPU-based simulations and a common interface could help researchers train and test controllers more easily.

  2. The authors compare their simulations with existing simulation software- RAPTOR.

  3. The experiments include a PID controller and a test on an equilibrium not used during training. The authors openly report cases where the learned controllers perform worse.

  4. The appendices clearly acknowledge several important limitations.

Critical weaknesses
  1. The experiments cover very few plasma conditions. The evaluated settings come from only two EAST discharges, and Profile uses just one discharge. Ideally a benchmark should cover validations for many shots (hundreds, if not thousands), not just a couple. It is also unclear why these discharges were selected or how well they represent the conditions the benchmark is meant to cover. These shots also provide the initial conditions hence this environment is too narrow to represent real conditions.

  2. The NTM model has not been checked against experimental island growth and suppression. The authors acknowledge this limitation. Matching the temperature evolution produced by RAPTOR does not show that the NTM model correctly describes how magnetic islands grow or respond to control. Since suppressing these islands is the main Profile task, this leaves uncertainty about what successful performance means for real plasma control.

  3. The effects of the simplifying assumptions are unclear. Profile keeps density fixed, uses simulated temperature measurements, and represents disruption through an island-width threshold. The magnetic and profile tasks also run separately. These choices may be reasonable, but the paper needs to show that they preserve the important difficulties of the control problem. The small heat-diffusivity test does not answer this question for the NTM model or the measurements.

  4. The main text should give more attention to the limitations that affect how the results should be understood, especially the untested NTM model and simplified measurements.

Questions for authors
  1. Why were these two discharges chosen, and what range of plasma conditions should the benchmark cover?

  2. What evidence supports the NTM model settings, and the chosen disruption threshold? What is the basis of selecting this threshold and is this valid for real plasma experiments?

  3. How reliable is the simulator in different initial conditions ? Are they validated?

  4. Can the authors state more clearly which parts have been checked against experiments and which have only been compared with other simulations?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.