- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents TokamaCon, which provides two environments for developing reinforcement learning (RL) algorithms based on the EAST tokamak. IsoFlux addresses the control of the plasma current and boundary shape (i.e., magnetic equilibrium control), while Profile addresses kinetic profile control. Each environment is built on a specific EAST shot, with the dynamics model and initial conditions derived from the measurements and reconstructions of that shot. The paper mainly describes the two environments (state, action, reward, and related physical background) and reports the results of representative RL methods such as proximal policy optimization (PPO) trained on them. In the experiments, the PPO variants appear to learn the task reasonably well on IsoFlux, with fully-observed PPO (PPO-FO) achieving a reward close to the maximum. In contrast, on Profile, all variants remain at a similar and considerably lower reward, suggesting that the task is not effectively learned. Moreover, even on IsoFlux, PPO-FO degrades sharply under distribution shift, such as an unseen plasma equilibrium.
Critical strengths
The proposed environments allow machine learning (ML) researchers to experience tokamak control problems that approximate real-world operation, which could attract interest from the ML community in nuclear fusion.
Critical weaknesses
It remains unclear whether the proposed environments would meaningfully facilitate the development of RL algorithms for tokamak control. The benchmark provides only one main scenario per environment, each built on a single operating point of a single shot from a single machine (EAST), which limits its use for general algorithm comparison.
Questions for authors
I have the following questions for the authors:
- The paper does not explain why shots 152444 and 137220 were selected, or why the two environments are built on different shots. Could the authors clarify the selection criteria?
- The plasma control system of EAST already operates with PID-based controllers, and in IsoFlux, the PID baseline outperforms the deployable RL baselines (PPO and PPO-LSTM) while remaining far more robust on an unseen equilibrium. Could the authors clarify in which situations RL is expected to be necessary for tokamak control, and whether the proposed scenarios represent such situations?
- Since each environment provides only one main scenario, it is difficult to assess whether a policy learns generalizable control behavior or simply fits a specific operating point. Could the authors discuss how the benchmark could support the evaluation of generalization, and how policies developed on it could eventually be transferred to real tokamak operation?
- It is unclear whether the paper is primarily intended for ML researchers or for fusion researchers. What are the key takeaways of this benchmark, and for which community are they intended?
In addition, I would like to offer the following suggestions:
- Although ICLR welcomes submissions from diverse application domains, it is fundamentally an ML conference. The current manuscript, however, assumes considerable background in plasma physics, making it difficult to follow for most ML researchers.
- The font sizes in Figures 1-4 should be increased for better readability.
Overall, this work could serve as a good entry point for ML researchers to tokamak control and attract their interest in this domain. However, I still have concerns about the overall completeness and practical utility of the benchmark, and I therefore recommend rejection in its current form.