-
Towards practical reinforcement learning for tokamak magnetic control
Authors:
Brendan D. Tracey,
Andrea Michi,
Yuri Chervonyi,
Ian Davies,
Cosmin Paduraru,
Nevena Lazic,
Federico Felici,
Timo Ewalds,
Craig Donner,
Cristian Galperti,
Jonas Buchli,
Michael Neunert,
Andrea Huber,
Jonathan Evens,
Paula Kurylowicz,
Daniel J. Mankowitz,
Martin Riedmiller,
The TCV Team
Abstract:
Reinforcement learning (RL) has shown promising results for real-time control systems, including the domain of plasma magnetic control. However, there are still significant drawbacks compared to traditional feedback control approaches for magnetic confinement. In this work, we address key drawbacks of the RL method; achieving higher control accuracy for desired plasma properties, reducing the stea…
▽ More
Reinforcement learning (RL) has shown promising results for real-time control systems, including the domain of plasma magnetic control. However, there are still significant drawbacks compared to traditional feedback control approaches for magnetic confinement. In this work, we address key drawbacks of the RL method; achieving higher control accuracy for desired plasma properties, reducing the steady-state error, and decreasing the required time to learn new tasks. We build on top of \cite{degrave2022magnetic}, and present algorithmic improvements to the agent architecture and training procedure. We present simulation results that show up to 65\% improvement in shape accuracy, achieve substantial reduction in the long-term bias of the plasma current, and additionally reduce the training time required to learn new tasks by a factor of 3 or more. We present new experiments using the upgraded RL-based controllers on the TCV tokamak, which validate the simulation results achieved, and point the way towards routinely achieving accurate discharges using the RL approach.
△ Less
Submitted 5 October, 2023; v1 submitted 21 July, 2023;
originally announced July 2023.
-
Latest results on quiescent and post-disruption runaway electron mitigation experiments at Frascati Tokamak Upgrade
Authors:
D. Carnevale,
P. Buratti,
M. Baruzzo,
W. Bin,
F. Bombarda,
L. Boncagni,
C. Paz-Soldan,
L. Calacci,
M. Cappelli,
C. Castaldo,
S. Ceccuzzi,
C. Centioli,
C. Cianfarani,
S. Coda,
F. Cordella,
O. D Arcangelo,
J. Decker,
B. Duval,
B. Esposito,
L. Gabellieri,
S. Galeani,
S. Garavaglia,
C. Galperti,
G. Ghillardi,
G. Granucci
, et al. (16 additional authors not shown)
Abstract:
Results from the last FTU campaigns on the deuterium large (wrt FTU volume) pellet REs suppression capability, mainly due to the induced burst MHD activity expelling REs seed are presented for discharges with 0.5 MA and 5.3T. Clear indications of avalanche multiplication of REs following single pellet injection on 0.36 MA flat-top discharges is shown together with quantitative indications of dissi…
▽ More
Results from the last FTU campaigns on the deuterium large (wrt FTU volume) pellet REs suppression capability, mainly due to the induced burst MHD activity expelling REs seed are presented for discharges with 0.5 MA and 5.3T. Clear indications of avalanche multiplication of REs following single pellet injection on 0.36 MA flat-top discharges is shown together with quantitative indications of dissipative effects in terms of critical electrical field increase due to fan-like instabilities. Analysis of large fan-like instabilities on post-disruption RE beams, that seem to be correlated with low electrical field and background density drops, reveal their strong RE energy suppression capability suggesting a new strategy for RE energy suppression controlling large fan instabilities. We demonstrate how such density drops can be induced using modulated ECRH power on post-disruption beams.
△ Less
Submitted 25 May, 2021; v1 submitted 10 May, 2021;
originally announced May 2021.
-
Integrated real-time supervisory management for off-normal-event handling and feedback control of tokamak plasmas
Authors:
T. Vu,
F. Felici,
C. Galperti,
M. Maraschek,
A. Pau,
N. Rispoli,
O. Sauter,
B. Sieglin,
the TCV team,
the MST1 team
Abstract:
For long-pulse tokamaks, one of the main challenges in control strategy is to simultaneously reach multiple control objectives and to robustly handle in real-time (RT) unexpected events (off-normal-events -- ONEs) with a limited set of actuators. We have developed in our previous work a generic architecture of the plasma control system (PCS) including a supervisor and an actuator manager to deal w…
▽ More
For long-pulse tokamaks, one of the main challenges in control strategy is to simultaneously reach multiple control objectives and to robustly handle in real-time (RT) unexpected events (off-normal-events -- ONEs) with a limited set of actuators. We have developed in our previous work a generic architecture of the plasma control system (PCS) including a supervisor and an actuator manager to deal with these issues. We present in this paper recent developments of real-time decision-making by the supervisor to switch between different control scenarios (normal, backup, shutdown, disruption mitigation, etc.) during the discharge, based on off-normal-event states. We first standardize the evaluation of ONEs and thereby simplify significantly the supervisor decision logic, as well as facilitate the modifications and extensions of ONE states in the future. The whole PCS has been implemented on the TCV tokamak, applied to disruption avoidance with density limit experiments, demonstrating the excellent capabilities of the new RT integrated strategy.
△ Less
Submitted 30 October, 2020;
originally announced October 2020.
