Speaker
Description
Optimizing control systems in particle accelerators presents significant challenges, often requiring extensive manual effort and expert knowledge. Traditional tuning methods are time-consuming and may struggle to navigate the complexity of modern beamline architectures. To address these challenges, we introduce a simulation-based framework that leverages Reinforcement Learning (RL) [1] to enhance the control and optimization of beam transport systems. Built on top of the Elegant simulation engine [2], our Python-based platform automates the generation of simulations and transforms accelerator tuning tasks into RL environments with minimal user intervention. The framework features a modified Soft Actor-Critic (SAC) agent [3] enhanced with curriculum learning techniques [4], enabling robust performance across a variety of beamline configurations. Designed with accessibility and flexibility in mind, the system can be deployed by non-experts and adapted to optimize virtually any beamline. Early results demonstrate successful application across multiple simulated beamlines, validating the approach and offering promising potential for broader adoption. We continue to refine the framework toward a general-purpose solution—one that can serve both as an intelligent co-pilot for physicists and a testbed for RL researchers developing new algorithms. This work highlights the growing synergy between AI and accelerator physics [1, 3], and the critical role of computational innovation [2] in advancing experimental capabilities.
-
Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction (2nd ed.). MIT Press. http://incompleteideas.net/book/the-book-2nd.html
-
Borland, M. (2000). elegant: A flexible SDDS-compliant code for accelerator simulation. 6th International Computational Accelerator Physics Conference (ICAP 2000). https://doi.org/10.2172/761286
-
Haarnoja, T., Zhou, A., Abbeel, P., & Levine, S. (2018). Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In Proceedings of the 35th International Conference on Machine Learning (pp. 1861–1870). PMLR.
-
Narvekar, S., Peng, B., Leonetti, M., Sinapov, J., Taylor, M. E., & Stone, P. (2020). Curriculum learning for reinforcement learning domains: A framework and survey. Journal of Machine Learning Research, 21(181), 1–50. https://www.jmlr.org/papers/volume21/20-212/20-212.pdf
