Reinforcement Learning in Particle Accelerators: a practical example (2/2)

About

# Proposal: Reinforcement Learning for Particle Accelerator control: A real-world example

## Hour 1: Introduction to Reinforcement Learning for Particle Accelerators

- **Basic Concepts**: - Overview of Reinforcement Learning (RL) fundamentals. - Definitions and distinctions: - Model-free vs. model-based. - Off-policy vs. on-policy approaches. - **Applications and Considerations**: - Discussion of problem types and environmental variables affecting model selection in practical scenarios. - Analysis of drawbacks and benefits of different RL architectures. - **Practical examples**: - Real-world examples of RL in particle accelerators (e.g., CERN). - Case study introduction: Optimization of RF triple splittings in the Proton Synchrotron (PS).

## Hour 2: Optimizing RF Triple Splittings with Reinforcement Learning

- **Problem Definition**: - Explanation of PS RF operations and the triple splitting optimization challenge for LHC-type beams. - Overview of the physics and parameters involved in optimization. - **Optimization Approach**: - Justification for choosing RL and specific RL architectures. - Step-by-step walkthrough: - Initial simulations and trials. - Challenges and lessons learned. - Final operational solution deployed in the control room.

## Exercise Session: Training RL Agents for RF Optimization (1 hour)

- **Objective**: - Train RL agents to optimize RF double splitting settings in simulation for improved beam quality. - **Implementation**: - Use SWAN notebooks with provided skeleton code. - Define a custom gymnasium environment for the double splitting problem, given: - Pre-implemented simulation data loaders. - Basic loss function for optimization..

0:00:00 Slide 1

0:02:13 Slide 2

0:03:51 Slide 3

0:05:10 Slide 4

0:07:18 Slide 5