# Reinforcement Learning in Particle Accelerators: a practical example (2/2)

- Institution: CERN
- Department: Inverted CSC
- Speaker: Joel Axel Wulff
- Year: 2025
- Language: English
- Subjects: Computer science
- Source: [CERN](https://videos.cern.ch/record/3022137)
- Licence: [CERN](http://copyright.web.cern.ch)
- Who can watch: Tweens profiles and older (sources give no ratings; placed by Invisible Network from the source's own words)

# Proposal: Reinforcement Learning for Particle Accelerator control: A real-world example

## Hour 1: Introduction to Reinforcement Learning for Particle Accelerators

- **Basic Concepts**: - Overview of Reinforcement Learning (RL) fundamentals. - Definitions and distinctions: - Model-free vs. model-based. - Off-policy vs. on-policy approaches. - **Applications and Considerations**: - Discussion of problem types and environmental variables affecting model selection in practical scenarios. - Analysis of drawbacks and benefits of different RL architectures. - **Practical examples**: - Real-world examples of RL in particle accelerators (e.g., CERN). - Case study introduction: Optimization of RF triple splittings in the Proton Synchrotron (PS).

## Hour 2: Optimizing RF Triple Splittings with Reinforcement Learning

- **Problem Definition**: - Explanation of PS RF operations and the triple splitting optimization challenge for LHC-type beams. - Overview of the physics and parameters involved in optimization. - **Optimization Approach**: - Justification for choosing RL and specific RL architectures. - Step-by-step walkthrough: - Initial simulations and trials. - Challenges and lessons learned. - Final operational solution deployed in the control room.

## Exercise Session: Training RL Agents for RF Optimization (1 hour)

- **Objective**: - Train RL agents to optimize RF double splitting settings in simulation for improved beam quality. - **Implementation**: - Use SWAN notebooks with provided skeleton code. - Define a custom gymnasium environment for the double splitting problem, given: - Pre-implemented simulation data loaders. - Basic loss function for optimization..

0:00:00 Slide 1

0:02:13 Slide 2

0:03:51 Slide 3

0:05:10 Slide 4

0:07:18 Slide 5

0:08:48 Slide 6

0:10:20 Slide 7

0:14:59 Slide 8

0:16:09 Slide 9

0:17:22 Slide 10

0:21:11 Slide 11

0:23:00 Slide 12

0:26:06 Slide 13

0:27:24 Slide 14

0:34:38 Slide 15

0:36:14 Slide 16

0:37:32 Slide 17

0:38:43…

## Lectures

1. [Reinforcement Learning in Particle Accelerators: a practical example (2/2)](https://invisiblenetwork.ai/learn/cern/3022137/b4a8d032488a), 1 h 4 min (CERN's player)