# Reinforcement Learning in Particle Accelerators: a practical example (1/2)

- Institution: CERN
- Department: Inverted CSC
- Speaker: Joel Axel Wulff
- Year: 2025
- Language: English
- Subjects: Computer science
- Source: [CERN](https://videos.cern.ch/record/3022136)
- Licence: [CERN](http://copyright.web.cern.ch)
- Who can watch: Tweens profiles and older (sources give no ratings; placed by Invisible Network from the source's own words)

# Proposal: Reinforcement Learning for Particle Accelerator control: A real-world example

## Hour 1: Introduction to Reinforcement Learning for Particle Accelerators

- **Basic Concepts**: - Overview of Reinforcement Learning (RL) fundamentals. - Definitions and distinctions: - Model-free vs. model-based. - Off-policy vs. on-policy approaches. - **Applications and Considerations**: - Discussion of problem types and environmental variables affecting model selection in practical scenarios. - Analysis of drawbacks and benefits of different RL architectures. - **Practical examples**: - Real-world examples of RL in particle accelerators (e.g., CERN). - Case study introduction: Optimization of RF triple splittings in the Proton Synchrotron (PS).

## Hour 2: Optimizing RF Triple Splittings with Reinforcement Learning

- **Problem Definition**: - Explanation of PS RF operations and the triple splitting optimization challenge for LHC-type beams. - Overview of the physics and parameters involved in optimization. - **Optimization Approach**: - Justification for choosing RL and specific RL architectures. - Step-by-step walkthrough: - Initial simulations and trials. - Challenges and lessons learned. - Final operational solution deployed in the control room.

## Exercise Session: Training RL Agents for RF Optimization (1 hour)

- **Objective**: - Train RL agents to optimize RF double splitting settings in simulation for improved beam quality. - **Implementation**: - Use SWAN notebooks with provided skeleton code. - Define a custom gymnasium environment for the double splitting problem, given: - Pre-implemented simulation data loaders. - Basic loss function for optimization..

0:00:00 Slide 1

0:01:24 Slide 2

0:11:48 Slide 3

0:13:38 Slide 4

0:15:59 Slide 5

0:20:02 Slide 6

0:21:05 Slide 7

0:22:17 Slide 8

0:23:22 Slide 9

0:26:40 Slide 10

0:27:49 Slide 11

0:29:01 Slide 12

0:30:03 Slide 13

0:33:22 Slide 14

0:34:42 Slide 15

0:41:09 Slide 16

0:44:00 Slide 17

0:45:02…

## Lectures

1. [Reinforcement Learning in Particle Accelerators: a practical example (1/2)](https://invisiblenetwork.ai/learn/cern/3022136/bed530db9a0a), 55 min (CERN's player)