# LLMs in Production: RAG pipelines and beyond

- Institution: CERN
- Department: Inverted CSC
- Speaker: Jack Charlie Munday
- Year: 2025
- Language: English
- Subjects: Computer science
- Source: [CERN](https://videos.cern.ch/record/3022068)
- Licence: [CERN](http://copyright.web.cern.ch)
- Who can watch: Tweens profiles and older (sources give no ratings; placed by Invisible Network from the source's own words)

Running Large Language Models (LLMs) in production presents lots of complexities extending far beyond your choice in model. Key challenges include:

* How do you address knowledge staleness (i.e. your model being trained on out of date / not relevant information)? * How do you balance cost optimisation with model latency? * How do you reduce bias and factual hallucinations?

A widely adopted approach to address these is Retrieval Augmented Generation (RAG).

RAG pipelines implement a two-tiered approach (Retrieval & Generation): allowing models to be given domain-specific information prior to generating their response to a question. Through these techniques, "Off the Shelf" LLMs can be applied to a much wider domain context than what they were originally trained for.

In this lecture, we will explore how to improve the adaptability of LLMs without the need for fine-tuning: covering RAG and related architectures, physics-based approaches like `entropix` that allow for self-reasoning / context aware sampling, and the challenges with applying these techniques in a production context..

0:00:00 Slide 1

0:01:41 Slide 2

0:03:29 Slide 3

0:06:07 Slide 4

0:07:30 Slide 5

0:08:58 Slide 6

0:11:47 Slide 7

0:15:10 Slide 8

0:16:49 Slide 9

0:19:22 Slide 10

0:20:45 Slide 11

0:23:09 Slide 12

0:24:09 Slide 13

0:25:25 Slide 14

0:26:45 Slide 15

0:28:26 Slide 16

0:29:48 Slide 17

0:31:40 Slide 18

0:34:06 Slide 19

0:36:27 Slide 20

0:37:39 Slide 21

0:39:43 Slide 22

0:43:22 Slide 23

0:44:32 Slide 24

0:46:22 Slide 25

0:50:19 Slide 26

0:53:57 Slide 27

0:56:25 Slide 28

## Lectures

1. [LLMs in Production: RAG pipelines and beyond](https://invisiblenetwork.ai/learn/cern/3022068/22e1936800d9), 1 h 1 min (CERN's player)