# Foundation models

- Institution: CERN
- Department: CERN openlab summer student lecture programme
- Taught by: Sofia Vallecorsa, Ilaria Luise
- Year: 2024
- Language: English
- Source: [CERN](https://videos.cern.ch/record/3012941)
- Licence: [CERN](http://copyright.web.cern.ch)
- Who can watch: Teens profiles and older (sources give no ratings; placed by Invisible Network from the source's own words)

Description

Foundation models, also known as large-scale self-supervised models, have revolutionized the field of artificial intelligence. These models, such as ChatGPT and AlphaFold, are pre-trained on massive amounts of data and can be fine-tuned for a wide range of downstream tasks. In this lecture, we'll explore the key concepts behind foundation models and their impact on machine learning systems. In particular we will give a brief overview of the points below:

What are foundation models? Challenges and opportunities.

Strategies for training foundation models : self-supervision and pre-training.

How to reach adaptability and fine tuning.

Some examples

Bio

Ilaria Luise is a Senior Research Fellow at CERN, the European Center for Nuclear Research in Geneva. She works as a physicist within the Innovation Division at the CERN IT-Department. Her background is in experimental physics and big data management. She is Co-PI of the AtmoRep project, which is part of the CERN Innovation Programme on Environmental Applications (CIPEA). The project aims at building a foundation model for atmospheric dynamics in collaboration with ECMWF and the Jülich Supercomputing Center.

Sofia is a CERN physicist with extensive experience in software development in the high-energy physics domain, particularly in deep learning and quantum computing applications within CERN openlab. She has a PhD in physics obtained at the University of Geneva. Prior to joining CERN openlab, Sofia was responsible for the development of deep-learning-based technologies for the simulation of particle transport through detectors at CERN. She also worked to optimise the GeantV detector simulation prototype on modern hardware architectures. .

0:00:00 Slide 1

0:03:33 Slide 2

0:06:30 Slide 3

0:10:05 Slide 4

0:12:01 Slide 5

0:14:05 Slide 6

0:17:06 Slide 7

0:20:59 Slide 8

0:23:19 Slide 9

0:25:24 Slide 10

0:28:23 Slide 11

0:31:16 Slide 12

0:34:16 Slide 13

0:37:01 Slide 14

0:39:36 Slide 15…

## Lectures

1. [Foundation models](https://invisiblenetwork.ai/learn/cern/3012941/c87b91ce9986), 1 h 59 min (CERN's player)