# Apache Airflow in Production: Orchestrating Data Harvesting Pipelines

- Institution: CERN
- Speaker: Joao Ramiro
- Year: 2026
- Language: English
- Source: [CERN](https://videos.cern.ch/record/3021039)
- Licence: [CERN](https://copyright.web.cern)
- Who can watch: Teens profiles and older (sources give no ratings; placed by Invisible Network from the source's own words)

Apache Airflow (\[https://airflow.apache.org/\]\[1\]) is a key tool we use to orchestrate our data harvesting processes and manage production workflows. This talk will show how Airflow helps us design, schedule, and monitor harvesting pipelines, with concrete examples from daily operations. We will also highlight the main pros and cons we have encountered, including operational and scaling considerations. The session is aimed at engineers and technical users interested in practical workflow orchestration.

\[1\]: https://airflow.apache.org/

0:00:00 Slide 1

0:03:58 Slide 2

0:04:35 Slide 3

0:06:22 Slide 4

0:06:40 Slide 5

0:07:35 Slide 6

0:08:56 Slide 7

0:19:07 Slide 8

0:21:57 Slide 9

0:25:19 Slide 10

0:25:44 Slide 11

0:26:17 Slide 12

0:27:42 Slide 13

0:29:22 Slide 14

0:30:38 Slide 15

0:34:05 Slide 16

0:34:27 Slide 17

0:37:58 Slide 18

0:39:03 Slide 19

0:40:30 Slide 20

0:40:50 Slide 21

0:44:00 Slide 22

0:45:06 Slide 23

0:45:31 Slide 24

0:45:45 Slide 25

0:46:12 Slide 26br>

## Lectures

1. [Apache Airflow in Production: Orchestrating Data Harvesting Pipelines](https://invisiblenetwork.ai/learn/cern/3021039/9218a39dcad8), 48 min (CERN's player)