# Spark - a modern approach for distributed analytics

- Institution: CERN
- Department: Workshops
- Taught by: Kacper Surdy, Prasanth Kothuri
- Year: 2016
- Language: English
- Source: [CERN](https://videos.cern.ch/record/3009751)
- Licence: [CERN](http://copyright.web.cern.ch)
- Who can watch: Teens profiles and older (sources give no ratings; placed by Invisible Network from the source's own words)

The Hadoop ecosystem is the leading opensource platform for distributed storing and processing big data. It is a very popular system for implementing data warehouses and data lakes. Spark has also emerged to be one of the leading engines for data analytics. The Hadoop platform is available at CERN as a central service provided by the IT department.

By attending the session, a participant will acquire knowledge of the essential concepts need to benefit from the parallel data processing offered by Spark framework. The session is structured around practical examples and tutorials.

Main topics:

Architecture overview - work distribution, concepts of a worker and a driver

Computing concepts of transformations and actions

Data processing APIs - RDD, DataFrame, and SparkSQL

.

0:00:00 Slide 1

0:10:33 Slide 2

0:21:05 Slide 3

0:31:37 Slide 4

0:42:09 Slide 5

0:52:41 Slide 6

1:03:13 Slide 7

1:13:45 Slide 8

1:24:17 Slide 9

1:34:49 Slide 10

1:45:21 Slide 11

## Lectures

1. [Spark - a modern approach for distributed analytics](https://invisiblenetwork.ai/learn/cern/3009751/e7b6da84cf87), 1 h 46 min (CERN's player)