# Random Decision Forests on Apache Spark

- Institution: CERN
- Department: CERN Computing Seminar
- Speaker: Tom White
- Year: 2016
- Language: English
- Subjects: Computer science
- Source: [CERN](https://videos.cern.ch/record/3015833)
- Licence: [CERN](http://copyright.web.cern.ch)
- Who can watch: Tweens profiles and older (sources give no ratings; placed by Invisible Network from the source's own words)

Apache Spark continues to gain momentum as the new processing paradigm for Apache Hadoop, and for the data scientist, it has a lot to like: natively distributed, REPL, Python APIs in addition to native Scala, and a library of machine learning algorithms, MLlib.

Spark includes an implementation of random decision forests, an important and popular ensemble classifier/regressor algorithm. This talk will introduce Spark and random decision forests to the curious, and demonstrate the process of analyzing a real-world data set with them. The session will cover loading data and understanding the data set, and introduce ideas like training and test set evaluation, ensemble methods, feature types, and supporting concepts like impurity and entropy.

About the speaker

Tom White has been an Apache Hadoop committer since February 2007, and is a member of the Apache Software Foundation. He works for Cloudera, a company set up to offer Hadoop support and training. Previously he was as an independent Hadoop consultant, working with companies to set up, use, and extend Hadoop. He has written numerous articles for O'Reilly, java.net and IBM's developerWorks, and has spoken at several conferences, including at ApacheCon 2008 on Hadoop. Tom has a Bachelor's degree in Mathematics from the University of Cambridge and a Master's in Philosophy of Science from the University of Leeds, UK. .

0:00:00 Slide 1

0:04:22 Slide 2

0:04:43 Slide 3

0:05:54 Slide 4

0:06:25 Slide 5

0:07:25 Slide 6

0:08:34 Slide 7

0:08:43 Slide 8

0:10:40 Slide 9

0:13:03 Slide 10

0:13:10 Slide 11

0:14:44 Slide 12

0:15:43 Slide 13

0:16:00 Slide 14

0:16:09 Slide 15

0:16:41 Slide 16

0:18:38 Slide 17

0:20:09 Slide 18

0:22:12 Slide 19

0:22:32 Slide 20

0:22:48 Slide 21

0:27:17 Slide 22

0:28:55 Slide 23

0:29:52 Slide 24

0:30:47 Slide 25

0:31:14 Slide 26

0:31:20 Slide 27

0:31:44 Slide 28

0:32:17 Slide 29

0:33:22 Slide 30

0:34:33 Slide 31

0:35:07 Slide 32

0:35:25 Slide 33

0:35:36 Slide 34…

## Lectures

1. [Random Decision Forests on Apache Spark](https://invisiblenetwork.ai/learn/cern/3015833/9c4d41101b8e), 47 min (CERN's player)