# Scaling the AI Hierarchy of Needs with Hopsworks

- Institution: CERN
- Department: CERN Computing Seminar
- Speaker: Jim Dowling
- Year: 2018
- Language: English
- Subjects: Computer science
- Source: [CERN](https://videos.cern.ch/record/3010261)
- Licence: [CERN](http://copyright.web.cern.ch)
- Who can watch: Tweens profiles and older (sources give no ratings; placed by Invisible Network from the source's own words)

Hopsworks is an open-source data platform that integrates popular platforms for data processing such as Apache Spark, TensorFlow, Hops Hadoop, Kafka, and many others. All services provided by Hopsworks can be accessed using either a REST API or a User Interface. But the real value add of Hopsworks is that it makes Big Data and AI frameworks easier to use by introducing new concepts for collaborative data science (Projects, Users, and Datasets) and ubiquitous support for TLS certificates, opening the platform for integration with the outside world (IoT/mobile devices and external applications).

In this talk, we will discuss Hopsworks and how we are using it to provide Spark/TensorFlow/Hadoop-as-a-Service to hundreds of researchers in Sweden from the Rise ICE Data Center at www.hops.site. In particular, we will examine distributed TensorFlow both for parallel experimentation and distributed training, and how we built a cost-effective distributed deep learning platform-as-a-service on commodity Nvidia GeForce GPUs (1080Ti).

About the speaker

Jim Dowling is as an Associate Professor at KTH Royal Institute of Technology in Stockholm, the CEO of Logical Clocks AB, as well a Senior Researcher at SICS RISE. His research concentrates on building systems support for machine learning at scale. He is the lead architect of Hops Hadoop, the world's fastest and most scalable Hadoop distribution and only Hadoop platform with support for GPUs as a resource. He is a regular speaker at Big Data and AI industry conferences, and blogs at O'Reilly on AI. .

0:00:00 Slide 1

0:03:22 Slide 2

0:04:09 Slide 3

0:04:47 Slide 4

0:05:36 Slide 5

0:05:42 Slide 6

0:05:55 Slide 7

0:06:03 Slide 8

0:06:16 Slide 9

0:06:20 Slide 10

0:06:25 Slide 11

0:06:30 Slide 12

0:06:47 Slide 13

0:07:12 Slide 14

0:08:29 Slide 15

0:08:46 Slide 16

0:09:15 Slide 17

0:09:31 Slide 18

0:09:44 Slide 19

0:09:57 Slide 20

0:10:26 Slide 21

0:10:40 Slide 22

0:11:55 Slide 23

0:12:10 Slide 24

0:12:46…

## Lectures

1. [Scaling the AI Hierarchy of Needs with Hopsworks](https://invisiblenetwork.ai/learn/cern/3010261/de2650379504), 1 h 7 min (CERN's player)