Hyperparameter Optimization for Deep Learning Models Using High Performance Computing

About

Abstract

Over the past decade, machine learning (ML), and deep learning (DL) in particular, has outperformed traditional rule-based algorithms on a wide variety of tasks, including image recognition, object detection, and natural language processing. These advances have also created new opportunities for scientific research, including in high energy physics (HEP).

Training DL models, however, can be challenging, particularly when models are large and involve many tunable hyperparameters. To address this challenge, hyperparameter optimization (HPO) can be used to systematically explore the search space of possible hyperparameter configurations. When combined with the computational resources of modern high-performance computing (HPC) systems, large-scale HPO can substantially accelerate model development and optimization.

This talk will introduce HPO and discuss the key challenges involved in tuning deep-learning models at scale. It will then present a HEP use case demonstrating the successful application of large-scale HPO on HPC systems.

Bio

Eric Wulff is a data scientist and machine learning engineer in the Computing for Experiments group at CERN's IT department. He specializes in developing and applying AI models to address complex scientific and technical challenges, leveraging large-scale HPC systems for distributed training and hyperparameter optimization. As part of CERN openlab's management team, he provides strategic guidance on the intersection of AI and HPC in openlab R&D projects.

Eric holds a MSc in Engineering Physics from Lund University. Before joining CERN, he worked as a deep learning engineer, focusing on real-time object detection and video analytics using deep learning on edge devices.

Please note that photographs and videos may be taken during the event and used in communications related to it. By attending the lecture, you consent to being featured in these materials.

00:00:00 Slide 1

00:00:34 Slide 2

00:01:43 Slide 3

00:02:58…