Learn · Applied Accelerated Artificial Intelligence
Lecture 32 of 51Week 7
Fundamentals of Distributed AI Computing Session 2 Part 2
Prof. Satyajit Das and Prof. Satyadhyan Chickerur · IIT Guwahati KLE Technological University
Credits and licence
- From
- Applied Accelerated Artificial Intelligence, IIT Guwahati KLE Technological University (at NPTEL)
- Licence
- CC BY-NC-SA, as NPTEL states it. Non-commercial use only.
- Attribution
- NPTEL states: "Distributed under Creative Commons Attribution-ShareAlike / CC BY - NC - SA", beside a BY-NC-SA badge. Coordinated by IIT Guwahati.
- Who can watch
- Tweens profiles and older. How courses are placed
All the lectures
- Lecture 1: CPU Architecture
- Lecture 2 : Memory Hierarchy
- Lecture 3 : GPU Fundamentals & Modern
- Lecture 4: Interconnects - PCIe, NVLink and NVSwitch
- Lecture 5 : AI Software stack - OS, Drivers and Runtime Layers
- Lecture 1 : Docker fundamentals for GPU - enabled AI workloads
- Lecture 2 : Deterministic Environment & Containerized Jupyter
- Lecture 3 : CUDA in Containers & CI Validation
- Lecture 4 : Schedulers & GPU Allocation in Multi-User Environments
- Lecture 1 : Kubernetes Fundamentals for AI Workloads
- Lecture 2 : GPU Pod Scheduling,Node Selector, Tolerations
- Lectrue 3 : K8s Networking & Configuration for AI Workloads
- Lecture 4 : ConfigMaps, Secrets & Config Injection
- Lecture 1 : Tensors & Autograd
- Lecture 2 : Building Models with nn.Module
- Lecture 3 : The Training Loop
- Lecture 4 : Data loaded & Feeding the model efficiently
- Lecture 5 : First steps in performance
- Lecture 1 : TensorFlow Execution Models & Performance Basics
- Lecture 2 : Optimization Input Pipelines with tf.data
- Lecture 3 : Graph Execution & TensorFlow Optimization
- Lecture 4 : Introduction to XLA Compilation
- Lecture 5 : Model Interoperability: SavedModel vs ONNX
- Lecture 1 : LLM Foundation and terminology
- Lecture 2 : Mixed precision and numerical stability
- Lecture 3 : Memory Optimization and throughput
- Lecture 4 : Transfer learning for large models
- lecture 5 : Efficient LLM fine-tuning using PEFT AND LoRA
- Fundamentals of Distributed AI Computing Session 1 Part 1
- Fundamentals of Distributed AI Computing Session 1 Part 2
- Fundamentals of Distributed AI Computing Session 2 Part 1
- Fundamentals of Distributed AI Computing Session 2 Part 2
- Distributed Deep Learning using Tensorflow and Horovod
- Challenges with Distributed Deep Learning Training Convergence
- Fundamentals of Accelerating Deployment part 1
- Fundamentals of Accelerating Deployment part 2
- Accelerating neural network inference in PyTorch and TensorFlow part 1
- Accelerating neural network inference in PyTorch and TensorFlow part 2
- Accelerated Data Analytics part 1
- Accelerated Data Analytics part 2
- Accelerated Data Analytics part 3
- Accelerated Data Analytics part 4
- Accelerated Machine Learning
- Introduction to NLP Part 1
- Introduction to NLP Part 2
- Applied AI: Smart City ( Intelligent Video Analytics) Session 1 part 1
- Applied AI: Smart City ( Intelligent Video Analytics) Session 1 part 2
- Applied AI: Smart City ( Intelligent Video Analytics) Session 2 Deepstream part 1
- Applied AI: Smart City ( Intelligent Video Analytics) Session 2 Deepstream part 2
- Introduction to word embedding
- Text classification using word embedding