Learn · NPTEL

Course10610216384 lectures

Introduction to parallel programming with OpenMP and MPI

Prof. Yogish Sabharwal · IIT Delhi

Start with lecture 1 At NPTEL

Lectures

week-01

  1. Introduction to Parallel Programming Video
  2. Parallel Architectures and Programming Models Video
  3. Pipelining Video
  4. Superpipelining and VLIW Video
  5. Memory Latency Video
  6. Cache and Temporal Locality Video
  7. Cache, Memory bandwidth and Spatial Locality Video
  8. Intuition for Shared and Distributed Memory architectures Video
  9. Shared and Distributed Memory architectures Video
  10. Interconnection networks in Distributed Memory architectures Video

week-02

  1. OpenMP: A parallel Hello World Program Video
  2. Program with Single thread Video
  3. Program Memory with Multiple threads and Multi-tasking Video
  4. Context Switching Video
  5. OpenMP: Basic thread functions Video
  6. OpenMP: About OpenMP Video
  7. Shared Memory Consistency Models and the Sequential Consistency Model Video
  8. Race Conditions Video
  9. OpenMP: Scoping variables and some race conditions Video
  10. OpenMP: thread private variables and more constructs Video

week-03

  1. Computing sum: first attempt at parallelization Video
  2. Manual distribution of work and critical sections Video
  3. Distributing for loops and reduction Video
  4. Vector-Vector operations (Dot product) Video
  5. Matrix-Vector operations (Matrix-Vector Multiply) Video
  6. Matrix-Matrix operations (Matrix-Matrix Multiply) Video
  7. Introduction to tasks Video
  8. Task queues and task execution Video
  9. Accessing variables in tasks Video
  10. Completion of tasks and scoping variables in tasks Video
  11. Recursive task spawning and pitfalls Video

week-04

  1. Understanding LU Factorization Video
  2. Parallel LU Factorization Video
  3. Locks Video
  4. Advanced Task handling Video
  5. Matrix Multiplication using tasks Video
  6. The OpenMP Shared Memory Consistency Model Video

week-05

  1. Applications finite element method Video
  2. Applications deep learning Video
  3. Introduction to MPI and basic calls Video
  4. MPI calls to send and receive data Video
  5. MPI calls for broadcasting data Video
  6. MPI non blocking calls Video
  7. Application distributed histogram updation Video
  8. MPI collectives and MPI broadcast Video
  9. MPI gathering and scattering collectives Video
  10. MPI reduction and alltoall collectives Video
  11. Discussion on MPI collectives design Video

week-06

  1. Characteriziation of interconnects Video
  2. Linear arrays 2D mesh and torus Video
  3. d dimensional torus Video
  4. Hypercube Video
  5. Trees and cliques Video
  6. Hockney model Video
  7. Broadcast and Reduce with recursive doubling Video
  8. Scatter and Gather with recursive doubling Video
  9. Reduce scatter and All gather with recursive doubling Video
  10. Discussion of message sizes in analysis Video
  11. Revisiting Reduce scatter on 2D mesh Video
  12. Reduce scatter and Allreduce on the Hypercube Video
  13. Alltoall on the Hypercube Video
  14. Lower bounds Video
  15. Pipeline based algorithm for Allreduce Video
  16. An improved algorithm for Alltoall on the Hypercube using E cube routing Video
  17. Pipeline based algorithm for Broadcast Video

week-07

  1. Introduction to parallel graph algorithms Video
  2. Breadth First Search BFS using matrix algebra Video
  3. BFS Shared memory parallelization using OpenMP Video
  4. Distributed memory settings and data distribution Video
  5. Distributed BFS algorithm Video
  6. Performance considerations Video
  7. Prims Algorithm Video
  8. OpenMP based shared memory parallelization for MST Video
  9. MPI based distributed memory parallelization for MST Video

week-08

  1. Sequential Algorithm Adaptation from Prims Video
  2. Parallelization Strategy for Prims algorithm Video
  3. Dry run with the parallel strategy Video
  4. Johnsons algorithm with 1D data distribution Video
  5. Speedup analysis on a grid graph Video
  6. Floyds algorithm for all pair shortest paths Video
  7. Floyds algorithm with 2D data distribution Video
  8. Adaptation to transitive closures Video
  9. Parallelization strategy for connected components Video
  10. Analysis for parallel connected components Video

Syllabus and materials

At NPTEL, as the course publishes them.

Credits and licence

Institution
IIT Delhi
Department
Computer Science and Engineering
Teacher
Prof. Yogish Sabharwal
Level
Undergraduate
Language
English
Subjects
Computer science, Engineering
Source
NPTEL
Licence
CC BY-NC-SA, as NPTEL states it. Non-commercial use only. NPTEL's terms
Attribution
NPTEL states: "Distributed under Creative Commons Attribution-ShareAlike / CC BY - NC - SA", beside a BY-NC-SA badge.
Who can watch
NPTEL gives no rating. Invisible Network shows it to Tweens profiles and older, judged from what the source says about it. How courses are placed