# Introduction to parallel programming with OpenMP and MPI

- Institution: IIT Delhi
- Department: Computer Science and Engineering
- Teacher: Prof. Yogish Sabharwal
- Level: Undergraduate
- Language: English
- Subjects: Computer science, Engineering
- Source: [NPTEL](https://nptel.ac.in/courses/106102163)
- Licence: [CC BY-NC-SA](https://nptel.ac.in/faq) (Non-commercial use only.)
- Syllabus: [at the source](https://nptel.ac.in/courses/106102163)
- Who can watch: Tweens profiles and older (sources give no ratings; placed by Invisible Network from the source's own words)

## Lectures

1. [Introduction to Parallel Programming](https://invisiblenetwork.ai/learn/nptel/106102163/d5c4552382b1) (Video)
2. [Parallel Architectures and Programming Models](https://invisiblenetwork.ai/learn/nptel/106102163/341a13963ad0) (Video)
3. [Pipelining](https://invisiblenetwork.ai/learn/nptel/106102163/b13f60847cac) (Video)
4. [Superpipelining and VLIW](https://invisiblenetwork.ai/learn/nptel/106102163/bd81bfa8d784) (Video)
5. [Memory Latency](https://invisiblenetwork.ai/learn/nptel/106102163/cdf1479318e3) (Video)
6. [Cache and Temporal Locality](https://invisiblenetwork.ai/learn/nptel/106102163/0def2d33d156) (Video)
7. [Cache, Memory bandwidth and Spatial Locality](https://invisiblenetwork.ai/learn/nptel/106102163/f4a7fcdfe305) (Video)
8. [Intuition for Shared and Distributed Memory architectures](https://invisiblenetwork.ai/learn/nptel/106102163/6942d3d7f6b6) (Video)
9. [Shared and Distributed Memory architectures](https://invisiblenetwork.ai/learn/nptel/106102163/7e46b15c245e) (Video)
10. [Interconnection networks in Distributed Memory architectures](https://invisiblenetwork.ai/learn/nptel/106102163/a0e6d3aafeef) (Video)
11. [OpenMP: A parallel Hello World Program](https://invisiblenetwork.ai/learn/nptel/106102163/a1b3efdc47ce) (Video)
12. [Program with Single thread](https://invisiblenetwork.ai/learn/nptel/106102163/29c0c4f5550a) (Video)
13. [Program Memory with Multiple threads and Multi-tasking](https://invisiblenetwork.ai/learn/nptel/106102163/9293e124c6f5) (Video)
14. [Context Switching](https://invisiblenetwork.ai/learn/nptel/106102163/83f1d69a5eeb) (Video)
15. [OpenMP: Basic thread functions](https://invisiblenetwork.ai/learn/nptel/106102163/afeb2191eda3) (Video)
16. [OpenMP: About OpenMP](https://invisiblenetwork.ai/learn/nptel/106102163/b405fa6dd380) (Video)
17. [Shared Memory Consistency Models and the Sequential Consistency Model](https://invisiblenetwork.ai/learn/nptel/106102163/8aa895354262) (Video)
18. [Race Conditions](https://invisiblenetwork.ai/learn/nptel/106102163/791fad76b116) (Video)
19. [OpenMP: Scoping variables and some race conditions](https://invisiblenetwork.ai/learn/nptel/106102163/2c263e5b3730) (Video)
20. [OpenMP: thread private variables and more constructs](https://invisiblenetwork.ai/learn/nptel/106102163/c7d446b792e5) (Video)
21. [Computing sum: first attempt at parallelization](https://invisiblenetwork.ai/learn/nptel/106102163/f5a85339964e) (Video)
22. [Manual distribution of work and critical sections](https://invisiblenetwork.ai/learn/nptel/106102163/793e8c0388e9) (Video)
23. [Distributing for loops and reduction](https://invisiblenetwork.ai/learn/nptel/106102163/ba64736d462f) (Video)
24. [Vector-Vector operations (Dot product)](https://invisiblenetwork.ai/learn/nptel/106102163/402d9ccd2d90) (Video)
25. [Matrix-Vector operations (Matrix-Vector Multiply)](https://invisiblenetwork.ai/learn/nptel/106102163/bf548da3725f) (Video)
26. [Matrix-Matrix operations (Matrix-Matrix Multiply)](https://invisiblenetwork.ai/learn/nptel/106102163/e5a26a42c62d) (Video)
27. [Introduction to tasks](https://invisiblenetwork.ai/learn/nptel/106102163/2c9b315456c2) (Video)
28. [Task queues and task execution](https://invisiblenetwork.ai/learn/nptel/106102163/b31cb64396ac) (Video)
29. [Accessing variables in tasks](https://invisiblenetwork.ai/learn/nptel/106102163/7f0cf338a79e) (Video)
30. [Completion of tasks and scoping variables in tasks](https://invisiblenetwork.ai/learn/nptel/106102163/20ac24a653d8) (Video)
31. [Recursive task spawning and pitfalls](https://invisiblenetwork.ai/learn/nptel/106102163/83d111189d18) (Video)
32. [Understanding LU Factorization](https://invisiblenetwork.ai/learn/nptel/106102163/1aa6d283a4bc) (Video)
33. [Parallel LU Factorization](https://invisiblenetwork.ai/learn/nptel/106102163/2af197640590) (Video)
34. [Locks](https://invisiblenetwork.ai/learn/nptel/106102163/aa561e098232) (Video)
35. [Advanced Task handling](https://invisiblenetwork.ai/learn/nptel/106102163/a8a72f754777) (Video)
36. [Matrix Multiplication using tasks](https://invisiblenetwork.ai/learn/nptel/106102163/2605cc790cfc) (Video)
37. [The OpenMP Shared Memory Consistency Model](https://invisiblenetwork.ai/learn/nptel/106102163/877a2253bae9) (Video)
38. [Applications finite element method](https://invisiblenetwork.ai/learn/nptel/106102163/7789106f4d45) (Video)
39. [Applications deep learning](https://invisiblenetwork.ai/learn/nptel/106102163/28c02d5383b1) (Video)
40. [Introduction to MPI and basic calls](https://invisiblenetwork.ai/learn/nptel/106102163/7cafd42b59b0) (Video)
41. [MPI calls to send and receive data](https://invisiblenetwork.ai/learn/nptel/106102163/17d289b94c83) (Video)
42. [MPI calls for broadcasting data](https://invisiblenetwork.ai/learn/nptel/106102163/ccffe813d966) (Video)
43. [MPI non blocking calls](https://invisiblenetwork.ai/learn/nptel/106102163/e34639df3394) (Video)
44. [Application distributed histogram updation](https://invisiblenetwork.ai/learn/nptel/106102163/0554aeb93a1b) (Video)
45. [MPI collectives and MPI broadcast](https://invisiblenetwork.ai/learn/nptel/106102163/40650469cc67) (Video)
46. [MPI gathering and scattering collectives](https://invisiblenetwork.ai/learn/nptel/106102163/c78d4f18b841) (Video)
47. [MPI reduction and alltoall collectives](https://invisiblenetwork.ai/learn/nptel/106102163/6da7dd8003e1) (Video)
48. [Discussion on MPI collectives design](https://invisiblenetwork.ai/learn/nptel/106102163/78e2444cb6be) (Video)
49. [Characteriziation of interconnects](https://invisiblenetwork.ai/learn/nptel/106102163/948fa3cf65aa) (Video)
50. [Linear arrays 2D mesh and torus](https://invisiblenetwork.ai/learn/nptel/106102163/6e95bf28686d) (Video)
51. [d dimensional torus](https://invisiblenetwork.ai/learn/nptel/106102163/d4099533eca1) (Video)
52. [Hypercube](https://invisiblenetwork.ai/learn/nptel/106102163/113077b037cb) (Video)
53. [Trees and cliques](https://invisiblenetwork.ai/learn/nptel/106102163/e7905febc17c) (Video)
54. [Hockney model](https://invisiblenetwork.ai/learn/nptel/106102163/d76ba4aedf8f) (Video)
55. [Broadcast and Reduce with recursive doubling](https://invisiblenetwork.ai/learn/nptel/106102163/28ea2b2fe772) (Video)
56. [Scatter and Gather with recursive doubling](https://invisiblenetwork.ai/learn/nptel/106102163/4450f2ba2b6f) (Video)
57. [Reduce scatter and All gather with recursive doubling](https://invisiblenetwork.ai/learn/nptel/106102163/494bb04fdc79) (Video)
58. [Discussion of message sizes in analysis](https://invisiblenetwork.ai/learn/nptel/106102163/cde1a473dc2a) (Video)
59. [Revisiting Reduce scatter on 2D mesh](https://invisiblenetwork.ai/learn/nptel/106102163/e3f2f1419406) (Video)
60. [Reduce scatter and Allreduce on the Hypercube](https://invisiblenetwork.ai/learn/nptel/106102163/c7b979d9d02c) (Video)
61. [Alltoall on the Hypercube](https://invisiblenetwork.ai/learn/nptel/106102163/15fe67f9d9d1) (Video)
62. [Lower bounds](https://invisiblenetwork.ai/learn/nptel/106102163/a4ced1fe21cb) (Video)
63. [Pipeline based algorithm for Allreduce](https://invisiblenetwork.ai/learn/nptel/106102163/6e6a44add87e) (Video)
64. [An improved algorithm for Alltoall on the Hypercube using E cube routing](https://invisiblenetwork.ai/learn/nptel/106102163/9103740a5a09) (Video)
65. [Pipeline based algorithm for Broadcast](https://invisiblenetwork.ai/learn/nptel/106102163/0a5f2964e93b) (Video)
66. [Introduction to parallel graph algorithms](https://invisiblenetwork.ai/learn/nptel/106102163/18cc732bd08d) (Video)
67. [Breadth First Search BFS using matrix algebra](https://invisiblenetwork.ai/learn/nptel/106102163/2d571d1f37f1) (Video)
68. [BFS Shared memory parallelization using OpenMP](https://invisiblenetwork.ai/learn/nptel/106102163/0d4c5b650fc5) (Video)
69. [Distributed memory settings and data distribution](https://invisiblenetwork.ai/learn/nptel/106102163/db9708b8b9e9) (Video)
70. [Distributed BFS algorithm](https://invisiblenetwork.ai/learn/nptel/106102163/6430cc581c34) (Video)
71. [Performance considerations](https://invisiblenetwork.ai/learn/nptel/106102163/14f5da89eb71) (Video)
72. [Prims Algorithm](https://invisiblenetwork.ai/learn/nptel/106102163/11ce94cf071c) (Video)
73. [OpenMP based shared memory parallelization for MST](https://invisiblenetwork.ai/learn/nptel/106102163/c0905f33a255) (Video)
74. [MPI based distributed memory parallelization for MST](https://invisiblenetwork.ai/learn/nptel/106102163/dcb55b6d3abc) (Video)
75. [Sequential Algorithm Adaptation from Prims](https://invisiblenetwork.ai/learn/nptel/106102163/72e33d553966) (Video)
76. [Parallelization Strategy for Prims algorithm](https://invisiblenetwork.ai/learn/nptel/106102163/7687a417bad4) (Video)
77. [Dry run with the parallel strategy](https://invisiblenetwork.ai/learn/nptel/106102163/026c59d7f260) (Video)
78. [Johnsons algorithm with 1D data distribution](https://invisiblenetwork.ai/learn/nptel/106102163/3619a61e78bb) (Video)
79. [Speedup analysis on a grid graph](https://invisiblenetwork.ai/learn/nptel/106102163/2bc8a1095ea7) (Video)
80. [Floyds algorithm for all pair shortest paths](https://invisiblenetwork.ai/learn/nptel/106102163/de54532f340c) (Video)
81. [Floyds algorithm with 2D data distribution](https://invisiblenetwork.ai/learn/nptel/106102163/84d5229c897e) (Video)
82. [Adaptation to transitive closures](https://invisiblenetwork.ai/learn/nptel/106102163/f689e53c504b) (Video)
83. [Parallelization strategy for connected components](https://invisiblenetwork.ai/learn/nptel/106102163/1fca174dc967) (Video)
84. [Analysis for parallel connected components](https://invisiblenetwork.ai/learn/nptel/106102163/5ab697c3894a) (Video)