# Deep Learning - IIT Ropar

- Institution: IIT Ropar
- Department: Computer Science and Engineering
- Teacher: Prof. Sudarshan Iyengar
- Level: Undergraduate
- Language: English
- Subjects: Computer science, Engineering
- Source: [NPTEL](https://nptel.ac.in/courses/106106184)
- Licence: [CC BY-NC-SA](https://nptel.ac.in/faq) (Non-commercial use only.)
- Syllabus: [at the source](https://nptel.ac.in/courses/106106184)
- Who can watch: Tweens profiles and older (sources give no ratings; placed by Invisible Network from the source's own words)

Deep Learning has received a lot of attention over the past few years and has been employed successfully by companies like Google, Microsoft, IBM, Facebook, Twitter etc. to solve a wide range of problems in Computer Vision and Natural Language Processing. In this course we will learn about the building blocks used in these Deep Learning based solutions. Specifically, we will learn about feedforward neural networks, convolutional neural networks, recurrent neural networks and attention mechanisms. We will also look at various optimization algorithms such as Gradient Descent, Nesterov Accelerated Gradient Descent, Adam, AdaGrad and RMSProp which are used for training such deep neural networks. At the end of this course students would have knowledge of deep architectures used for solving various Vision and NLP tasks

Intended audience: Any Interested Learners

Prerequisites: Working knowledge of Linear Algebra, Probability Theory. It would be beneficial if the participants have done a course on Machine Learning.

## Lectures

1. [Biological Neuron](https://invisiblenetwork.ai/learn/nptel/106106184/2893f6ca7a0b) (YouTube)
2. [From Spring to Winter of AI](https://invisiblenetwork.ai/learn/nptel/106106184/44f877be7f4a) (YouTube)
3. [The Deep Revival](https://invisiblenetwork.ai/learn/nptel/106106184/c08b045c2c5f) (YouTube)
4. [From Cats to Convolutional Neural Networks](https://invisiblenetwork.ai/learn/nptel/106106184/958f70287f10) (YouTube)
5. [Faster, higher, stronger](https://invisiblenetwork.ai/learn/nptel/106106184/9827c80af1bb) (YouTube)
6. [The Curious Case of Sequences](https://invisiblenetwork.ai/learn/nptel/106106184/54214595d479) (YouTube)
7. [Beating humans at their own games (literally)](https://invisiblenetwork.ai/learn/nptel/106106184/17fcafb6d648) (YouTube)
8. [The Madness (2013-)](https://invisiblenetwork.ai/learn/nptel/106106184/c437b972394e) (YouTube)
9. [(Need for) Sanity](https://invisiblenetwork.ai/learn/nptel/106106184/b838977b5710) (YouTube)
10. [Motivation from Biological Neurons](https://invisiblenetwork.ai/learn/nptel/106106184/db3318a0dcc1) (YouTube)
11. [McCulloch Pitts Neuron, Thresholding Logic](https://invisiblenetwork.ai/learn/nptel/106106184/e7023c7d3cec) (YouTube)
12. [Perceptrons](https://invisiblenetwork.ai/learn/nptel/106106184/619fa2372352) (YouTube)
13. [Error and Error Surfaces](https://invisiblenetwork.ai/learn/nptel/106106184/4a4464cfbf31) (YouTube)
14. [Perceptron Learning Algorithm](https://invisiblenetwork.ai/learn/nptel/106106184/919c15d6d9bc) (YouTube)
15. [Proof of Convergence of Perceptron Learning Algorithm](https://invisiblenetwork.ai/learn/nptel/106106184/b76b7c3ecc02) (YouTube)
16. [Linearly Separable Boolean Functions](https://invisiblenetwork.ai/learn/nptel/106106184/fc2023c92d7f) (YouTube)
17. [Representation Power of a Network of Perceptrons](https://invisiblenetwork.ai/learn/nptel/106106184/4ab08850919f) (YouTube)
18. [Sigmoid Neuron](https://invisiblenetwork.ai/learn/nptel/106106184/967473e62e7f) (YouTube)
19. [A typical Supervised Machine Learning Setup](https://invisiblenetwork.ai/learn/nptel/106106184/1cfd5151bcdd) (YouTube)
20. [Learning Parameters: (Infeasible) guess work](https://invisiblenetwork.ai/learn/nptel/106106184/83a466ee859c) (YouTube)
21. [Learning Parameters: Gradient Descent](https://invisiblenetwork.ai/learn/nptel/106106184/5fe9836f648e) (YouTube)
22. [Representation Power of Multilayer Network of Sigmoid Neurons](https://invisiblenetwork.ai/learn/nptel/106106184/8d0840ace2d8) (YouTube)
23. [Feedforward Neural Networks (a.k.a multilayered network of neurons)](https://invisiblenetwork.ai/learn/nptel/106106184/2e21cfd49c74) (YouTube)
24. [Learning Paramters of Feedforward Neural Networks (Intuition)](https://invisiblenetwork.ai/learn/nptel/106106184/451e2ecaefe7) (YouTube)
25. [Output functions and Loss functions](https://invisiblenetwork.ai/learn/nptel/106106184/5a86cff14fb1) (YouTube)
26. [Backpropagation (Intuition)](https://invisiblenetwork.ai/learn/nptel/106106184/db952ec9dba9) (YouTube)
27. [Backpropagation: Computing Gradients w.r.t. the Output Units](https://invisiblenetwork.ai/learn/nptel/106106184/dd19d5bc2563) (YouTube)
28. [Backpropagation: Computing Gradients w.r.t. Hidden Units](https://invisiblenetwork.ai/learn/nptel/106106184/22d1aa9021c4) (YouTube)
29. [Backpropagation: Computing Gradients w.r.t. Parameters](https://invisiblenetwork.ai/learn/nptel/106106184/068114ad477c) (YouTube)
30. [Backpropagation: Pseudo code](https://invisiblenetwork.ai/learn/nptel/106106184/d70949c03dbe) (YouTube)
31. [Derivative of the activation function](https://invisiblenetwork.ai/learn/nptel/106106184/5953a3a0682d) (YouTube)
32. [Information content, Entropy & cross entropy](https://invisiblenetwork.ai/learn/nptel/106106184/0dd319d44d69) (YouTube)
33. [Recap: Learning Parameters: Guess Work, Gradient Descent](https://invisiblenetwork.ai/learn/nptel/106106184/165d7da4365a) (YouTube)
34. [Contours Maps](https://invisiblenetwork.ai/learn/nptel/106106184/d141cf16485e) (YouTube)
35. [Momentum based Gradient Descent](https://invisiblenetwork.ai/learn/nptel/106106184/77ff8563899b) (YouTube)
36. [Nesterov Accelerated Gradient Descent](https://invisiblenetwork.ai/learn/nptel/106106184/8e23550d7ea7) (YouTube)
37. [Stochastic And Mini-Batch Gradient Descent](https://invisiblenetwork.ai/learn/nptel/106106184/f134e2918692) (YouTube)
38. [Tips for Adjusting Learning Rate and Momentum](https://invisiblenetwork.ai/learn/nptel/106106184/fd02091d76ff) (YouTube)
39. [Line Search](https://invisiblenetwork.ai/learn/nptel/106106184/ab96e3869d4b) (YouTube)
40. [Gradient Descent with Adaptive Learning Rate](https://invisiblenetwork.ai/learn/nptel/106106184/dcec72b0a4d0) (YouTube)
41. [Bias Correction in Adam](https://invisiblenetwork.ai/learn/nptel/106106184/4049bb089f48) (YouTube)
42. [Eigenvalues and Eigenvectors](https://invisiblenetwork.ai/learn/nptel/106106184/975c9631b61d) (YouTube)
43. [Linear Algebra : Basic Definitions](https://invisiblenetwork.ai/learn/nptel/106106184/e1fe7e34c040) (YouTube)
44. [Eigenvalue Decompositon](https://invisiblenetwork.ai/learn/nptel/106106184/b03df147de6d) (YouTube)
45. [Principal Component Analysis and its Interpretations](https://invisiblenetwork.ai/learn/nptel/106106184/badb97c1b3a1) (YouTube)
46. [PCA : Interpretation 2](https://invisiblenetwork.ai/learn/nptel/106106184/ca5733b65b05) (YouTube)
47. [PCA : Interpretation 3](https://invisiblenetwork.ai/learn/nptel/106106184/05ac185920a4) (YouTube)
48. [PCA : Interpretation 3 (Contd.)](https://invisiblenetwork.ai/learn/nptel/106106184/0ccf7513c265) (YouTube)
49. [PCA : Practical Example](https://invisiblenetwork.ai/learn/nptel/106106184/239b1dff1061) (YouTube)
50. [Singular Value Decomposition](https://invisiblenetwork.ai/learn/nptel/106106184/981507f79c9e) (YouTube)
51. [Introduction to Autoncoders](https://invisiblenetwork.ai/learn/nptel/106106184/b89084bdb0b8) (YouTube)
52. [Link between PCA and Autoencoders](https://invisiblenetwork.ai/learn/nptel/106106184/36f9c07b3101) (YouTube)
53. [Regularization in autoencoders (Motivation)](https://invisiblenetwork.ai/learn/nptel/106106184/700ef864ae48) (YouTube)
54. [Denoising Autoencoders](https://invisiblenetwork.ai/learn/nptel/106106184/ba38ab0ab812) (YouTube)
55. [Sparse Autoencoders](https://invisiblenetwork.ai/learn/nptel/106106184/052d1d1a889b) (YouTube)
56. [Contractive Autoencoders](https://invisiblenetwork.ai/learn/nptel/106106184/dfe6eb067abe) (YouTube)
57. [Bias and Variance](https://invisiblenetwork.ai/learn/nptel/106106184/00996dc91f50) (YouTube)
58. [Train error vs Test error](https://invisiblenetwork.ai/learn/nptel/106106184/e56104a6146b) (YouTube)
59. [Train error vs Test error (Recap)](https://invisiblenetwork.ai/learn/nptel/106106184/782acb223081) (YouTube)
60. [True error and Model complexity](https://invisiblenetwork.ai/learn/nptel/106106184/50fe0329c089) (YouTube)
61. [L2 regularization](https://invisiblenetwork.ai/learn/nptel/106106184/897f132bec2c) (YouTube)
62. [Dataset augmentation](https://invisiblenetwork.ai/learn/nptel/106106184/a728ce19bcbe) (YouTube)
63. [Parameter sharing and tying](https://invisiblenetwork.ai/learn/nptel/106106184/5781647ee17c) (YouTube)
64. [Adding Noise to the inputs](https://invisiblenetwork.ai/learn/nptel/106106184/9d65525765e6) (YouTube)
65. [Adding Noise to the outputs](https://invisiblenetwork.ai/learn/nptel/106106184/a5a4b27ceba0) (YouTube)
66. [Early stopping](https://invisiblenetwork.ai/learn/nptel/106106184/00c37326e007) (YouTube)
67. [Ensemble Methods](https://invisiblenetwork.ai/learn/nptel/106106184/09eb9b413dad) (YouTube)
68. [Dropout](https://invisiblenetwork.ai/learn/nptel/106106184/9f796814e5a5) (YouTube)
69. [A quick recap of training deep neural networks](https://invisiblenetwork.ai/learn/nptel/106106184/320f7e8551c7) (YouTube)
70. [Unsupervised pre-training](https://invisiblenetwork.ai/learn/nptel/106106184/bd4b6fd911ff) (YouTube)
71. [Better activation functions](https://invisiblenetwork.ai/learn/nptel/106106184/e14a900d3405) (YouTube)
72. [Better initialization strategies](https://invisiblenetwork.ai/learn/nptel/106106184/d04a4c635ea5) (YouTube)
73. [Batch Normalization](https://invisiblenetwork.ai/learn/nptel/106106184/a228fc184de0) (YouTube)
74. [One-hot representations of words](https://invisiblenetwork.ai/learn/nptel/106106184/a01ee0c89086) (YouTube)
75. [Distributed Representations of words](https://invisiblenetwork.ai/learn/nptel/106106184/e9d1083a5ba9) (YouTube)
76. [SVD for learning word representations](https://invisiblenetwork.ai/learn/nptel/106106184/b9088b16a7f7) (YouTube)
77. [SVD for learning word representations (Contd.)](https://invisiblenetwork.ai/learn/nptel/106106184/aa74c2f4f046) (YouTube)
78. [Continuous bag of words model](https://invisiblenetwork.ai/learn/nptel/106106184/ee97fb5a0841) (YouTube)
79. [Skip-gram model](https://invisiblenetwork.ai/learn/nptel/106106184/bc4e378ef1aa) (YouTube)
80. [Skip-gram model (Contd.)](https://invisiblenetwork.ai/learn/nptel/106106184/8d3b2e7b8122) (YouTube)
81. [Contrastive estimation](https://invisiblenetwork.ai/learn/nptel/106106184/a41c5d684c47) (YouTube)
82. [Hierarchical softmax](https://invisiblenetwork.ai/learn/nptel/106106184/a9f5622eae91) (YouTube)
83. [GloVe representations](https://invisiblenetwork.ai/learn/nptel/106106184/9308ce58acf7) (YouTube)
84. [Evaluating word representations](https://invisiblenetwork.ai/learn/nptel/106106184/dc4fe25d6c6d) (YouTube)
85. [Relation between SVD and Word2Vec](https://invisiblenetwork.ai/learn/nptel/106106184/86545acde8a9) (YouTube)
86. [The convolution operation](https://invisiblenetwork.ai/learn/nptel/106106184/1e48a0887d8d) (YouTube)
87. [Relation between input size, output size and filter size](https://invisiblenetwork.ai/learn/nptel/106106184/0257a39f8b61) (YouTube)
88. [Convolutional Neural Networks](https://invisiblenetwork.ai/learn/nptel/106106184/1c24d9bb883d) (YouTube)
89. [Convolutional Neural Networks (Contd.)](https://invisiblenetwork.ai/learn/nptel/106106184/8268588d42c2) (YouTube)
90. [CNNs (success stories on ImageNet)](https://invisiblenetwork.ai/learn/nptel/106106184/f7d0fcfb4705) (YouTube)
91. [CNNs (success stories on ImageNet) (Contd.)](https://invisiblenetwork.ai/learn/nptel/106106184/6345d05da379) (YouTube)
92. [Image Classification continued (GoogLeNet and ResNet)](https://invisiblenetwork.ai/learn/nptel/106106184/2e7a79dd2f6a) (YouTube)
93. [Visualizing patches which maximally activate a neuron](https://invisiblenetwork.ai/learn/nptel/106106184/bfd41e11a33e) (YouTube)
94. [Visualizing filters of a CNN](https://invisiblenetwork.ai/learn/nptel/106106184/e69d93b8dd28) (YouTube)
95. [Occlusion experiments](https://invisiblenetwork.ai/learn/nptel/106106184/2cdf5229748a) (YouTube)
96. [Finding influence of input pixels using backpropagation](https://invisiblenetwork.ai/learn/nptel/106106184/a357948e38c5) (YouTube)
97. [Guided Backpropagation](https://invisiblenetwork.ai/learn/nptel/106106184/d47a9c3b3967) (YouTube)
98. [Optimization over images](https://invisiblenetwork.ai/learn/nptel/106106184/fc90a0cd4202) (YouTube)
99. [Create images from embeddings](https://invisiblenetwork.ai/learn/nptel/106106184/92cae748de37) (YouTube)
100. [Deep Dream](https://invisiblenetwork.ai/learn/nptel/106106184/6a08c8b31d23) (YouTube)
101. [Deep Art](https://invisiblenetwork.ai/learn/nptel/106106184/08a86f0015be) (YouTube)
102. [Fooling Deep Convolutional Neural Networks](https://invisiblenetwork.ai/learn/nptel/106106184/081b17a883c6) (YouTube)
103. [Sequence Learning Problems](https://invisiblenetwork.ai/learn/nptel/106106184/f4b9457a8b0b) (YouTube)
104. [Recurrent Neural Networks](https://invisiblenetwork.ai/learn/nptel/106106184/43d51ccd0628) (YouTube)
105. [Backpropagation through time](https://invisiblenetwork.ai/learn/nptel/106106184/4b17ad9ba83c) (YouTube)
106. [The problem of Exploding and Vanishing Gradients](https://invisiblenetwork.ai/learn/nptel/106106184/7f75f2afc094) (YouTube)
107. [Some Gory Details](https://invisiblenetwork.ai/learn/nptel/106106184/e788c03f9ab1) (YouTube)
108. [Selective Read, Selective Write, Selective Forget - The Whiteboard Analogy](https://invisiblenetwork.ai/learn/nptel/106106184/ddf6791aab39) (YouTube)
109. [Long Short Term Memory(LSTM) and Gated Recurrent Units(GRUs)](https://invisiblenetwork.ai/learn/nptel/106106184/6a70d91a0ef4) (YouTube)
110. [How LSTMs avoid the problem of vanishing gradients](https://invisiblenetwork.ai/learn/nptel/106106184/da77afaf15c9) (YouTube)
111. [How LSTMs avoid the problem of vanishing gradients (Contd.)](https://invisiblenetwork.ai/learn/nptel/106106184/f06690414aa4) (YouTube)
112. [Introduction to Encoder Decoder Models](https://invisiblenetwork.ai/learn/nptel/106106184/8adaa881841c) (YouTube)
113. [Applications of Encoder Decoder models](https://invisiblenetwork.ai/learn/nptel/106106184/6cfa3ed869a8) (YouTube)
114. [Attention Mechanism](https://invisiblenetwork.ai/learn/nptel/106106184/5799675c6e41) (YouTube)
115. [Attention Mechanism (Contd.)](https://invisiblenetwork.ai/learn/nptel/106106184/9b29dde7e61e) (YouTube)
116. [Attention over images](https://invisiblenetwork.ai/learn/nptel/106106184/130207e7fe40) (YouTube)
117. [Hierarchical Attention](https://invisiblenetwork.ai/learn/nptel/106106184/902fc5620702) (YouTube)