# Deep Learning for Natural Language Processing

- Institution: IIT Kharagpur
- Department: Electrical Engineering
- Teacher: Prof. Pawan Goyal
- Level: Undergraduate
- Language: English
- Subjects: Engineering
- Source: [NPTEL](https://nptel.ac.in/courses/108105572)
- Licence: [CC BY-NC-SA](https://nptel.ac.in/faq) (Non-commercial use only.)
- Syllabus: [at the source](https://nptel.ac.in/courses/108105572)
- Who can watch: Tweens profiles and older (sources give no ratings; placed by Invisible Network from the source's own words)

Natural Language Processing finds its applications nearly everywhere -- be it Machine Translation,Question answering, Text Summarization, Dialogs, etc. In the last decade, Deep Learning basedmethods have given very good performance across a variety of NLP tasks, and have become a default choice for NLP problems. This course aims to give a thorough understanding of various deep learning architectures along with their specific use-cases in NLP.The course will also introduce the fundamental ideas behind training as well as fine-tuning/prompting the Large Language Models, which include in-context-learning, Parameter-efficient-fine-tuning,Reinforcement Learning through Human Feedback (RLHF). The course will also offer hands-on tutorials to help students master this subject.

Intended audience: Advanced UG and PG students

Prerequisites: Participants should have done a course on MachineLearning, and should know basics of PythonProgramming

## Lectures

1. [Lecture 01 : Introduction to the Course](https://invisiblenetwork.ai/learn/nptel/108105572/379e772dc7ca) (YouTube)
2. [Lecture 02 : Text Processing Basics,Tokenization](https://invisiblenetwork.ai/learn/nptel/108105572/3e7e4311ef73) (YouTube)
3. [Lecture 03 : N-gram Language Models: Part 1](https://invisiblenetwork.ai/learn/nptel/108105572/c1e2fb1b117d) (YouTube)
4. [Lecture 04 : N-gram Language Models: Part 2](https://invisiblenetwork.ai/learn/nptel/108105572/4824f7007947) (YouTube)
5. [Lecture 05 : NLP Tasks and Paradigms](https://invisiblenetwork.ai/learn/nptel/108105572/40a8bf60b8f1) (YouTube)
6. [Tutorial - 1](https://invisiblenetwork.ai/learn/nptel/108105572/65d49cf36359) (YouTube)
7. [Lecture 06 : Supervised Learning](https://invisiblenetwork.ai/learn/nptel/108105572/2dfeb26f6168) (YouTube)
8. [Lecture 07 : Shallow Neural Networks](https://invisiblenetwork.ai/learn/nptel/108105572/f94f43019df7) (YouTube)
9. [Lecture 08 : Deep Neural Networks](https://invisiblenetwork.ai/learn/nptel/108105572/8be5ac1f360c) (YouTube)
10. [Lecture 09 : Backpropagation](https://invisiblenetwork.ai/learn/nptel/108105572/19d5165690fb) (YouTube)
11. [Lecture 10 : Gradient Descent and Initialization](https://invisiblenetwork.ai/learn/nptel/108105572/21b35911e65c) (YouTube)
12. [Tutorial - 2](https://invisiblenetwork.ai/learn/nptel/108105572/c757aa792e4c) (YouTube)
13. [Lecture 11 : Word Representation](https://invisiblenetwork.ai/learn/nptel/108105572/a793b7421376) (YouTube)
14. [Lecture 12 : Learning Word Representation - Part I](https://invisiblenetwork.ai/learn/nptel/108105572/143eee46769e) (YouTube)
15. [Lecture 13 : Learning Word Representation - Part II](https://invisiblenetwork.ai/learn/nptel/108105572/b4f76136cff4) (YouTube)
16. [Lecture 14 : Word Vectors: Other Extensions](https://invisiblenetwork.ai/learn/nptel/108105572/3cfff8db25f4) (YouTube)
17. [Lecture 15 : Cross-Lingual Representations](https://invisiblenetwork.ai/learn/nptel/108105572/cc009260ca51) (YouTube)
18. [Tutorial-3](https://invisiblenetwork.ai/learn/nptel/108105572/a9ae2f0c4ccb) (YouTube)
19. [Lecture 16 : RNN Language Models](https://invisiblenetwork.ai/learn/nptel/108105572/fba0062dc294) (YouTube)
20. [Lecture 17 : RNN Applications : Text Generation, Sequence Labeling, Text Classification](https://invisiblenetwork.ai/learn/nptel/108105572/1e99d4493abb) (YouTube)
21. [Lecture 18 : RNN for Sequence to Sequence](https://invisiblenetwork.ai/learn/nptel/108105572/f70b1ccf233a) (YouTube)
22. [Lecture 19 : Decoding Strategies](https://invisiblenetwork.ai/learn/nptel/108105572/cbe2bb202cba) (YouTube)
23. [Lecture 20 : Better RNN Units : GRU, LSTM](https://invisiblenetwork.ai/learn/nptel/108105572/fa0b50d591cf) (YouTube)
24. [Tutorial - 4](https://invisiblenetwork.ai/learn/nptel/108105572/6d02970749d5) (YouTube)
25. [Lecture 21 : Introduction to Transformers](https://invisiblenetwork.ai/learn/nptel/108105572/cc9c5551dd72) (YouTube)
26. [Lecture 22 :Transformers : Part 2](https://invisiblenetwork.ai/learn/nptel/108105572/5c6d76d8e774) (YouTube)
27. [Lecture 23 : Transformers : Part 3](https://invisiblenetwork.ai/learn/nptel/108105572/6bfe26350994) (YouTube)
28. [Lecture 24 : Transformers : Part 4](https://invisiblenetwork.ai/learn/nptel/108105572/abbc2ec8751c) (YouTube)
29. [Lecture 25 : Efficient Transformers](https://invisiblenetwork.ai/learn/nptel/108105572/23a8e81c04fb) (YouTube)
30. [Tutorial-5](https://invisiblenetwork.ai/learn/nptel/108105572/1de657ca23ec) (YouTube)
31. [Lecture 26 : Pretraining](https://invisiblenetwork.ai/learn/nptel/108105572/affa46ebe5aa) (YouTube)
32. [Lecture 27 : Pretraining Transformer Encoder](https://invisiblenetwork.ai/learn/nptel/108105572/f2e06f3b871b) (YouTube)
33. [Lecture 28 : Pretraining Transformer Encoder, Encoder-Decoder](https://invisiblenetwork.ai/learn/nptel/108105572/4cefc957faf5) (YouTube)
34. [Lecture 29 : Pretraining Transformer Decoder](https://invisiblenetwork.ai/learn/nptel/108105572/a25ed979ac0d) (YouTube)
35. [Lecture 30 : More on Pretraining](https://invisiblenetwork.ai/learn/nptel/108105572/69e6a186527c) (YouTube)
36. [Tutorial-6](https://invisiblenetwork.ai/learn/nptel/108105572/d20dbf843f69) (YouTube)
37. [Lecture 31 : Applications : Question Answering - I](https://invisiblenetwork.ai/learn/nptel/108105572/2fbe7f183878) (YouTube)
38. [Lecture 32 : Applications : Question Answering - II](https://invisiblenetwork.ai/learn/nptel/108105572/9d6fcde732fd) (YouTube)
39. [Lecture 33 : Applications : Dialogue Systems - I](https://invisiblenetwork.ai/learn/nptel/108105572/63ba1281cfd7) (YouTube)
40. [Lecture 34 : Applications : Dialogue Systems - II](https://invisiblenetwork.ai/learn/nptel/108105572/50577f07ac2c) (YouTube)
41. [Lecture 35 : Applications : Text Summarization](https://invisiblenetwork.ai/learn/nptel/108105572/0f16c50e418d) (YouTube)
42. [Tutorial-7](https://invisiblenetwork.ai/learn/nptel/108105572/9cf25a6f9f6a) (YouTube)
43. [Lecture 36 : Instruction Fine-Tuning-I](https://invisiblenetwork.ai/learn/nptel/108105572/1c4aac57fb1e) (YouTube)
44. [Lecture 37 : Instruction Fine-Tuning-II](https://invisiblenetwork.ai/learn/nptel/108105572/bd141e3c8b2e) (YouTube)
45. [Lecture 38 : Reinforcement Learning from Human Feedback - I](https://invisiblenetwork.ai/learn/nptel/108105572/76e48541eeb7) (YouTube)
46. [Lecture 39 : Reinforcement Learning from Human Feedback - II](https://invisiblenetwork.ai/learn/nptel/108105572/8fdb7bd0059a) (YouTube)
47. [Lecture 40 : Aligning to User Preferences via Direct Preference Optimization](https://invisiblenetwork.ai/learn/nptel/108105572/9f4cb8fda958) (YouTube)
48. [Tutorial-8](https://invisiblenetwork.ai/learn/nptel/108105572/2328ae237e49) (YouTube)
49. [Lecture 41 : Prompting-I](https://invisiblenetwork.ai/learn/nptel/108105572/0bfd25aaeb63) (YouTube)
50. [Lecture 42 : Prompting : Why does in-context learning work?](https://invisiblenetwork.ai/learn/nptel/108105572/c05d7d873722) (YouTube)
51. [Lecture 43 : Advanced Prompting Techniques](https://invisiblenetwork.ai/learn/nptel/108105572/0a78dd118f4a) (YouTube)
52. [Lecture 44 : Tool-aided Language Models](https://invisiblenetwork.ai/learn/nptel/108105572/4f5915e16a42) (YouTube)
53. [Lecture 45 : Automatic Prompt Engineering](https://invisiblenetwork.ai/learn/nptel/108105572/3200e9e2b1b4) (YouTube)
54. [Tutorial - 9](https://invisiblenetwork.ai/learn/nptel/108105572/ab02e8dfb70c) (YouTube)
55. [Lecture 46 : Parameter-efficient fine-tuning - I](https://invisiblenetwork.ai/learn/nptel/108105572/8549c7d4d6f1) (YouTube)
56. [Lecture 47 : Parameter-efficient fine-tuning - II](https://invisiblenetwork.ai/learn/nptel/108105572/89760d77875a) (YouTube)
57. [Lecture 48 : Efficient fine-tuning for quantized LMs-I](https://invisiblenetwork.ai/learn/nptel/108105572/cc927b03cf13) (YouTube)
58. [Lecture 49 : Efficient fine-tuning for quantized LMs-II](https://invisiblenetwork.ai/learn/nptel/108105572/28514ecd9241) (YouTube)
59. [Lecture 50 : Other Parameter Efficient Methods: Pruning, Distillation](https://invisiblenetwork.ai/learn/nptel/108105572/c2fd85fd31d4) (YouTube)
60. [Tutorial-10](https://invisiblenetwork.ai/learn/nptel/108105572/89f029ef2975) (YouTube)
61. [Lecture 51 : Scaling Laws of LLMs](https://invisiblenetwork.ai/learn/nptel/108105572/972ab25fce64) (YouTube)
62. [Lecture 52 : Modern LLMs and Architecture Variations - I](https://invisiblenetwork.ai/learn/nptel/108105572/83b06a4094a5) (YouTube)
63. [Lecture 53 : Modern LLMs and Architecture Variations: Positional Embeddings](https://invisiblenetwork.ai/learn/nptel/108105572/4ed8efd2c6dd) (YouTube)
64. [Lecture 54 : Long Sequence Modeling](https://invisiblenetwork.ai/learn/nptel/108105572/f852e668c237) (YouTube)
65. [Lecture 55 : Retrieval Augmented Generation](https://invisiblenetwork.ai/learn/nptel/108105572/c2f0f35781ea) (YouTube)
66. [Tutorial-11](https://invisiblenetwork.ai/learn/nptel/108105572/021dede68663) (YouTube)
67. [Lecture 56 : Model Interpretability](https://invisiblenetwork.ai/learn/nptel/108105572/4b0519fca977) (YouTube)
68. [Lecture 57 : Model Interpretability - Multilingual](https://invisiblenetwork.ai/learn/nptel/108105572/fd2845a0af60) (YouTube)
69. [Lecture 58 : Model Interpretability - III](https://invisiblenetwork.ai/learn/nptel/108105572/922527731eb1) (YouTube)
70. [Lecture 59 : Trustworty LLMs : Taxonomy](https://invisiblenetwork.ai/learn/nptel/108105572/b066d9585369) (YouTube)
71. [Lecture 60 : Trustworty LLMs : Machine Unlearning](https://invisiblenetwork.ai/learn/nptel/108105572/c9f4d135dc64) (YouTube)
72. [Tutorial-12](https://invisiblenetwork.ai/learn/nptel/108105572/766105b47d69) (YouTube)