Course10810557272 lectures
Deep Learning for Natural Language Processing
Prof. Pawan Goyal · IIT Kharagpur
About
Natural Language Processing finds its applications nearly everywhere -- be it Machine Translation,Question answering, Text Summarization, Dialogs, etc. In the last decade, Deep Learning basedmethods have given very good performance across a variety of NLP tasks, and have become a default choice for NLP problems. This course aims to give a thorough understanding of various deep learning architectures along with their specific use-cases in NLP.The course will also introduce the fundamental ideas behind training as well as fine-tuning/prompting the Large Language Models, which include in-context-learning, Parameter-efficient-fine-tuning,Reinforcement Learning through Human Feedback (RLHF). The course will also offer hands-on tutorials to help students master this subject.
Intended audience: Advanced UG and PG students
Prerequisites: Participants should have done a course on MachineLearning, and should know basics of PythonProgramming
Lectures
Week 1 :
- Lecture 01 : Introduction to the Course
- Lecture 02 : Text Processing Basics,Tokenization
- Lecture 03 : N-gram Language Models: Part 1
- Lecture 04 : N-gram Language Models: Part 2
- Lecture 05 : NLP Tasks and Paradigms
- Tutorial - 1
Week 2 :
- Lecture 06 : Supervised Learning
- Lecture 07 : Shallow Neural Networks
- Lecture 08 : Deep Neural Networks
- Lecture 09 : Backpropagation
- Lecture 10 : Gradient Descent and Initialization
- Tutorial - 2
Week 3 :
- Lecture 11 : Word Representation
- Lecture 12 : Learning Word Representation - Part I
- Lecture 13 : Learning Word Representation - Part II
- Lecture 14 : Word Vectors: Other Extensions
- Lecture 15 : Cross-Lingual Representations
- Tutorial-3
Week 4 :
- Lecture 16 : RNN Language Models
- Lecture 17 : RNN Applications : Text Generation, Sequence Labeling, Text Classification
- Lecture 18 : RNN for Sequence to Sequence
- Lecture 19 : Decoding Strategies
- Lecture 20 : Better RNN Units : GRU, LSTM
- Tutorial - 4
Week 5 :
- Lecture 21 : Introduction to Transformers
- Lecture 22 :Transformers : Part 2
- Lecture 23 : Transformers : Part 3
- Lecture 24 : Transformers : Part 4
- Lecture 25 : Efficient Transformers
- Tutorial-5
Week 6 :
- Lecture 26 : Pretraining
- Lecture 27 : Pretraining Transformer Encoder
- Lecture 28 : Pretraining Transformer Encoder, Encoder-Decoder
- Lecture 29 : Pretraining Transformer Decoder
- Lecture 30 : More on Pretraining
- Tutorial-6
Week 7 :
- Lecture 31 : Applications : Question Answering - I
- Lecture 32 : Applications : Question Answering - II
- Lecture 33 : Applications : Dialogue Systems - I
- Lecture 34 : Applications : Dialogue Systems - II
- Lecture 35 : Applications : Text Summarization
- Tutorial-7
Week 8 :
- Lecture 36 : Instruction Fine-Tuning-I
- Lecture 37 : Instruction Fine-Tuning-II
- Lecture 38 : Reinforcement Learning from Human Feedback - I
- Lecture 39 : Reinforcement Learning from Human Feedback - II
- Lecture 40 : Aligning to User Preferences via Direct Preference Optimization
- Tutorial-8
Week 9 :
- Lecture 41 : Prompting-I
- Lecture 42 : Prompting : Why does in-context learning work?
- Lecture 43 : Advanced Prompting Techniques
- Lecture 44 : Tool-aided Language Models
- Lecture 45 : Automatic Prompt Engineering
- Tutorial - 9
Week 10 :
- Lecture 46 : Parameter-efficient fine-tuning - I
- Lecture 47 : Parameter-efficient fine-tuning - II
- Lecture 48 : Efficient fine-tuning for quantized LMs-I
- Lecture 49 : Efficient fine-tuning for quantized LMs-II
- Lecture 50 : Other Parameter Efficient Methods: Pruning, Distillation
- Tutorial-10
Week 11 :
- Lecture 51 : Scaling Laws of LLMs
- Lecture 52 : Modern LLMs and Architecture Variations - I
- Lecture 53 : Modern LLMs and Architecture Variations: Positional Embeddings
- Lecture 54 : Long Sequence Modeling
- Lecture 55 : Retrieval Augmented Generation
- Tutorial-11
Week 12 :
Syllabus and materials
At NPTEL, as the course publishes them.
Credits and licence
- Institution
- IIT Kharagpur
- Department
- Electrical Engineering
- Teacher
- Prof. Pawan Goyal
- Level
- Undergraduate
- Language
- English
- Subjects
- Engineering
- Source
- NPTEL
- Licence
- CC BY-NC-SA, as NPTEL states it. Non-commercial use only. NPTEL's terms
- Attribution
- NPTEL states: "Distributed under Creative Commons Attribution-ShareAlike / CC BY - NC - SA", beside a BY-NC-SA badge.
- Who can watch
- NPTEL gives no rating. Invisible Network shows it to Tweens profiles and older, judged from what the source says about it. How courses are placed
More like this
NPTEL101105083
Course
Space Flight Mechanics
99 lectures · Source's player
NPTEL101105084
Course
Aircraft Structures - I
42 lectures · Source's player
NPTEL101105085
Course
Introduction to CFD
65 lectures · Source's player
NPTEL101105089
Course
Aerodynamic Design of Axial Flow Compressors & Fans
69 lectures · Source's player
NPTEL101105090
Course
Smart Structures
62 lectures · Source's player
NPTEL101105660
Course
Industrial Aerodynamics
60 lectures · Source's player
NPTEL101105662
Course
Mechanics of Composite Materials and Structures
64 lectures · Source's player
NPTEL101105024
Course
High Speed Aero Dynamics
40 lectures · Video
NPTEL101105030
Course
Space Flight Mechanics
45 lectures · Video
NPTEL101105059
Course
Introduction to Aerodynamics
46 lectures · Video
NPTEL101105077
Course
Satellite Attitude Dynamics and Control
76 lectures · Video
NPTEL101105081
Course
Vibration And Structural Dynamics
42 lectures · Video