Learn · Deep Learning for Natural Language Processing
Lecture 26 of 72Week 5 :
Lecture 22 :Transformers : Part 2
Prof. Pawan Goyal · IIT Kharagpur
Credits and licence
- From
- Deep Learning for Natural Language Processing, IIT Kharagpur (at NPTEL)
- Licence
- CC BY-NC-SA, as NPTEL states it. Non-commercial use only.
- Attribution
- NPTEL states: "Distributed under Creative Commons Attribution-ShareAlike / CC BY - NC - SA", beside a BY-NC-SA badge.
- Who can watch
- Tweens profiles and older. How courses are placed
All the lectures
- Lecture 01 : Introduction to the Course
- Lecture 02 : Text Processing Basics,Tokenization
- Lecture 03 : N-gram Language Models: Part 1
- Lecture 04 : N-gram Language Models: Part 2
- Lecture 05 : NLP Tasks and Paradigms
- Tutorial - 1
- Lecture 06 : Supervised Learning
- Lecture 07 : Shallow Neural Networks
- Lecture 08 : Deep Neural Networks
- Lecture 09 : Backpropagation
- Lecture 10 : Gradient Descent and Initialization
- Tutorial - 2
- Lecture 11 : Word Representation
- Lecture 12 : Learning Word Representation - Part I
- Lecture 13 : Learning Word Representation - Part II
- Lecture 14 : Word Vectors: Other Extensions
- Lecture 15 : Cross-Lingual Representations
- Tutorial-3
- Lecture 16 : RNN Language Models
- Lecture 17 : RNN Applications : Text Generation, Sequence Labeling, Text Classification
- Lecture 18 : RNN for Sequence to Sequence
- Lecture 19 : Decoding Strategies
- Lecture 20 : Better RNN Units : GRU, LSTM
- Tutorial - 4
- Lecture 21 : Introduction to Transformers
- Lecture 22 :Transformers : Part 2
- Lecture 23 : Transformers : Part 3
- Lecture 24 : Transformers : Part 4
- Lecture 25 : Efficient Transformers
- Tutorial-5
- Lecture 26 : Pretraining
- Lecture 27 : Pretraining Transformer Encoder
- Lecture 28 : Pretraining Transformer Encoder, Encoder-Decoder
- Lecture 29 : Pretraining Transformer Decoder
- Lecture 30 : More on Pretraining
- Tutorial-6
- Lecture 31 : Applications : Question Answering - I
- Lecture 32 : Applications : Question Answering - II
- Lecture 33 : Applications : Dialogue Systems - I
- Lecture 34 : Applications : Dialogue Systems - II
- Lecture 35 : Applications : Text Summarization
- Tutorial-7
- Lecture 36 : Instruction Fine-Tuning-I
- Lecture 37 : Instruction Fine-Tuning-II
- Lecture 38 : Reinforcement Learning from Human Feedback - I
- Lecture 39 : Reinforcement Learning from Human Feedback - II
- Lecture 40 : Aligning to User Preferences via Direct Preference Optimization
- Tutorial-8
- Lecture 41 : Prompting-I
- Lecture 42 : Prompting : Why does in-context learning work?
- Lecture 43 : Advanced Prompting Techniques
- Lecture 44 : Tool-aided Language Models
- Lecture 45 : Automatic Prompt Engineering
- Tutorial - 9
- Lecture 46 : Parameter-efficient fine-tuning - I
- Lecture 47 : Parameter-efficient fine-tuning - II
- Lecture 48 : Efficient fine-tuning for quantized LMs-I
- Lecture 49 : Efficient fine-tuning for quantized LMs-II
- Lecture 50 : Other Parameter Efficient Methods: Pruning, Distillation
- Tutorial-10
- Lecture 51 : Scaling Laws of LLMs
- Lecture 52 : Modern LLMs and Architecture Variations - I
- Lecture 53 : Modern LLMs and Architecture Variations: Positional Embeddings
- Lecture 54 : Long Sequence Modeling
- Lecture 55 : Retrieval Augmented Generation
- Tutorial-11
- Lecture 56 : Model Interpretability
- Lecture 57 : Model Interpretability - Multilingual
- Lecture 58 : Model Interpretability - III
- Lecture 59 : Trustworty LLMs : Taxonomy
- Lecture 60 : Trustworty LLMs : Machine Unlearning
- Tutorial-12