# Introduction to Large Language Models (LLMs) - Institution: IIT Delhi IIT Bombay - Department: Computer Science and Engineering - Taught by: Prof. Tanmoy Chakraborty, Prof. Soumen Chakraborti - Level: Undergraduate - Language: English - Subjects: Computer science, Engineering - Source: [NPTEL](https://nptel.ac.in/courses/106102576) - Licence: [CC BY-NC-SA](https://nptel.ac.in/faq) (Non-commercial use only.) - Syllabus: [at the source](https://nptel.ac.in/courses/106102576) - Who can watch: Tweens profiles and older (sources give no ratings; placed by Invisible Network from the source's own words) This course introduces the fundamental concepts underlying Large Language Models (LLMs). It starts with an introduction to the various problems in NLP, and discusses how to approach the problem of language modeling using deep learning. It describes the architectural intricacies of Transformers and the pre-training objectives of the different Transformer-based models. It also discusses the recent advances in LLM research, including LLM alignment, prompting, parameter-efficient adaptation, hallucination, bias and ethical considerations. This course prepares a student to comprehend, critique and approach various research problems on LLMs. Intended audience: UG and PG students in CSE, EE, ECE, IT, Maths, etc. Prerequisites: Mandatory: Machine Learning, Python Programming Optional: Deep LearningIntroduction To Machine Learning: https://nptel.ac.in/courses/106105152Python for Data Science: https://nptel.ac.in/courses/106106212 ## Lectures 1. [Lec 01 : Introduction and Recent Advances](https://invisiblenetwork.ai/learn/nptel/106102576/d5e72a15eaf8) (YouTube) 2. [Lec 02 : Introduction to Natural Language Processing](https://invisiblenetwork.ai/learn/nptel/106102576/e062406d38a1) (YouTube) 3. [Lec 03 : Introduction to Statistical Language Models](https://invisiblenetwork.ai/learn/nptel/106102576/a5198ad45df5) (YouTube) 4. [Lec 04 : Statistical LM: Advanced Smoothing and Evaluation](https://invisiblenetwork.ai/learn/nptel/106102576/427149e94aca) (YouTube) 5. [Lec 05 : Introduction to Deep Learning](https://invisiblenetwork.ai/learn/nptel/106102576/bbe0fd178a34) (YouTube) 6. [Lec 06 : Introduction to PyTorch](https://invisiblenetwork.ai/learn/nptel/106102576/370efc62ed88) (YouTube) 7. [Lec 07 : Word Representation: Word2Vec & fastText](https://invisiblenetwork.ai/learn/nptel/106102576/67bf4a5ec0cf) (YouTube) 8. [Lec 08 : Word Representation: GloVe](https://invisiblenetwork.ai/learn/nptel/106102576/e14b5d6038c7) (YouTube) 9. [Lec 09 : Tokenization Strategies](https://invisiblenetwork.ai/learn/nptel/106102576/1e7b80b93da9) (YouTube) 10. [Lec 10 : Neural Language Models: CNN & RNN](https://invisiblenetwork.ai/learn/nptel/106102576/8bbb434cfee4) (YouTube) 11. [Lec 11 : Neural Language Models: LSTM & GRU](https://invisiblenetwork.ai/learn/nptel/106102576/901527f066ba) (YouTube) 12. [Lec 12 : Sequence-to-Sequence Models](https://invisiblenetwork.ai/learn/nptel/106102576/b46b06afbaa2) (YouTube) 13. [Lec 13 : Decoding Strategies](https://invisiblenetwork.ai/learn/nptel/106102576/7d71e6e15387) (YouTube) 14. [Lec 14 : Attention in Sequence-to-Sequence Models](https://invisiblenetwork.ai/learn/nptel/106102576/59ccf73b3762) (YouTube) 15. [Lec 15 : Introduction to Transformer: Self & Multi-Head Attention](https://invisiblenetwork.ai/learn/nptel/106102576/a6efe4873b47) (YouTube) 16. [Lec 16 : Introduction to Transformer: Positional Encoding and Layer Normalization](https://invisiblenetwork.ai/learn/nptel/106102576/4a02ecc370f7) (YouTube) 17. [Lec 17 : Implementation of Transformer using PyTorch](https://invisiblenetwork.ai/learn/nptel/106102576/036dba1bebd5) (YouTube) 18. [Lec 18 : Pre-Training Strategies: ELMo, BERT](https://invisiblenetwork.ai/learn/nptel/106102576/1f8a05856e28) (YouTube) 19. [Lec 19 : Pre-Training Strategies: Encoder-decoder and Decoder-only Models](https://invisiblenetwork.ai/learn/nptel/106102576/ed72b2b0420d) (YouTube) 20. [Lec 20 : Introduction to HuggingFace](https://invisiblenetwork.ai/learn/nptel/106102576/3f6e1341d114) (YouTube) 21. [Lec 21 : Instruction Tuning](https://invisiblenetwork.ai/learn/nptel/106102576/018a143f685a) (YouTube) 22. [Lec 22 : Prompt-based Learning](https://invisiblenetwork.ai/learn/nptel/106102576/5622022b8246) (YouTube) 23. [Lec 23 : Advanced Prompting and Prompt Sensitivity](https://invisiblenetwork.ai/learn/nptel/106102576/41b67c03e361) (YouTube) 24. [Lec 24 : Alignment of Language Models-I](https://invisiblenetwork.ai/learn/nptel/106102576/17604d875474) (YouTube) 25. [Lec 25 : Alignment of Language Models-II](https://invisiblenetwork.ai/learn/nptel/106102576/ec74d3495d63) (YouTube) 26. [Lec 26 : Knowledge and Retrieval: Knowledge Graph](https://invisiblenetwork.ai/learn/nptel/106102576/014ee16ac78d) (YouTube) 27. [Lec 27 : Knowledge and Retrieval: Knowledge Graph Completion and Evaluation](https://invisiblenetwork.ai/learn/nptel/106102576/bdcfe0c95e64) (YouTube) 28. [Lec 28 : Knowledge and Retrieval: Translation and Rotation Models](https://invisiblenetwork.ai/learn/nptel/106102576/857739c0ae09) (YouTube) 29. [Lec 29 : Parameter Efficient Fine-Tuning (PEFT)](https://invisiblenetwork.ai/learn/nptel/106102576/50772a451efa) (YouTube) 30. [Lec 30 : Quantization, Pruning & Distillation](https://invisiblenetwork.ai/learn/nptel/106102576/56a267091e36) (YouTube) 31. [Lec 31 : An Alternate Formulation of Transformers: Residual Stream Perspective](https://invisiblenetwork.ai/learn/nptel/106102576/6c1e52d75017) (YouTube) 32. [Lec 32 : Interpretability Techniques](https://invisiblenetwork.ai/learn/nptel/106102576/80e4adf5d749) (YouTube) 33. [Lec 33 : Knowledge and Retrieval: Multiplicative models](https://invisiblenetwork.ai/learn/nptel/106102576/d5eeec701b8c) (YouTube) 34. [Lec 34 : Knowledge and Retrieval: Modeling Hierarchies](https://invisiblenetwork.ai/learn/nptel/106102576/ffdae71bab07) (YouTube) 35. [Lec 35 : Knowledge and Retrieval: Temporal Knowledge Graphs](https://invisiblenetwork.ai/learn/nptel/106102576/1b06f4183855) (YouTube) 36. [Lec 36 : Responsible LLMs](https://invisiblenetwork.ai/learn/nptel/106102576/3defa4a95fc9) (YouTube) 37. [Lec 37 : Conclusion: Expert Panel Discussion](https://invisiblenetwork.ai/learn/nptel/106102576/2c9176ff6f7f) (YouTube)