Course10610800274 lectures
Fundamentals of Generative AI and Large Language Models: Theory and Practice
Prof. Sriram Ganapathy, Prof. Ashwini Kodipalli and Prof. Baishali Garai · IISc Bangalore PES University RV University
About
This course provides a rigorous exploration of modern generative AI, covering latent-variable models such as Autoencoders, VAEs, GANs, and Diffusion Models that form the backbone of contemporary generative systems.
Learners will develop a deep understanding of probabilistic modeling, adversarial training dynamics, denoising diffusion processes, and latent space manipulation for high-fidelity data generation.
The sequence modeling module bridges classical NLP with modern architectures, introducing RNNs, LSTMs, and culminating in the Transformer framework that powers today’s Large Language Models.
Emphasis is placed on mathematical foundations, architectural intuition, and practical implementation using widely adopted deep learning libraries.
By the end of the course, learners will be equipped to analyze, design, and implement state-of-the-art generative models and LLM-based systems across diverse application domains.
Intended audience: UG and PG students of all the AICTE affiliated institutions
Prerequisites: Students must have completed introductory courses in Programming and Machine Learning/Deep Learning.Knowledge of Python and basic mathematical concepts is necessary to follow the hands-on exercises.
Lectures
Week 1 Fundamentals of Deep Learning for Generative AI
- Introduction to Generative AI
- Activation Functions & Loss Functions in Deep Learning
- Optimizers - Part A
- Optimizers - Part B
- Convolutional Neural Network - Part A
- Convolutional Neural Network - Part B
- Introduction to Google Colab
- Practical Exercise - 1
- Practical Exercise - 2
Week 2 Autoencoders: The Representation Learning Model
- Introduction to Autoencoder
- Training objective: Reconstruction Loss (MSE, Binary Cross-Entropy)
- Types of Autoencoders: Architecture and Code-Level Understanding
- Regularization Techniques in AE: Denoising
- Regularization Techniques in AE: Sparse
- Regularization Techniques in AE: Contractive
- Numerical Example, Limitations of AE
- Practical Exercise - 1
- Practical Exercise - 2
Week 3 Variational Autoencoders (VAEs): Foundations
- Intuition behind KL Divergence - Part A
- Intuition behind KL Divergence - Part B
- Introduction to VAE and the working of the Encoder
- Working of Probabilistic Decoder, ELBO, Loss Function of VAE
- Reparameterization Trick
- Numerical Example
Week 4 Advanced Variational Autoencoders (VAEs): Practical Applications
- Practical Exercise - 1
- Entanglement, Disentanglement Latent Space, and Beta-VAE
- Conditional VAE
- Latent Space Interpolation
- Practical Exercise - 2
- Practical Exercise - 3
Week 5 Generative Adversarial Networks (GANs): Foundations
- Motivation for GANs
- GAN Architecture
- GAN Objective and Loss Functions
- GAN Convergence and Nash equilibrium
- Introduction to DCGAN
- Practical Exercise - 1
- Practical Exercise - 2
Week 6 Variants of GAN
Week 7 Diffusion Model- Part I
- Introduction to Diffusion Model
- Mathematics of Diffusion Models
- "Denoising Diffusion Probabilistic Model- Forward Process"
- "Denoising Diffusion Probabilistic Model- Reverse Process"
- Hand on on forward Diffusion process
Week 8 Diffusion Model Part- 2
- U-Net for Denoising
- Classifier Guided Diffusion Model
- Classifier free Diffusion Models
- Stable Diffusion Model
- Hands-on on reverse diffusion process
Week 9 Sequential Models to Transformers
- Foundations of Natural Language Processing
- Sequential Modeling with RNNs and LSTM
- Evolution From LSTMs to Transformers
- Transformer Architecture: Encoder
- Practical Exercise - 1
Week 10 Large Language Models
- Transformer Architecture: Decoder
- Bidirectional Encoder Representations from Transformer (BERT)
- Generative Pre-trained Transformer (GPT) Decoder-only Large Language Model
- Prompt Engineering Basics
- Hands-on on LLM
Week 11 Topics in Large Language Models – Part I
- LLMs – Recap
- In-context Learning
- Low-Rank Adaptation (LoRA)
- RAG Principle
- RAG Recent Advances
- Tutorial on In-context Learning, RAG and LoRA
Week 12 LLM Capabilities, Multimodal AI, Ethics
Syllabus and materials
At NPTEL, as the course publishes them.
Credits and licence
- Institution
- IISc Bangalore PES University RV University
- Department
- Computer Science and Engineering
- Taught by
- Prof. Sriram Ganapathy, Prof. Ashwini Kodipalli, Prof. Baishali Garai
- Level
- Undergraduate
- Language
- English
- Subjects
- Computer science, Engineering
- Source
- NPTEL
- Licence
- CC BY-NC-SA, as NPTEL states it. Non-commercial use only. NPTEL's terms
- Attribution
- NPTEL states: "Distributed under Creative Commons Attribution-ShareAlike / CC BY - NC - SA", beside a BY-NC-SA badge. Coordinated by IISc Bangalore.
- Who can watch
- NPTEL gives no rating. Invisible Network shows it to Tweens profiles and older, judged from what the source says about it. How courses are placed
More like this
Course
Matrix Methods in Data Analysis, Signal Processing, and Machine Learning
34 lectures · Video
Course
Computational Science and Engineering I
50 lectures · Video
Course
Engineering Dynamics
223 lectures · Video
Course
CS50’s Introduction to Computer Science
12 lectures · Video
Course
Frontiers of Biomedical Engineering
25 lectures · Audio
Course
CS50's Introduction to Programming with Python
10 lectures · Video