Course10610300146 lectures
Generative AI for Computer Vision
Prof. Arijit Sur · IIT Guwahati
About
This course explores how Generative AI is applied to modern computer vision tasks. Unlike existing NPTEL courses, it specifically emphasized on vision-based generative AI models. It begins with mathematical foundations and classical vision techniques, followed by deep learning architectures. The course then introduces generative learning paradigms including GANs, VAEs, diffusion models, and transformers with a discussion regarding evaluation metrics and training challenges like mode collapse, diffusion noise scheduling, etc. Moreover, it includes LLM models for vision applications like GPT-4V, LLaMA, PaLM-E, Flamingo, etc. This course is primarily focusing on deep generative learning for computer vision tasks like Image Captioning, VQA, Scene Understanding etc. It further discusses multimodal generative models and agentic AI systems for automatic image synthesis and reasoning.
Intended audience: Final/Pre-final year B.Tech/BE, M.Tech/ME, MS, PhD students, Industry professionals, and Faculty members.
Prerequisites: Basics of Machine Learning and Computer Vision. Neural Networks for Vision and NLP.
Lectures
Week 1: Foundations
- Introduction
- Generative Modelling
- Generative Vision Models
- Mathematical Preliminaries - I (Linear Algebra)
- Mathematical Preliminaries - II (Basic Probability - I)
- Mathematical Preliminaries - II (Basic Probability - II)
Week 2: Deep Learning Basics
- Mathematical Preliminaries - I (Linear Algebra)
- Neural Network Fundamentals
- Mathematical Preliminaries - II (Basic Probability - I)
- Neural Network Fundamentals (Multi-Layer Perceptron)
- Mathematical Preliminaries - II (Basic Probability - II)
- Neural Network Fundamentals (Backpropagation)
- Neural Network Fundamentals (Overfitting)
Week 3: Deep Sequential Learning
- Convolutional Neural Network - I
- Convolutional Neural Network - II
- Convolutional Neural Network - III
Week 4: Deep Sequential Learning - II
Week 5: Introduction to Generative AI
- Training of RNN
- LSTM
- Sequence Modelling
- Generative AI for Vision Tasks - I
- Generative AI for Vision Tasks - II
Week 6: Auto-regressive models, Transformers
Week 7: Auto-regressive models, Transformers
Week 8: Vision Transformers, Variational Auto - Encoder
Week 9 : Diffusion Models
- Diffusion Models - I
- Diffusion Models - II
- Generative Adversarial Network - I
- Generative Adversarial Network - II
Week 10: Large Language Models for Vision
Week 11: Multimodal Generative Modelling
- Multimodal Generative Models - I
- Multimodal Generative Models - II
- Multimodal Generative Models - III
Week 12: Agentic AI and Research Directions
Syllabus and materials
At NPTEL, as the course publishes them.
Credits and licence
- Institution
- IIT Guwahati
- Department
- Computer Science and Engineering
- Teacher
- Prof. Arijit Sur
- Level
- Undergraduate
- Language
- English
- Subjects
- Computer science, Engineering
- Source
- NPTEL
- Licence
- CC BY-NC-SA, as NPTEL states it. Non-commercial use only. NPTEL's terms
- Attribution
- NPTEL states: "Distributed under Creative Commons Attribution-ShareAlike / CC BY - NC - SA", beside a BY-NC-SA badge.
- Who can watch
- NPTEL gives no rating. Invisible Network shows it to Tweens profiles and older, judged from what the source says about it. How courses are placed