Learn · NPTEL

Course10610614360 lectures

Reinforcement Learning

Prof. Balaraman Ravindran · IIT Madras

Start with lecture 1 At NPTEL

About

Reinforcement learning is a paradigm that aims to model the trial-and-error learning process that is needed in many problem situations where explicit instructive signals are not available. It has roots in operations research, behavioral psychology and AI. The goal of the course is to introduce the basic mathematical foundations of reinforcement learning, as well as highlight some of the recent directions of research.

Intended audience: Any interested learner

Lectures

Week 1

  1. Introduction to RL YouTube
  2. RL Framework and Applications YouTube
  3. Introduction to Immediate RL YouTube
  4. Bandit Optimalities YouTube
  5. Value Function Based Methods YouTube

Week 2

  1. UCB 1 YouTube
  2. Concentration Bounds YouTube
  3. UCB 1 Theorem YouTube
  4. PAC Bounds YouTube
  5. Median Elimination YouTube
  6. Thompson Sampling YouTube

Week 3

  1. Policy Search YouTube
  2. REINFORCE YouTube
  3. Contextual Bandits YouTube
  4. Full RL Introduction YouTube
  5. Returns, Value Functions and MDPs YouTube

Week 4

  1. MDP Modelling YouTube
  2. Bellman Equation YouTube
  3. Bellman Optimality Equation YouTube
  4. Cauchy Sequence and Green's Equation YouTube
  5. Banach Fixed Point Theorem YouTube
  6. Convergence Proof YouTube

Week 5

  1. Lpi Convergence YouTube
  2. Value Iteration YouTube
  3. Policy Iteration YouTube
  4. Dynamic Programming YouTube
  5. Monte Carlo YouTube
  6. Control in Monte Carlo YouTube

Week 6

  1. Off Policy MC YouTube
  2. UCT YouTube
  3. TD(0) YouTube
  4. TD(0) Control YouTube
  5. Q-Learning YouTube
  6. Afterstate YouTube

Week 7

  1. Eligibility Traces YouTube
  2. Backward View of Eligibility Traces YouTube
  3. Eligibility Trace Control YouTube
  4. Thompson Sampling Recap YouTube

Week 8

  1. Function Approximation YouTube
  2. Linear Parameterization YouTube
  3. State Aggregation Methods YouTube
  4. Function Approximation and Eligibility Traces YouTube
  5. LSTD and LSTDQ YouTube
  6. LSPI and Fitted Q YouTube

Week 9

  1. DQN and Fitted Q-Iteration YouTube
  2. Policy Gradient Approach YouTube
  3. Actor Critic and REINFORCE YouTube
  4. REINFORCE (cont'd) YouTube
  5. Policy Gradient with Function Approximation YouTube

Week 10

  1. Hierarchical Reinforcement Learning YouTube
  2. Types of Optimality YouTube
  3. Semi Markov Decision Processes YouTube
  4. Options YouTube
  5. Learning with Options YouTube
  6. Hierarchical Abstract Machines YouTube

Week 11

  1. MAXQ YouTube
  2. MAXQ Value Function Decomposition YouTube
  3. Option Discovery YouTube

Week 12

  1. POMDP Introduction YouTube
  2. Solving POMDP YouTube

Syllabus and materials

At NPTEL, as the course publishes them.

Credits and licence

Institution
IIT Madras
Department
Computer Science and Engineering
Teacher
Prof. Balaraman Ravindran
Level
Undergraduate
Language
English
Subjects
Computer science, Engineering
Source
NPTEL
Licence
CC BY-NC-SA, as NPTEL states it. Non-commercial use only. NPTEL's terms
Attribution
NPTEL states: "Distributed under Creative Commons Attribution-ShareAlike / CC BY - NC - SA", beside a BY-NC-SA badge.
Who can watch
NPTEL gives no rating. Invisible Network shows it to Tweens profiles and older, judged from what the source says about it. How courses are placed