Learn · Stochastic Approximation: Theory and Applications
Lecture 18 of 46Week 4 Foundations of ODE Analysis: Existence and Stability
Asymptotic Behaviour of Solutions to ODEs
Prof. Gugan Chandrashekhar Mallika Thoppe · IISc Bangalore
Credits and licence
- From
- Stochastic Approximation: Theory and Applications, IISc Bangalore (at NPTEL)
- Licence
- CC BY-NC-SA, as NPTEL states it. Non-commercial use only.
- Attribution
- NPTEL states: "Distributed under Creative Commons Attribution-ShareAlike / CC BY - NC - SA", beside a BY-NC-SA badge.
- Who can watch
- Tweens profiles and older. How courses are placed
All the lectures
- Overview of Stochastic Approximation
- Estimating the Mean of a Random Variable
- Estimating change rates of webpages
- An Introduction to Reinforcement Learning.
- Policy Evaluation in Reinforcement Learning
- Probability Spaces- A Measure Theoretic Perspective
- Random Variables as Measurable Maps
- Expectation of Random Variable as Lebesgue Integration
- Conditional Expectation: A Formal Introduction
- Properties of Conditional Expectation.
- Martingales: Definition and Examples
- Doob’s Uncrossing Lemma
- Doob’s Forward Convergence Theorem for Supermartingales
- $L^2$ Martingales
- Revisiting the Martingale Convergence Theorem via Stopped Processes
- Existence and Uniqueness of Solutions to Ordinary Differential Equations
- Continuity of ODE Solutions with Respect to Initial Conditions
- Asymptotic Behaviour of Solutions to ODEs
- Stability of ODEs and Lyapunov Methods
- Foundations of Stochastic Approximation – Assumptions and Key Definitions
- Almost Sure Convergence of Stochastic Approximation Iterates
- Concluding the Convergence Proof: Internal Chain Transitivity, Connectedness, and Invariance
- How Stochastic Approximation Iterates Track an ODE: The Key Lemma
- Proof of the Key Lemma - Part I
- Proof of the Key Lemma - Part II
- Extensions, Variants, and Applications of Stochastic Approximation
- Convergence Rate for Linear Stochastic Approximation - Part 1
- Convergence Rate for Linear Stochastic Approximation - Part 2
- Lower Bounds and the Minimax Risk in Estimation
- Stability Requirements in Stochastic Approximation
- Almost Sure Boundedness of Iterates: Theorem and Example
- Distributed Estimation of the Mean of a Random Vector
- Robust Distributed Learning under Adversaries
- Motivation for Stochastic Recursive Inclusion in Distributed Mean Estimation
- Well-Posedness of Differential Inclusions
- Convergence of Distributed Robust Mean Estimation Iterates through the Lens of Differential Inclusions
- Almost Sure Convergence via Robbins–Siegmund Theorem – Part 1
- Almost Sure Convergence via Robbins–Siegmund Theorem – Part 2
- Introduction to Reinforcement Learning and Value Function Approximation
- Temporal Difference Algorithm Through the Lens of Stochastic Approximation
- How Good Is the TD Solution? Fixed Point Analysis in Linear Approximation
- Almost Sure Convergence Analysis of Temporal Difference Learning via the ODE Method
- Best Policy Algorithm for Q-Value Functions: A Stochastic Approximation Formulation
- Asymptotic Analysis of Q-Learning Algorithm
- Asymptotic Behaviour of the Q-Learning Limit ODE — A Switching Systems Perspective
- Concluding the Asymptotic Analysis of Q-Learning