Learn · Stochastic Approximation: Theory and Applications
Lecture 36 of 46Week 10 Almost Sure Convergence in Distributed Robust Mean Estimation
Convergence of Distributed Robust Mean Estimation Iterates through the Lens of Differential Inclusions
Prof. Gugan Chandrashekhar Mallika Thoppe · IISc Bangalore
Credits and licence
- From
- Stochastic Approximation: Theory and Applications, IISc Bangalore (at NPTEL)
- Licence
- CC BY-NC-SA, as NPTEL states it. Non-commercial use only.
- Attribution
- NPTEL states: "Distributed under Creative Commons Attribution-ShareAlike / CC BY - NC - SA", beside a BY-NC-SA badge.
- Who can watch
- Tweens profiles and older. How courses are placed
All the lectures
- Overview of Stochastic Approximation
- Estimating the Mean of a Random Variable
- Estimating change rates of webpages
- An Introduction to Reinforcement Learning.
- Policy Evaluation in Reinforcement Learning
- Probability Spaces- A Measure Theoretic Perspective
- Random Variables as Measurable Maps
- Expectation of Random Variable as Lebesgue Integration
- Conditional Expectation: A Formal Introduction
- Properties of Conditional Expectation.
- Martingales: Definition and Examples
- Doob’s Uncrossing Lemma
- Doob’s Forward Convergence Theorem for Supermartingales
- $L^2$ Martingales
- Revisiting the Martingale Convergence Theorem via Stopped Processes
- Existence and Uniqueness of Solutions to Ordinary Differential Equations
- Continuity of ODE Solutions with Respect to Initial Conditions
- Asymptotic Behaviour of Solutions to ODEs
- Stability of ODEs and Lyapunov Methods
- Foundations of Stochastic Approximation – Assumptions and Key Definitions
- Almost Sure Convergence of Stochastic Approximation Iterates
- Concluding the Convergence Proof: Internal Chain Transitivity, Connectedness, and Invariance
- How Stochastic Approximation Iterates Track an ODE: The Key Lemma
- Proof of the Key Lemma - Part I
- Proof of the Key Lemma - Part II
- Extensions, Variants, and Applications of Stochastic Approximation
- Convergence Rate for Linear Stochastic Approximation - Part 1
- Convergence Rate for Linear Stochastic Approximation - Part 2
- Lower Bounds and the Minimax Risk in Estimation
- Stability Requirements in Stochastic Approximation
- Almost Sure Boundedness of Iterates: Theorem and Example
- Distributed Estimation of the Mean of a Random Vector
- Robust Distributed Learning under Adversaries
- Motivation for Stochastic Recursive Inclusion in Distributed Mean Estimation
- Well-Posedness of Differential Inclusions
- Convergence of Distributed Robust Mean Estimation Iterates through the Lens of Differential Inclusions
- Almost Sure Convergence via Robbins–Siegmund Theorem – Part 1
- Almost Sure Convergence via Robbins–Siegmund Theorem – Part 2
- Introduction to Reinforcement Learning and Value Function Approximation
- Temporal Difference Algorithm Through the Lens of Stochastic Approximation
- How Good Is the TD Solution? Fixed Point Analysis in Linear Approximation
- Almost Sure Convergence Analysis of Temporal Difference Learning via the ODE Method
- Best Policy Algorithm for Q-Value Functions: A Stochastic Approximation Formulation
- Asymptotic Analysis of Q-Learning Algorithm
- Asymptotic Behaviour of the Q-Learning Limit ODE — A Switching Systems Perspective
- Concluding the Asymptotic Analysis of Q-Learning