Learn · Foundations of Machine Learning
Lecture 49 of 61Week 10
Optimization error in 1-hidden layer nets
Prof. J. SakethaNath · IIT Hyderabad
Credits and licence
- From
- Foundations of Machine Learning, IIT Hyderabad (at NPTEL)
- Licence
- CC BY-NC-SA, as NPTEL states it. Non-commercial use only.
- Attribution
- NPTEL states: "Distributed under Creative Commons Attribution-ShareAlike / CC BY - NC - SA", beside a BY-NC-SA badge. Coordinated by IIT Madras.
- Who can watch
- Tweens profiles and older. How courses are placed
All the lectures
- What is ML ?
- Stochastic framework for Supervised Learning
- Risk Minimization and Bayes optimal
- Stochastic Optimization and ERM
- Linear Regression
- Linear Regression - Estimation Error
- Supervised Learning >> Denoising/Curve-fitting
- Linear modelling
- Linear Classification
- Logistic Regression
- Logistic Regression - ERM
- Linear Classification - Estimation Error - Part 1
- Linear Classification - Estimation Error - Part 2
- Gradient Descent - Primal & Dual view
- Smooth Functions and examples
- Gradient Descent - Convergence to stationary point
- Rapid Convergence to global optima in Linear Regression
- KL condition and convergence
- Fast Convergence to global optima in Logistic Regression
- Stochastic Gradient Descent (SGD)
- SGD vs GD for Linear Regression
- SGD vs GD for Logistic Regression
- Model Selection
- What is Regularization ?
- Linear algebra motivation for min-norm regularization
- Regularization for controlling effective dimension
- Bias-variance trade-off for estimation error
- Equivalent forms of regularization
- Regularization in linear classification
- Implicit regularization with GD/SGD
- Linear modelling in infinite-dimensional spaces
- Introduction to kernels
- Linear kernels
- Polynomial Kernels
- Few properties of kernels
- Characterization of kernels
- Universal kernels
- Universal kernel machines
- Bayes consistency with universal kernel m/c
- Rate of generalization with kernel m/c
- Overparametrization
- Modern perspective of ML: Minimal vs optimal regularization
- The case for latent variable modelling
- Modularity in modelling: Deep learning
- Modelling with Deep learning
- Estimation with Deep learning
- Backpropagation-1
- Backpropagation-2
- Optimization error in 1-hidden layer nets
- Implicit bias in 1-hidden layer nets
- Optimization error in deep learning
- Likelihood estimation
- Energy based models, MLE
- MLE examples
- Generative regression
- Generative classification
- Discriminative Models and MCLE
- Langevin Sampling
- Ensembling: Bias-variance trade-off
- Ensembling analysis: Bagging
- Concluding Remarks