Lecture 41 of 41Boosting and Classifier ensembles
Risk minimization view of AdaBoost
Prof. P.S. Sastry · IISc Bangalore
Credits and licence
- From
- Pattern Recognition, IISc Bangalore (at NPTEL)
- Licence
- CC BY-NC-SA, as NPTEL states it. Non-commercial use only.
- Attribution
- NPTEL states: "Distributed under Creative Commons Attribution-ShareAlike / CC BY - NC - SA", beside a BY-NC-SA badge.
- Who can watch
- Teens profiles and older. How courses are placed
All the lectures
- Introduction to Statistical Pattern Recognition
- Overview of Pattern Classifiers
- The Bayes Classifier for minimizing Risk
- Estimating Bayes Error; Minimax and Neymann-Pearson classifiers
- Implementing Bayes Classifier; Estimation of Class Conditional Densities
- Maximum Likelihood estimation of different densities
- Bayesian estimation of parameters of density functions, MAP estimates
- Bayesian Estimation examples; the exponential family of densities and ML estimates
- Sufficient Statistics; Recursive formulation of ML and Bayesian estimates
- Mixture Densities, ML estimation and EM algorithm
- Convergence of EM algorithm; overview of Nonparametric density estimation
- Nonparametric estimation, Parzen Windows, nearest neighbour methods
- Linear Discriminant Functions; Perceptron -- Learning Algorithm and convergence proof
- Linear Least Squares Regression; LMS algorithm
- AdaLinE and LMS algorithm; General nonliner least-squares regression
- Logistic Regression; Statistics of least squares method; Regularized Least Squares
- Fisher Linear Discriminant
- Linear Discriminant functions for multi-class case; multi-class logistic regression
- Learning and Generalization; PAC learning framework
- Overview of Statistical Learning Theory; Empirical Risk Minimization
- Consistency of Empirical Risk Minimization
- Consistency of Empirical Risk Minimization; VC-Dimension
- Complexity of Learning problems and VC-Dimension
- VC-Dimension Examples; VC-Dimension of hyperplanes
- Overview of Artificial Neural Networks
- Multilayer Feedforward Neural networks with Sigmoidal activation functions;
- Backpropagation Algorithm; Representational abilities of feedforward networks
- Feedforward networks for Classification and Regression; Backpropagation in Practice
- Radial Basis Function Networks; Gaussian RBF networks
- Learning Weights in RBF networks; K-means clustering algorithm
- Support Vector Machines -- Introduction, obtaining the optimal hyperplane
- SVM formulation with slack variables; nonlinear SVM classifiers
- Kernel Functions for nonlinear SVMs; Mercer and positive definite Kernels
- Support Vector Regression and ?-insensitive Loss function, examples of SVM learning
- Overview of SMO and other algorithms for SVM; ?-SVM and ?-SVR; SVM as a risk minimizer
- Positive Definite Kernels; RKHS; Representer Theorem
- Feature Selection and Dimensionality Reduction; Principal Component Analysis
- No Free Lunch Theorem; Model selection and model estimation; Bias-variance trade-off
- Assessing Learnt classifiers; Cross Validation;
- Bootstrap, Bagging and Boosting; Classifier Ensembles; AdaBoost
- Risk minimization view of AdaBoost