Pattern Recognition

Lectures

Overview of Pattern classification and regression

1. Introduction to Statistical Pattern Recognition Video
2. Overview of Pattern Classifiers Video

Bayesian decision making and Bayes Classifier

3. The Bayes Classifier for minimizing Risk Video
4. Estimating Bayes Error; Minimax and Neymann-Pearson classifiers Video

Parametric Estimation of Densities

5. Implementing Bayes Classifier; Estimation of Class Conditional Densities Video
6. Maximum Likelihood estimation of different densities Video
7. Bayesian estimation of parameters of density functions, MAP estimates Video
8. Bayesian Estimation examples; the exponential family of densities and ML estimates Video
9. Sufficient Statistics; Recursive formulation of ML and Bayesian estimates Video

Mixture Densities and EM Algorithm

10. Mixture Densities, ML estimation and EM algorithm Video
11. Convergence of EM algorithm; overview of Nonparametric density estimation Video

Nonparametric density estimation

12. Nonparametric estimation, Parzen Windows, nearest neighbour methods Video

Linear models for classification and regression

13. Linear Discriminant Functions; Perceptron -- Learning Algorithm and convergence proof Video
14. Linear Least Squares Regression; LMS algorithm Video
15. AdaLinE and LMS algorithm; General nonliner least-squares regression Video
16. Logistic Regression; Statistics of least squares method; Regularized Least Squares Video
17. Fisher Linear Discriminant Video
18. Linear Discriminant functions for multi-class case; multi-class logistic regression Video

Overview of statistical learning theory, Empirical Risk Minimization and VC-Dimension

19. Learning and Generalization; PAC learning framework Video
20. Overview of Statistical Learning Theory; Empirical Risk Minimization Video
21. Consistency of Empirical Risk Minimization Video
22. Consistency of Empirical Risk Minimization; VC-Dimension Video
23. Complexity of Learning problems and VC-Dimension Video
24. VC-Dimension Examples; VC-Dimension of hyperplanes Video

Artificial Neural Networks for Classification and regression

25. Overview of Artificial Neural Networks Video
26. Multilayer Feedforward Neural networks with Sigmoidal activation functions; Video
27. Backpropagation Algorithm; Representational abilities of feedforward networks Video
28. Feedforward networks for Classification and Regression; Backpropagation in Practice Video
29. Radial Basis Function Networks; Gaussian RBF networks Video
30. Learning Weights in RBF networks; K-means clustering algorithm Video

Support Vector Machines and Kernel based methods

31. Support Vector Machines -- Introduction, obtaining the optimal hyperplane Video
32. SVM formulation with slack variables; nonlinear SVM classifiers Video
33. Kernel Functions for nonlinear SVMs; Mercer and positive definite Kernels Video
34. Support Vector Regression and ?-insensitive Loss function, examples of SVM learning Video
35. Overview of SMO and other algorithms for SVM; ?-SVM and ?-SVR; SVM as a risk minimizer Video
36. Positive Definite Kernels; RKHS; Representer Theorem Video

Feature Selection, Model assessment and cross-validation

37. Feature Selection and Dimensionality Reduction; Principal Component Analysis Video
38. No Free Lunch Theorem; Model selection and model estimation; Bias-variance trade-off Video
39. Assessing Learnt classifiers; Cross Validation; Video

Boosting and Classifier ensembles

40. Bootstrap, Bagging and Boosting; Classifier Ensembles; AdaBoost Video
41. Risk minimization view of AdaBoost Video