Learn · Stanford Engineering Everywhere

CourseCS22920 lectures25 h 10 min

Machine Learning

Andrew Ng · Stanford University

Start with lecture 1 At Stanford Engineering Everywhere

About

This course provides a broad introduction to machine learning and statistical pattern recognition.

Topics include: supervised learning (generative/discriminative learning, parametric/non-parametric learning, neural networks, support vector machines); unsupervised learning (clustering, dimensionality reduction, kernel methods); learning theory (bias/variance tradeoffs; VC theory; large margins); reinforcement learning and adaptive control. The course will also discuss recent applications of machine learning, such as to robotic control, data mining, autonomous navigation, bioinformatics, speech recognition, and text and web data processing.

Students are expected to have the following background:

Prerequisites: - Knowledge of basic computer science principles and skills, at a level sufficient to write a reasonably non-trivial computer program.

- Familiarity with the basic probability theory. (Stat 116 is sufficient but not necessary.)

- Familiarity with the basic linear algebra (any one of Math 51, Math 103, Math 113, or CS 205 would be much more than necessary.)

Lectures

  1. The Motivation & Applications of Machine Learning, The Logistics of the Class, The Definition of Machine Learning, The Overview of Supervised Learning, The Overview of Learning Theory, The Overview of Unsupervised Learning, The Overview… Lecture 1 · 1 h 9 min · Video
  2. An Application of Supervised Learning - Autonomous Deriving, ALVINN, Linear Regression, Gradient Descent, Batch Gradient Descent, Stochastic Gradient Descent (Incremental Descent), Matrix Derivative Notation for Deriving Normal Equations,… Lecture 2 · 1 h 16 min · Video
  3. The Concept of Underfitting and Overfitting, The Concept of Parametric Algorithms and Non-parametric Algorithms, Locally Weighted Regression, The Probabilistic Interpretation of Linear Regression, The motivation of Logistic Regression,… Lecture 3 · 1 h 13 min · Video
  4. Newton's Method, Exponential Family, Bernoulli Example, Gaussian Example, General Linear Models (GLMs), Multinomial Example, Softmax Regression Lecture 4 · 1 h 13 min · Video
  5. Discriminative Algorithms, Generative Algorithms, Gaussian Discriminant Analysis (GDA), GDA and Logistic Regression, Naive Bayes, Laplace Smoothing Lecture 5 · 1 h 16 min · Video
  6. Multinomial Event Model, Non-linear Classifiers, Neural Network, Applications of Neural Network, Intuitions about Support Vector Machine (SVM), Notation for SVM, Functional and Geometric Margins Lecture 6 · 1 h 14 min · Video
  7. Optimal Margin Classifier, Lagrange Duality, Karush-Kuhn-Tucker (KKT) Conditions, SVM Dual, The Concept of Kernels Lecture 7 · 1 h 16 min · Video
  8. Kernels, Mercer's Theorem, Non-linear Decision Boundaries and Soft Margin SVM, Coordinate Ascent Algorithm, The Sequential Minimization Optimization (SMO) Algorithm, Applications of SVM Lecture 8 · 1 h 17 min · Video
  9. Bias/variance Tradeoff, Empirical Risk Minimization (ERM), The Union Bound, Hoeffding Inequality, Uniform Convergence - The Case of Finite H, Sample Complexity Bound, Error Bound, Uniform Convergence Theorem & Corollary Lecture 9 · 1 h 14 min · Video
  10. Uniform Convergence - The Case of Infinite H, The Concept of 'Shatter' and VC Dimension, SVM Example, Model Selection, Cross Validation, Feature Selection Lecture 10 · 1 h 13 min · Video
  11. Bayesian Statistics and Regularization, Online Learning, Advice for Applying Machine Learning Algorithms, Debugging/fixing Learning Algorithms, Diagnostics for Bias & Variance, Optimization Algorithm Diagnostics, Diagnostic Example -… Lecture 11 · 1 h 22 min · Video
  12. The Concept of Unsupervised Learning, K-means Clustering Algorithm, K-means Algorithm, Mixtures of Gaussians and the EM Algorithm, Jensen's Inequality, The EM Algorithm, Summary Lecture 12 · 1 h 14 min · Video
  13. Mixture of Gaussian, Mixture of Naive Bayes - Text clustering (EM Application), Factor Analysis, Restrictions on a Covariance Matrix, The Factor Analysis Model, EM for Factor Analysis Lecture 13 · 1 h 15 min · Video
  14. The Factor Analysis Model,0 EM for Factor Analysis, Principal Component Analysis (PCA), PCA as a Dimensionality Reduction Algorithm, Applications of PCA, Face Recognition by Using PCA Lecture 14 · 1 h 21 min · Video
  15. Latent Semantic Indexing (LSI), Singular Value Decomposition (SVD) Implementation, Independent Component Analysis (ICA), The Application of ICA, Cumulative Distribution Function (CDF), ICA Algorithm, The Applications of ICA Lecture 15 · 1 h 17 min · Video
  16. Applications of Reinforcement Learning, Markov Decision Process (MDP), Defining Value & Policy Functions, Value Function, Optimal Value Function, Value Iteration, Policy Iteration Lecture 16 · 1 h 13 min · Video
  17. Generalization to Continuous States, Discretization & Curse of Dimensionality, Models/Simulators, Fitted Value Iteration, Finding Optimal Policy Lecture 17 · 1 h 17 min · Video
  18. State-action Rewards, Finite Horizon MDPs, The Concept of Dynamical Systems, Examples of Dynamical Models, Linear Quadratic Regulation (LQR), Linearizing a Non-Linear Model, Computing Rewards, Riccati Equation Lecture 18 · 1 h 17 min · Video
  19. Advice for Applying Machine Learning, Debugging Reinforcement Learning (RL) Algorithm, Linear Quadratic Regularization (LQR), Differential Dynamic Programming (DDP), Kalman Filter & Linear Quadratic Gaussian (LQG), Predict/update Steps of… Lecture 19 · 1 h 16 min · Video
  20. Partially Observable MDPs (POMDPs), Policy Search, Reinforce Algorithm, Pegasus Algorithm, Pegasus Policy Search, Applications of Reinforcement Learning Lecture 20 · 1 h 17 min · Video

Syllabus and materials

At Stanford Engineering Everywhere, as the course publishes them.

Credits and licence

Institution
Stanford University
Department
Computer Science
Teacher
Andrew Ng
Level
Undergraduate
Language
English
Subjects
Computer science
Source
Stanford Engineering Everywhere
Licence
CC BY-NC-SA 4.0, as Stanford Engineering Everywhere states it. Non-commercial use only. Stanford Engineering Everywhere's terms
Attribution
Stanford Engineering Everywhere, Stanford University. Licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License, unless otherwise expressly indicated.
Who can watch
Stanford Engineering Everywhere gives no rating. Invisible Network shows it to Tweens profiles and older, judged from what the source says about it. How courses are placed