# Foundations of Machine Learning

- Institution: IIT Hyderabad
- Department: Computer Science and Engineering
- Teacher: Prof. J. SakethaNath
- Level: Undergraduate
- Language: English
- Subjects: Computer science, Engineering
- Source: [NPTEL](https://nptel.ac.in/courses/106106001)
- Licence: [CC BY-NC-SA](https://nptel.ac.in/faq) (Non-commercial use only.)
- Syllabus: [at the source](https://nptel.ac.in/courses/106106001)
- Who can watch: Tweens profiles and older (sources give no ratings; placed by Invisible Network from the source's own words)

This is a course on Machine Learning that introduces the subject in a formal way with a focus on basic and foundational concepts. A holistic view of machine learning is taken, including aspects of optimization (SGD) that are ignored in classical narratives. Topics that are relevant to modern models are highlighted so that the connection to state-of-the-art is not missed in the theoretical exploration. For example, in case of linear regression, apart from classical topics like bias-variance trade-off and regularization, an analysis in the overparametrization regime is also taught that immediately connects you to modern day complex models with billions of parameters. Finally, foundational topics for generative models are also covered, including energy-based models and Langevin sampling.

Intended audience: Students wishing to start research in core machine learning areas

Prerequisites: Anyone with a good understanding of basic engineering math is eligible. Especially subjects like linear algebra, probability theory, and optimization.

## Lectures

1. [What is ML ?](https://invisiblenetwork.ai/learn/nptel/106106001/115381febed7) (YouTube)
2. [Stochastic framework for Supervised Learning](https://invisiblenetwork.ai/learn/nptel/106106001/904a36774057) (YouTube)
3. [Risk Minimization and Bayes optimal](https://invisiblenetwork.ai/learn/nptel/106106001/c8b3c1baf349) (YouTube)
4. [Stochastic Optimization and ERM](https://invisiblenetwork.ai/learn/nptel/106106001/2a39e315e3db) (YouTube)
5. [Linear Regression](https://invisiblenetwork.ai/learn/nptel/106106001/402d44a3584c) (YouTube)
6. [Linear Regression - Estimation Error](https://invisiblenetwork.ai/learn/nptel/106106001/a4da49c862a2) (YouTube)
7. [Supervised Learning >> Denoising/Curve-fitting](https://invisiblenetwork.ai/learn/nptel/106106001/db0cac1c0de9) (YouTube)
8. [Linear modelling](https://invisiblenetwork.ai/learn/nptel/106106001/cf362e086e32) (YouTube)
9. [Linear Classification](https://invisiblenetwork.ai/learn/nptel/106106001/afa257717a77) (YouTube)
10. [Logistic Regression](https://invisiblenetwork.ai/learn/nptel/106106001/7bfaafc9afdd) (YouTube)
11. [Logistic Regression - ERM](https://invisiblenetwork.ai/learn/nptel/106106001/91f36200b87a) (YouTube)
12. [Linear Classification - Estimation Error - Part 1](https://invisiblenetwork.ai/learn/nptel/106106001/1c10f7ee76f6) (YouTube)
13. [Linear Classification - Estimation Error - Part 2](https://invisiblenetwork.ai/learn/nptel/106106001/7ff489597076) (YouTube)
14. [Gradient Descent - Primal & Dual view](https://invisiblenetwork.ai/learn/nptel/106106001/d154f2c330c1) (YouTube)
15. [Smooth Functions and examples](https://invisiblenetwork.ai/learn/nptel/106106001/18b71314b705) (YouTube)
16. [Gradient Descent - Convergence to stationary point](https://invisiblenetwork.ai/learn/nptel/106106001/4fb893e76181) (YouTube)
17. [Rapid Convergence to global optima in Linear Regression](https://invisiblenetwork.ai/learn/nptel/106106001/5126a6d00669) (YouTube)
18. [KL condition and convergence](https://invisiblenetwork.ai/learn/nptel/106106001/aeedc431d84d) (YouTube)
19. [Fast Convergence to global optima in Logistic Regression](https://invisiblenetwork.ai/learn/nptel/106106001/8dbc3e71ff84) (YouTube)
20. [Stochastic Gradient Descent (SGD)](https://invisiblenetwork.ai/learn/nptel/106106001/bfdc19ca3c68) (YouTube)
21. [SGD vs GD for Linear Regression](https://invisiblenetwork.ai/learn/nptel/106106001/c902a8cad886) (YouTube)
22. [SGD vs GD for Logistic Regression](https://invisiblenetwork.ai/learn/nptel/106106001/684780e7d32f) (YouTube)
23. [Model Selection](https://invisiblenetwork.ai/learn/nptel/106106001/5302c890baf5) (YouTube)
24. [What is Regularization ?](https://invisiblenetwork.ai/learn/nptel/106106001/e2736e8504ba) (YouTube)
25. [Linear algebra motivation for min-norm regularization](https://invisiblenetwork.ai/learn/nptel/106106001/ed0318bf1e2a) (YouTube)
26. [Regularization for controlling effective dimension](https://invisiblenetwork.ai/learn/nptel/106106001/e15b8c669d63) (YouTube)
27. [Bias-variance trade-off for estimation error](https://invisiblenetwork.ai/learn/nptel/106106001/064edf3acaae) (YouTube)
28. [Equivalent forms of regularization](https://invisiblenetwork.ai/learn/nptel/106106001/5813e7e04d4a) (YouTube)
29. [Regularization in linear classification](https://invisiblenetwork.ai/learn/nptel/106106001/40f512864732) (YouTube)
30. [Implicit regularization with GD/SGD](https://invisiblenetwork.ai/learn/nptel/106106001/b8c32780492c) (YouTube)
31. [Linear modelling in infinite-dimensional spaces](https://invisiblenetwork.ai/learn/nptel/106106001/b67251e48a22) (YouTube)
32. [Introduction to kernels](https://invisiblenetwork.ai/learn/nptel/106106001/e0286cda008b) (YouTube)
33. [Linear kernels](https://invisiblenetwork.ai/learn/nptel/106106001/721d89feae88) (YouTube)
34. [Polynomial Kernels](https://invisiblenetwork.ai/learn/nptel/106106001/cb88fb399159) (YouTube)
35. [Few properties of kernels](https://invisiblenetwork.ai/learn/nptel/106106001/2d63bdd31331) (YouTube)
36. [Characterization of kernels](https://invisiblenetwork.ai/learn/nptel/106106001/dd00afd190ce) (YouTube)
37. [Universal kernels](https://invisiblenetwork.ai/learn/nptel/106106001/cdbf00ea8f1e) (YouTube)
38. [Universal kernel machines](https://invisiblenetwork.ai/learn/nptel/106106001/5205062d0b9f) (YouTube)
39. [Bayes consistency with universal kernel m/c](https://invisiblenetwork.ai/learn/nptel/106106001/39192afa104c) (YouTube)
40. [Rate of generalization with kernel m/c](https://invisiblenetwork.ai/learn/nptel/106106001/200aa1301cf2) (YouTube)
41. [Overparametrization](https://invisiblenetwork.ai/learn/nptel/106106001/e09fdd976847) (YouTube)
42. [Modern perspective of ML: Minimal vs optimal regularization](https://invisiblenetwork.ai/learn/nptel/106106001/ab5c69d2c1b7) (YouTube)
43. [The case for latent variable modelling](https://invisiblenetwork.ai/learn/nptel/106106001/7da9ce9ce171) (YouTube)
44. [Modularity in modelling: Deep learning](https://invisiblenetwork.ai/learn/nptel/106106001/0f2a83ea336c) (YouTube)
45. [Modelling with Deep learning](https://invisiblenetwork.ai/learn/nptel/106106001/ca2f56b4c81b) (YouTube)
46. [Estimation with Deep learning](https://invisiblenetwork.ai/learn/nptel/106106001/c2f2cd34a049) (YouTube)
47. [Backpropagation-1](https://invisiblenetwork.ai/learn/nptel/106106001/73a11e041c17) (YouTube)
48. [Backpropagation-2](https://invisiblenetwork.ai/learn/nptel/106106001/40feaced771e) (YouTube)
49. [Optimization error in 1-hidden layer nets](https://invisiblenetwork.ai/learn/nptel/106106001/ec282744c0a5) (YouTube)
50. [Implicit bias in 1-hidden layer nets](https://invisiblenetwork.ai/learn/nptel/106106001/1681a334fc28) (YouTube)
51. [Optimization error in deep learning](https://invisiblenetwork.ai/learn/nptel/106106001/4649ca4ca9e4) (YouTube)
52. [Likelihood estimation](https://invisiblenetwork.ai/learn/nptel/106106001/ed30f6a4debd) (YouTube)
53. [Energy based models, MLE](https://invisiblenetwork.ai/learn/nptel/106106001/e4d14827f931) (YouTube)
54. [MLE examples](https://invisiblenetwork.ai/learn/nptel/106106001/fa7d8ca06ccc) (YouTube)
55. [Generative regression](https://invisiblenetwork.ai/learn/nptel/106106001/143a8e23046d) (YouTube)
56. [Generative classification](https://invisiblenetwork.ai/learn/nptel/106106001/ec091d236a9d) (YouTube)
57. [Discriminative Models and MCLE](https://invisiblenetwork.ai/learn/nptel/106106001/f998ff850eb8) (YouTube)
58. [Langevin Sampling](https://invisiblenetwork.ai/learn/nptel/106106001/1fd50f43db53) (YouTube)
59. [Ensembling: Bias-variance trade-off](https://invisiblenetwork.ai/learn/nptel/106106001/04962efc15c6) (YouTube)
60. [Ensembling analysis: Bagging](https://invisiblenetwork.ai/learn/nptel/106106001/c466311ac8af) (YouTube)
61. [Concluding Remarks](https://invisiblenetwork.ai/learn/nptel/106106001/bee50d798617) (YouTube)