# Matrix Calculus for Machine Learning and Beyond

- Institution: Massachusetts Institute of Technology
- Department: Mathematics
- Taught by: Prof. Alan Edelman, Prof. Steven G. Johnson
- Term: January IAP 2023
- Level: Undergraduate
- Language: English
- Subjects: Mathematics
- Source: [MIT OpenCourseWare](https://ocw.mit.edu/courses/18-s096-matrix-calculus-for-machine-learning-and-beyond-january-iap-2023/)
- Licence: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/) (Non-commercial use only.)
- Syllabus: [at the source](https://ocw.mit.edu/courses/18-s096-matrix-calculus-for-machine-learning-and-beyond-january-iap-2023/pages/syllabus/)
- Who can watch: Tweens profiles and older (sources give no ratings; placed by Invisible Network from the source's own words)

We all know that calculus courses such as 18.01 Single Variable Calculus and 18.02 Multivariable Calculus cover univariate and vector calculus, respectively. Modern applications such as machine learning and large-scale optimization require the next big step, “matrix calculus” and calculus on arbitrary vector spaces.

This class covers a coherent approach to matrix calculus showing techniques that allow you to think of a matrix holistically (not just as an array of scalars), generalize and compute derivatives of important matrix factorizations and many other complicated-looking operations, and understand how differentiation formulas must be reimagined in large-scale computing.

## Lectures

1. [Lecture 1 Part 1: Introduction and Motivation](https://invisiblenetwork.ai/learn/mit/18-s096-matrix-calculus-for-machine-learning-and-beyond-january-iap-2023/f353f31b95bb) (Video)
2. [Lecture 1 Part 2: Derivatives as Linear Operators](https://invisiblenetwork.ai/learn/mit/18-s096-matrix-calculus-for-machine-learning-and-beyond-january-iap-2023/789be121b242) (Video)
3. [Lecture 2 Part 1: Derivatives in Higher Dimensions: Jacobians and Matrix Functions](https://invisiblenetwork.ai/learn/mit/18-s096-matrix-calculus-for-machine-learning-and-beyond-january-iap-2023/6bbbdda4cebc) (Video)
4. [Lecture 2 Part 2: Vectorization of Matrix Functions](https://invisiblenetwork.ai/learn/mit/18-s096-matrix-calculus-for-machine-learning-and-beyond-january-iap-2023/8b8e76d8d1c4) (Video)
5. [Lecture 3 Part 1: Kronecker Products and Jacobians](https://invisiblenetwork.ai/learn/mit/18-s096-matrix-calculus-for-machine-learning-and-beyond-january-iap-2023/6a84ca0268f2) (Video)
6. [Lecture 3 Part 2: Finite-Difference Approximations](https://invisiblenetwork.ai/learn/mit/18-s096-matrix-calculus-for-machine-learning-and-beyond-january-iap-2023/4ebcc2e978be) (Video)
7. [Lecture 4 Part 1: Gradients and Inner Products in Other Vector Spaces](https://invisiblenetwork.ai/learn/mit/18-s096-matrix-calculus-for-machine-learning-and-beyond-january-iap-2023/5de1c1f24209) (Video)
8. [Lecture 4 Part 2: Nonlinear Root Finding, Optimization, and Adjoint Gradient Methods](https://invisiblenetwork.ai/learn/mit/18-s096-matrix-calculus-for-machine-learning-and-beyond-january-iap-2023/5bffd49c5641) (Video)
9. [Lecture 5 Part 1: Derivative of Matrix Determinant and Inverse](https://invisiblenetwork.ai/learn/mit/18-s096-matrix-calculus-for-machine-learning-and-beyond-january-iap-2023/a440e785548f) (Video)
10. [Lecture 5 Part 2: Forward Automatic Differentiation via Dual Numbers](https://invisiblenetwork.ai/learn/mit/18-s096-matrix-calculus-for-machine-learning-and-beyond-january-iap-2023/7bac931c550e) (Video)
11. [Lecture 5 Part 3: Differentiation on Computational Graphs](https://invisiblenetwork.ai/learn/mit/18-s096-matrix-calculus-for-machine-learning-and-beyond-january-iap-2023/4a9a25b9991c) (Video)
12. [Lecture 6 Part 1: Adjoint Differentiation of ODE Solutions](https://invisiblenetwork.ai/learn/mit/18-s096-matrix-calculus-for-machine-learning-and-beyond-january-iap-2023/c4d51a7a877f) (Video)
13. [Lecture 6 Part 2: Calculus of Variations and Gradients of Functionals](https://invisiblenetwork.ai/learn/mit/18-s096-matrix-calculus-for-machine-learning-and-beyond-january-iap-2023/d1c762e83336) (Video)
14. [Lecture 7 Part 1: Derivatives of Random Functions](https://invisiblenetwork.ai/learn/mit/18-s096-matrix-calculus-for-machine-learning-and-beyond-january-iap-2023/4a332e1e0471) (Video)
15. [Lecture 7 Part 2: Second Derivatives, Bilinear Forms, and Hessian Matrices](https://invisiblenetwork.ai/learn/mit/18-s096-matrix-calculus-for-machine-learning-and-beyond-january-iap-2023/6ee539a023eb) (Video)
16. [Lecture 8 Part 1: Derivatives of Eigenproblems](https://invisiblenetwork.ai/learn/mit/18-s096-matrix-calculus-for-machine-learning-and-beyond-january-iap-2023/89fdf5a8f2bd) (Video)
17. [Lecture 8 Part 2: Automatic Differentiation on Computational Graphs](https://invisiblenetwork.ai/learn/mit/18-s096-matrix-calculus-for-machine-learning-and-beyond-january-iap-2023/2f38ef5629fd) (Video)