Course10810417470 lectures
Applied Linear Algebra for Signal Processing, Data Analytics and Machine Learning
Prof. Aditya K. Jagannatham · IIT Kanpur
About
This course aims to introduce students to all the basic and advanced concepts in Linear Algebra with a strong focus on applications. Linear Algebra is one of the fundamental tools that has applications in diverse fields such as Machine Learning, Data Analytics, Signal Processing, Wireless Communication, Operations Research, Control and Finance. The various topics and applications that will be covered in different areas are as follows:
Wireless: MIMO/ OFDM systems, Beamforming, Channel Estimation
Machine Learning: Regression, Clustering, EM Algorithm, Perceptron, SVM/ Kernel SVM, Principal Component Analysis (PCA), Face recognition
Signal Processing: Signal Estimation, Regularization, Compressive Sensing, Image Compression, Robotics and Dynamical systems
Data Analytics: Recommender systems, Data completion, Data prediction, forecasting, Optimal Estimation, Financial models
Operations Resarch: Markov chains, inventory management, supply chain management
Miscellaneous Applications: Electrical circuits, Graph models and social networks, Traffic flow management
The course is suitable for all UG/PG students and practicing engineers/ scientists/ managers from the diverse fields mentioned above and interested in learning about the novel cutting edge applications of linear algebra in various fields such as Machine Learning, Data Analytics, Signal Processing, Wireless Communication.
Intended audience: -Students in Electrical Engineering, Electronics and Communication Engineering, Mathematics, Economics, Computer Science -Practicing engineers -Technical and Non-technical managers of Telecomm companies -Students preparing for Competitive Exams with focus on Wireless Communication, Signal Processing, Machine Learning - Students pursuing projects or research in Machine Learning, Data Analytics and Signal Processing/ Communication
Lectures
Week 1
- Lec 01- Vector Properties: Addition, Linear Combination, Inner Product, Orthogonality, Norm
- Lec 02- Vectors: Unit Norm Vector, Cauchy-Schwarz inequality, Radar Application
- Lec 03- Inner Product Application: Beamforming in Wireless Communication Systems
- Lec 04- Matrices, Definition, Addition and Multiplication of Matrices
- Lec 05- Matrix: Column Space, Linear Independence, Rank of Matrix, Gaussian Elimination
Week 2
- Lec 06- Matrix: Determinant, Inverse Computation, Adjoint, Cofactor Concepts
- Lec 07- Applications of Matrices: Solution of System of Linear equations, MIMO Wireless Technology
- Lec 08- Applications of Matrices: Electric Circuits, Traffic flows
- Lec 09- Applications of Matrices: Graph Theory, Social Networks, Dominance Directed Graph, Influential Node
- Lec 10- Null Space of Matrix: Definition, Rank-Nullity Theorem, Application in Electric Circuits
Week 3
- Lec 11- Gram-Schmidt Orthogonalization
- Lec 12- Gaussian Random Variable: Definition, Mean, Variance, Multivariate Gaussian, Covariance Matrix
- Lec 13- Linear Transformation of Gaussian Random Vectors
- Lec 14- Machine Learning Application: Gaussian Classification
- Lec 15- Eigenvalue: Definition, Characteristic Equation, Eigenvalue Decomposition
- Lec 16- Special Matrices: Rotation and Unitary Matrices, Application: Alamouti Code
Week 4
- Lec 17- Positive Semi-definite (PSD) Matrices: Definition, Properties, Eigenvalue Decomposition
- Lec 18- Positive Semidefinite Matrix: Example and Illustration of Eigenvalue Decomposition
- Lec 19- Machine Learning Application: Principal Component Analysis (PCA)
- Lec 20- Computer Vision Application: Face Recognition, Eigenfaces
Week 5
- Lec 21- Least Squares (LS) Solution, Pseudo-Inverse Concept
- Lec 22- Least Squares (LS) via Principle of Orthogonality, Projection Matrix, Properties
- Lec 23- Application: Pseudo-Inverse and MIMO Zero Forcing (ZF) Receiver
- Lec 24- Wireless Application: Multi-Antenna Channel Estimation
- Lec 25- Machine Learning Application: Linear Regression
Week 6
- Lec 26- Computation Mathematics Application: Polynomial Fitting
- Lec 27- Least Norm Solution
- Lec 28- Wireless Application: Multi-user Beamforming
- Lec 29- Singular Value Decomposition (SVD): Definition, Properties, Example
- Lec 30- SVD Application in MIMO Wireless Technology: Spatial-Multiplexing and High Data Rates
Week 7
- Lec 31- SVD for MIMO wireless optimization, water-filling algorithm, optimal power allocation
- Lec 32- SVD application for Machine Learning: Principal component analysis (PCA)
- Lec 33- Multiple signal classification (MUSIC) algorithm: system model
- Lec 34- MUSIC algorithm for Direction of Arrival (DoA) estimation
- Lec 35- Linear minimum mean square error (LMMSE) principle
- Lec 36- LMMSE estimate and error covariance matrix
Week 8
- Lec 37- LMMSE estimation in linear systems
- Lec 38- LMMSE application: Wireless channel estimation and example
- Lec 39- Time-series prediction via auto-regressive (AR) model
- Lec 40- Recommender system: design and rating prediction
- Lec 41- Recommender system: Illustration via movie rating prediction example
Week 9
- Lec 42- Fast Fourier transform (FFT) and Inverse fast Fourier transform (IFFT)
- Lec 43- IFFT/ FFT application in Orthogonal Frequency Division Multiplexing (OFDM) wireless technology
- Lec 44- OFDM system: Circulant matrices and properties
- Lec 45- OFDM system model: Transmitter and receiver processing
- Lec 46- Single-carrier frequency division for multiple access (SC-FDMA) technology
- Lec 47- Linear dynamical systems: definition and solution via matrix exponential
- Lec 48- Linear dynamical systems: matrix exponential via SVD
Week 10
- Lec 49- Machine Learning application: Support Vector Machines (SVM)
- Lec 50- Support Vector Machines (SVM): Problem formulation via maximum hyperplane separation
- Lec 51- Sparse regression: problem formulation and relation to Compressive Sensing (CS)
- Lec 52- Sparse regression: solution via the Orthogonal Matching Pursuit (OMP) algorithm
- Lec 53- OMP Example for Sparse Regression
- Lec 54- Machine Learning Application: Clustering
- Lec 55- K-Means Clustering algorithm
Week 11
- Lec 56- Introduction to Stochastic Processes and Markov Chains
- Lec 57- Discrete Time Markov Chains and Transition Probability Matrix
- Lec 58- Discrete Time Markov Chain Examples
- Lec 59- m-STEP Transition Probabilities for Discrete Time Markov Chains
- Lec 60- Limiting Behavior of Discrete Time Markov Chains
- Lec 61- Least Squares Revisited: Rank Deficient Matrix
- Lec 62- Least Squares using SVD
Week 12
- Lec 63- Weighted Least Squares
- Lec 64- Weighted Least Squares Example
- Lec 65- Woodbury Matrix Identity – Matrix Inversion Lemma
- Lec 66- Woodbury Matrix Identity – Proof
- Lec 67- Conditional Gaussian Density - Mean
- Lec 68- Conditional Gaussian Density - Covariance
- Lec 69- Scalar Linear Model for Gaussian Estimation
- Lec 70- MMSE Estimate and Covariance for the Scalar Linear Model
Syllabus and materials
At NPTEL, as the course publishes them.
Credits and licence
- Institution
- IIT Kanpur
- Department
- Electrical Engineering
- Teacher
- Prof. Aditya K. Jagannatham
- Level
- Undergraduate
- Language
- English
- Subjects
- Engineering
- Source
- NPTEL
- Licence
- CC BY-NC-SA, as NPTEL states it. Non-commercial use only. NPTEL's terms
- Attribution
- NPTEL states: "Distributed under Creative Commons Attribution-ShareAlike / CC BY - NC - SA", beside a BY-NC-SA badge.
- Who can watch
- NPTEL gives no rating. Invisible Network shows it to Tweens profiles and older, judged from what the source says about it. How courses are placed