Course10810814866 lectures
Neural Networks for Signal Processing - I
Prof. Shayan Srinivasa Garani · IISc Bangalore
About
This will be an introductory graduate level course in neural networks for signal processing. The course starts with a motivation of how the human brain is inspirational to building artificial neural networks. The neural networks are viewed as directed graphs with various network topologies towards learning tasks driven by optimization techniques. The course covers Rosenblatt’s perceptron, regression modeling, multilayer perceptron (MLP), kernel methods and radial basis functions (RBF), support vector machines (SVM), regularization theory and principal component analysis (Hebbian and kernel based). Towards the end, topics such as convolutional neural networks etc. that are based on the MLP will be touched upon. The course will have assignments that are theoretical and computer based working with actual data.
Intended audience: Graduate level, Senior UG can also participate, engineers and scientists within related industry.
Prerequisites: Basic mathematical background in probability, linear algebra, signals and systems or equivalent.
Lectures
Week 1 - Introduction to Neural Networks
- The Human Brain
- Introduction to Neural Networks
- Models of a neuron
- Feedback and network architectures
- Knowledge representation
- Prior information and invariances
Week 2 - Rosenblatt’s Perceptron
- Learning processes
- Perceptron 1
- Perceptron 2
- Batch perceptron algorithm
- Perceptron and Bayes classifier
Week 3 - Linear, logisitic regression, and multilayer perceptron
- Linear regression 1
- Linear regression 2
- Linear regression 3
- Logistic regression
- Multi-layer perceptron 1
Week 4 - Multilayer perceptron
Week 5 - Approximation of functions, CNN and Cover’s Theorem
- Universal approximation theorem
- Complexity Regularization and Cross validation
- Convolutional Neural Networks (CNN)
- Cover’s Theorem
Week 6 - Radial Basis Function (RBF)
- Multivariate interpolation problem
- Radial basis functions (RBF)
- Recursive least squares algorithm
- Comparison of RBF with MLP
- Kernel regression using RBFs
- Kernel Functions
- Basics of constrained optimization
Week 7 - Support Vector Machine (SVM)
- Optimization with equality constraint
- Optimization with inequality constraint
- Support Vector Machines (SVM)
- Optimal hyperplane for linearly separable patterns
- Quadratic optimization for finding optimal hyperplane
Week 8 - Support Vector Machine (SVM) - II
- Optimal hyperplane for non-linearly separable patterns
- Inner product kernel and Mercer’s theorem
- Optimal design of an SVM
- ε-insensitive loss function
- XOR problem revisited using SVMs
- Hilbert Space
Week 9 - Reproducing Kernel Hilbert Space and Regularization
- Reproducing Kernel Hilbert Space
- Representer Theorem
- Generalized applicability of the representer theorem
- Regularization Theory
- Euler-Lagrange Equation
- Regularization Networks
Week 10 - Structural Risk Minimization and Bias-Variance Dilemma
- Generalized RBF networks
- XOR problem revisited using RBF
- Structural Risk Minimization
- Bias-Variance Dilemma
- Estimation of regularization parameters
Week 11 - L1 regularization
Week 12 - Assorted topics
Syllabus and materials
At NPTEL, as the course publishes them.
Credits and licence
- Institution
- IISc Bangalore
- Department
- Electrical Engineering
- Teacher
- Prof. Shayan Srinivasa Garani
- Level
- Graduate
- Language
- English
- Subjects
- Engineering
- Source
- NPTEL
- Licence
- CC BY-NC-SA, as NPTEL states it. Non-commercial use only. NPTEL's terms
- Attribution
- NPTEL states: "Distributed under Creative Commons Attribution-ShareAlike / CC BY - NC - SA", beside a BY-NC-SA badge.
- Who can watch
- NPTEL gives no rating. Invisible Network shows it to Tweens profiles and older, judged from what the source says about it. How courses are placed