Course10810118565 lectures
Stochastic Control and Communication
Prof. Ankur A. Kulkarni · IIT Bombay
About
The legacy of control is marked by a historic misconception that stochastic control problems could be solved using deterministic controllers superimposed with optimal state estimators and conventional communication protocols. Over time, we realized that this architecture is suboptimal and co-design of communication and control offers orders of magnitude of improvements. This applies to systems such as smart cities, transportation systems, power grids, gaming and financial markets. This complex subject requires simultaneous application of control theory and communication theory. At present these elements are taught separately in courses designed either exclusively for deterministic control theory, or exclusively for communication theory. This leaves behind a gap, namely blending insights from both disciplines to arrive at approaches for networked control problems. This course seeks to fill this gap. It will begin with centralized stochastic control, and then highlight open issues that arise in decentralization and the role of communication theory. Finally it will cover communication theory and establish viewpoints from which stochastic control and communication theory can be approached simultaneously.
Intended audience: Students, researchers and practitioners of control and automation across any discipline.
Prerequisites: Comfort with probability.INTENDED AUDIENCE: Students, researchers and practitioners of control and automation across any discipline.
Lectures
Week 1
- Lecture 1 : Decision Making under Uncertainty
- Lecture 2 : Expected Utility Theory
- Lecture 3 : Expected Utility Theory II
- Lecture 4 : Expected Utility Theory III
- Lecture 5 : Role of Information in Decision Making
Week 2
- Lecture 6 : State space modelling of sequential decision making; example of Inventory Control Problem -I
- Lecture 7 : Inventory Control Problem II
- Lecture 8 : Policy as a solution concept
- Lecture 9 : Introduction to MDP
- Lecture 10 : Types of Policy in MDP
Week 3
- Lecture 11 : Interpreting randomised decision rules
- Lecture 12 : Stationary Transition Probability: State Diagram Representation and example of Markov policies
- Lecture 13 : Example of History Dependent Policies
- Lecture 14 : Complexity of the problem using brute force approach
- Lecture 15 : Principle of Optimality
Week 4
- Lecture 16 : Dynamic Programming Algorithm
- Lecture 17 : DP Algo applied to Inventory Control Problem
- Lecture 18 : DP Algo applied to Inventory Control Problem (Continued)
- Lecture 19 : DP Algo applied to Inventory Control Problem (Continued)
- Lecture 20 : Optimal Stopping Problem
Week 5
- Lecture 21 : Optimal Stopping Example: Secretary Problem
- Lecture 22 : Optimal Stopping Example: Secretary Problem (Continued)
- Lecture 23 : Optimal Stopping Example: Secretary Problem (Continued)
- Lecture 24 : Linear System Quadratic Cost Problem
- Lecture 25 : Linear System Quadratic Cost Problem (Continued)
Week 6
- Lecture 26 : Solving it via DP algorithm (Continued)
- Lecture 27 : Equivalence between Optimal HR Policyand optimal Markov Deterministic Policy
- Lecture 28 : Stochastic Control under incomplete state information
- Lecture 29 : Stochastic Control under incomplete state information (Continued)
- Lecture 30 : Stochastic Control under incomplete state information: Example
- Lecture 31 : Stochastic Control under incomplete state information: Example (Continued)
- Lecture 32 : Stochastic Control under incomplete state information: Example (Continued)
- Lecture 33 : Stochastic Control under incomplete state information: Example (Continued)
Week 7
- Lecture 34 : LQ systems with Imperfect Information- I
- Lecture 35 : LQ systems with Imperfect Information- II
- Lecture 36 : LQ systems with Imperfect Information-III
- Lecture 37 : LQ systems with Imperfect Information-IV
- Lecture 38 : Filtering-I
- Lecture 39 : Filtering-II
Week 8
- Lecture 40 : Kalman Filtering-I
- Lecture 41 : Kalman Filtering-II
- Lecture 42 : Kalman Filtering-III
- Lecture 43 : Belief State Formulation-I
- Lecture 44 : Belief State Formulation-II
Week 9
- Lecture 45 : Information Structures-I
- Lecture 46 : Information Structures-II
- Lecture 47 : Witsenhausen Problem-I
- Lecture 48 : Witsenhausen Problem-II
- Lecture 49 : Witsenhausen Problem-III
Week 10
- Lecture 50 : Witsenhausen Problem-IV
- Lecture 51 : Witsenhausen Problem-V
- Lecture 52 : Witsenhausen Problem-VI
- Lecture 53 : Witsenhausen Problem-VII
Week 11
- Lecture 54 : Team Decision Theory-I
- Lecture 55 : Team Decision Theory-II
- Lecture 56 : Team Decision Theory-III
- Lecture 57 : Team Decision Theory-IV
- Lecture 58 : Team Decision Theory-V
- Lecture 59 : Team Decision Theory-VI
- Lecture 60 : Team Decision Theory-VII
Week 12
Syllabus and materials
At NPTEL, as the course publishes them.
Credits and licence
- Institution
- IIT Bombay
- Department
- Electrical Engineering
- Teacher
- Prof. Ankur A. Kulkarni
- Level
- Graduate
- Language
- English
- Subjects
- Engineering
- Source
- NPTEL
- Licence
- CC BY-NC-SA, as NPTEL states it. Non-commercial use only. NPTEL's terms
- Attribution
- NPTEL states: "Distributed under Creative Commons Attribution-ShareAlike / CC BY - NC - SA", beside a BY-NC-SA badge.
- Who can watch
- NPTEL gives no rating. Invisible Network shows it to Tweens profiles and older, judged from what the source says about it. How courses are placed