# Information Theory

- Institution: IISc Bangalore
- Department: Electrical Engineering
- Teacher: Prof. Himanshu Tyagi
- Level: Undergraduate
- Language: English
- Subjects: Engineering
- Source: [NPTEL](https://nptel.ac.in/courses/108108168)
- Licence: [CC BY-NC-SA](https://nptel.ac.in/faq) (Non-commercial use only.)
- Syllabus: [at the source](https://nptel.ac.in/courses/108108168)
- Who can watch: Tweens profiles and older (sources give no ratings; placed by Invisible Network from the source's own words)

## Lectures

1. [What is information?](https://invisiblenetwork.ai/learn/nptel/108108168/8fa951ccece1) (Video)
2. [How to model uncertainty?](https://invisiblenetwork.ai/learn/nptel/108108168/1b9777000312) (Video)
3. [Basic concepts of probability](https://invisiblenetwork.ai/learn/nptel/108108168/808a86bd684a) (Video)
4. [Estimates of random variables](https://invisiblenetwork.ai/learn/nptel/108108168/c1d103b8f511) (Video)
5. [Limit theorems](https://invisiblenetwork.ai/learn/nptel/108108168/3c6afdfd60fa) (Video)
6. [Review](https://invisiblenetwork.ai/learn/nptel/108108168/e7afcc09c906) (Video)
7. [Source model](https://invisiblenetwork.ai/learn/nptel/108108168/f47afb69bf72) (Video)
8. [Motivating examples](https://invisiblenetwork.ai/learn/nptel/108108168/9e0fdd4b4a2e) (Video)
9. [A compression problem](https://invisiblenetwork.ai/learn/nptel/108108168/a7170d4078f1) (Video)
10. [Shannon entropy](https://invisiblenetwork.ai/learn/nptel/108108168/bcf81c693824) (Video)
11. [Random hash](https://invisiblenetwork.ai/learn/nptel/108108168/abb2db46ab6a) (Video)
12. [Review 2](https://invisiblenetwork.ai/learn/nptel/108108168/388e0f250188) (Video)
13. [Uncertainty and randomness](https://invisiblenetwork.ai/learn/nptel/108108168/98f245f67bec) (Video)
14. [Total variation distance](https://invisiblenetwork.ai/learn/nptel/108108168/b965cb866234) (Video)
15. [Generating almost random bits](https://invisiblenetwork.ai/learn/nptel/108108168/0fa3589e1517) (Video)
16. [Generating samples from a distribution using uniform randomness](https://invisiblenetwork.ai/learn/nptel/108108168/712267407fa1) (Video)
17. [Typical sets and entropy](https://invisiblenetwork.ai/learn/nptel/108108168/c04a9e0b7275) (Video)
18. [Review 3](https://invisiblenetwork.ai/learn/nptel/108108168/eaa663592bc6) (Video)
19. [Hypothesis testing and estimation](https://invisiblenetwork.ai/learn/nptel/108108168/fc7ba7226dd0) (Video)
20. [Examples.](https://invisiblenetwork.ai/learn/nptel/108108168/129d3a92f346) (Video)
21. [The log-likelihood ratio test](https://invisiblenetwork.ai/learn/nptel/108108168/849f59038a2f) (Video)
22. [Kullback-Leibler divergence and Stein's lemma](https://invisiblenetwork.ai/learn/nptel/108108168/8ca8d2acddbb) (Video)
23. [Properties of KL divergence](https://invisiblenetwork.ai/learn/nptel/108108168/c0f159f59f52) (Video)
24. [Review 4](https://invisiblenetwork.ai/learn/nptel/108108168/4960aac1495d) (Video)
25. [Information per coin-toss](https://invisiblenetwork.ai/learn/nptel/108108168/2053f92c63e4) (Video)
26. [Multiple hypothesis testing](https://invisiblenetwork.ai/learn/nptel/108108168/ac6ddc6fb826) (Video)
27. [Error analysis of multiple hypothesis testing](https://invisiblenetwork.ai/learn/nptel/108108168/0548f2bf6922) (Video)
28. [Mutual information](https://invisiblenetwork.ai/learn/nptel/108108168/4cde961da6b9) (Video)
29. [Fano's inequality](https://invisiblenetwork.ai/learn/nptel/108108168/e332103e8d6d) (Video)
30. [Measures of information](https://invisiblenetwork.ai/learn/nptel/108108168/0054e28c9792) (Video)
31. [Chain rules](https://invisiblenetwork.ai/learn/nptel/108108168/33804c056c10) (Video)
32. [Chain rules](https://invisiblenetwork.ai/learn/nptel/108108168/33804c056c10-2) (Video)
33. [Shape of measures of information](https://invisiblenetwork.ai/learn/nptel/108108168/0038485595c9) (Video)
34. [Data processing inequality](https://invisiblenetwork.ai/learn/nptel/108108168/eca78688190f) (Video)
35. [Midyear Review](https://invisiblenetwork.ai/learn/nptel/108108168/a10f20e6ccfd) (Video)
36. [Proof of Fano's inequality](https://invisiblenetwork.ai/learn/nptel/108108168/e612a6915b7d) (Video)
37. [Capacity as information radius](https://invisiblenetwork.ai/learn/nptel/108108168/9a2d54f7c4a0) (Video)
38. [Proof of Pinsker's inequality](https://invisiblenetwork.ai/learn/nptel/108108168/06fc79869734) (Video)
39. [Continuity of entropy](https://invisiblenetwork.ai/learn/nptel/108108168/f5325c830b0a) (Video)
40. [Lower bound for compression](https://invisiblenetwork.ai/learn/nptel/108108168/619f23d528e3) (Video)
41. [Lower bound for hypothesis testing](https://invisiblenetwork.ai/learn/nptel/108108168/15449c61412d) (Video)
42. [Review 7](https://invisiblenetwork.ai/learn/nptel/108108168/11f5a9484671) (Video)
43. [Lower bound for random number generation](https://invisiblenetwork.ai/learn/nptel/108108168/e82871280f8f) (Video)
44. [Strong converse](https://invisiblenetwork.ai/learn/nptel/108108168/550ed8fdeb33) (Video)
45. [Lower bound for minmax statistical estimation](https://invisiblenetwork.ai/learn/nptel/108108168/9583c1c33be1) (Video)
46. [Variable length source codes](https://invisiblenetwork.ai/learn/nptel/108108168/218be30a1a87) (Video)
47. [Review 8](https://invisiblenetwork.ai/learn/nptel/108108168/e9efc16303e4) (Video)
48. [Kraft's inequality](https://invisiblenetwork.ai/learn/nptel/108108168/0aba90aeb55a) (Video)
49. [Shannon code](https://invisiblenetwork.ai/learn/nptel/108108168/c14c783fc6e9) (Video)
50. [Huffman code](https://invisiblenetwork.ai/learn/nptel/108108168/9d27f4cfb21c) (Video)
51. [Minmax Redundancy](https://invisiblenetwork.ai/learn/nptel/108108168/8cded919763c) (Video)
52. [Type based universal compression](https://invisiblenetwork.ai/learn/nptel/108108168/d79e0ffdd0d1) (Video)
53. [Review 9](https://invisiblenetwork.ai/learn/nptel/108108168/cae4f742823c) (Video)
54. [Arithmetic code](https://invisiblenetwork.ai/learn/nptel/108108168/761ab5b75c03) (Video)
55. [Online probability assignment](https://invisiblenetwork.ai/learn/nptel/108108168/7a3eb8721870) (Video)
56. [Compression of databases: A scheme](https://invisiblenetwork.ai/learn/nptel/108108168/58e5a4976e02) (Video)
57. [Compression of databases: A lower bound](https://invisiblenetwork.ai/learn/nptel/108108168/6cdb6b98a8e3) (Video)
58. [Repetition code](https://invisiblenetwork.ai/learn/nptel/108108168/3145a3014446) (Video)
59. [Channel capacity](https://invisiblenetwork.ai/learn/nptel/108108168/c457840ec95a) (Video)
60. [Sphere packing bound for BSC](https://invisiblenetwork.ai/learn/nptel/108108168/3048bb418535) (Video)
61. [Random coding bound for BSC](https://invisiblenetwork.ai/learn/nptel/108108168/f481144a7774) (Video)
62. [Random coding bound for general channel](https://invisiblenetwork.ai/learn/nptel/108108168/8550165b9821) (Video)
63. [Review 11](https://invisiblenetwork.ai/learn/nptel/108108168/4bb51c44f70a) (Video)
64. [Converse proof for channel coding theorem](https://invisiblenetwork.ai/learn/nptel/108108168/d2082f754210) (Video)
65. [Additive Gaussian Noise channel](https://invisiblenetwork.ai/learn/nptel/108108168/4ad027429092) (Video)
66. [Mutual information and differential entropy](https://invisiblenetwork.ai/learn/nptel/108108168/8c3b623b2854) (Video)
67. [Channel coding theorem for Gaussan channel](https://invisiblenetwork.ai/learn/nptel/108108168/cebba6120bc2) (Video)
68. [Parallel channels and water-filling](https://invisiblenetwork.ai/learn/nptel/108108168/9c04c969a14d) (Video)
69. [Live Session 23-09-2020](https://invisiblenetwork.ai/learn/nptel/108108168/8fa951ccece1-2) (Video)
70. [Live Session 07-10-2020](https://invisiblenetwork.ai/learn/nptel/108108168/1b9777000312-2) (Video)