CourseRES.6-012258 lectures29 h
Introduction to Probability
Prof. John Tsitsiklis and Prof. Patrick Jaillet · Massachusetts Institute of Technology
About
The tools of probability theory, and of the related field of statistical inference, are the keys for being able to analyze and make sense of data. These tools underlie important advances in many fields, from the basic sciences to engineering and management.
This resource is a companion site to 6.041SC Probabilistic Systems Analysis and Applied Probability. It covers the same content, using videos developed for an edX version of the course.
Lectures
Part I: The Fundamentals
- 01.1 Lecture Overview
- 01.2 Sample Space
- 01.3 Sample Space Examples
- 01.4 Probability Axioms
- 01.5 Simple Properties of Probabilities
- 01.6 More Properties of Probabilities
- 01.7 A Discrete Example
- 01.8 A Continuous Example
- 01.9 Countable Additivity
- 01.10 Interpretations & Uses of Probabilities
- S01.0 Mathematical Background Overview
- S01.1 Sets
- S01.2 De Morgan's Laws
- S01.3 Sequences and their Limits
- S01.4 When Does a Sequence Converge?
- S01.5 Infinite Series
- S01.6 The Geometric Series
- S01.7 About the Order of Summation in Series with Multiple Indices
- S01.8 Countable and Uncountable Sets
- S01.9 Proof that a Set of Real Numbers is Uncountable
- S01.10 Bonferroni's Inequality
- 02.1 Lecture Overview
- 02.2 Conditional Probabilities
- 02.3 A Die Roll Example
- 02.4 Conditional Probabilities Obey the Same Axioms
- 02.5 A Radar Example and Three Basic Tools
- 02.6 The Multiplication Rule
- 02.7 Total Probability Theorem
- 02.8 Bayes' Rule
- 03.1 Lecture Overview
- 03.2 A Coin Tossing Example
- 03.3 Independence of Two Events
- 03.4 Independence of Event Complements
- 03.5 Conditional Independence
- 03.6 Independence Versus Conditional Independence
- 03.7 Independence of a Collection of Events
- 03.8 Independence Versus Pairwise Independence
- 03.9 Reliability
- 03.10 The King's Sibling
- 04.1 Lecture Overview
- 04.2 The Counting Principle
- 04.3 Die Roll Example
- 04.4 Combinations
- 04.5 Binomial Probabilities
- 04.6 A Coin Tossing Example
- 04.7 Partitions
- 04.8 Each Person Gets An Ace
- 04.9 Multinomial Probabilities
- 05.1 Lecture Overview
- 05.2 Definition of Random Variables
- 05.3 Probability Mass Functions
- 05.4 Bernoulli & Indicator Random Variables
- 05.5 Uniform Random Variables
- 05.6 Binomial Random Variables
- 05.7 Geometric Random Variables
- 05.8 Expectation
- 05.9 Elementary Properties of Expectation
- 05.10 The Expected Value Rule
- 05.11 Linearity of Expectations
- S05.1 Supplement: Functions
- 06.1 Lecture Overview
- 06.2 Variance
- 06.3 The Variance of the Bernoulli & the Uniform
- 06.4 Conditional PMFs & Expectations Given an Event
- 06.5 Total Expectation Theorem
- 06.6 Geometric PMF Memorylessness & Expectation
- 06.7 Joint PMFs and the Expected Value Rule
- 06.8 Linearity of Expectations & the Mean of the Binomial
- 07.1 Lecture Overview
- 07.2 Conditional PMFs
- 07.3 Conditional Expectation & the Total Expectation Theorem
- 07.4 Independence of Random Variables
- 07.5 Example
- 07.6 Independence & Expectations
- 07.7 Independence, Variances & the Binomial Variance
- 07.8 The Hat Problem
- S07.1 The Inclusion-Exclusion Formula
- 07.2 The Variance of the Geometric
- 07.3 Independence of Random Variables Versus Independence of Events
- 08.1 Lecture Overview
- 08.2 Probability Density Functions
- 08.4 Means & Variances
- 08.5 Mean & Variance of the Uniform
- 08.6 Exponential Random Variables
- 08.7 Cumulative Distribution Functions
- 08.8 Normal Random Variables
- 08.9 Calculation of Normal Probabilities
- 09.1 Lecture Overview
- 09.2 Conditioning A Continuous Random Variable on an Event
- 09.3 Conditioning Example
- 09.5 Total Probability & Expectation Theorems
- 09.6 Mixed Random Variables
- 09.8 From The Joint to the Marginal
- 09.9 Continuous Analogs of Various Properties
- 09.10 Joint CDFs
- S09.1 Buffon's Needle & Monte Carlo Simulation
- 10.1 Lecture Overview
- 18.10 Total Probability & Total Expectation Theorems
- 10.5 Independence
- 10.6 Stick-Breaking Example
- 10.7 Independent Normals
- 10.8 Bayes Rule Variations
- 10.9 Mixed Bayes Rule
- 10.10 Detection of a Binary Signal
- 10.11 Inference of the Bias of a Coin
- 11.1 Lecture Overview
- 11.2 The PMF of a Function of a Discrete Random Variable
- 11.3 A Linear Function of a Continuous Random Variable
- 11.4 A Linear Function of a Normal Random Variable
- 11.6 The Monotonic Case
- 11.7 The Intuition for the Monotonic Case
- 11.8 A Nonmonotonic Example
- S11.1 Simulation
- 12.1 Lecture Overview
- 12.2 The Sum of Independent Discrete Random Variables
- 12.3 The Sum of Independent Continuous Random Variables
- 12.4 The Sum of Independent Normal Random Variables
- 12.5 Covariance
- 12.6 Covariance Properties
- 12.7 The Variance of the Sum of Random Variables
- 12.8 The Correlation Coefficient
- 12.9 Proof of Key Properties of the Correlation Coefficient
- 12.10 Interpreting the Correlation Coefficient
- 12.11 Correlations Matter
- 13.1 Lecture Overview
- 13.2 Conditional Expectation as a Random Variable
- 13.3 The Law of Iterated Expectations
- 13.4 Stick-Breaking Revisited
- 13.5 Forecast Revisions
- 13.6 The Conditional Variance
- 13.7 Derivation of the Law of Total Variance
- 13.8 A Simple Example
- 13.9 Section Means and Variances
- 13.10 Mean of the Sum of a Random Number of Random Variables
- 13.11 Variance of the Sum of a Random Number of Random Variables
- 13.1 Conditional Expectation Properties
Part II: Inference & Limit Theorems
- 14.1 Lecture Overview
- 14.2 Overview of some Application Domains
- 14.3 Types of Inference Problems
- 14.4 The Bayesian Inference Framework
- 14.5 Discrete Parameter, Discrete Observation
- 14.6 Discrete Parameter, Continuous Observation
- 14.7 Continuous Parameter, Continuous Observation
- 14.8 Inferring the Unknown Bias of a Coin and the Beta Distribution
- 14.9 Inferring the Unknown Bias of a Coin—Point Estimates
- 14.10 Summary
- S14.1 The Beta Formula
- 15.1 Lecture Overview
- 15.3 Estimating a Normal Random Variable in the Presence of Additive Noise
- 15.4 The Case of Multiple Observations
- 15.5 The Mean Squared Error
- 15.6 Multiple Parameters; Trajectory Estimation
- 15.7 Linear Normal Models
- 15.8 Trajectory Estimation Illustration
- 16.1 Lecture Overview
- 16.2 LMS Estimation in the Absence of Observations
- 16.3 LMS Estimation of One Random Variable Based on Another
- 16.4 LMS Performance Evaluation
- 16.5 Example: The LMS Estimate
- 16.6 Example Continued: LMS Performance Evaluation
- 16.7 LMS Estimation with Multiple Observations or Unknowns
- 16.8 Properties of the LMS Estimation Error
- 17.1 Lecture Overview
- 18.2 LLMS Formulation
- 17.3 Solution to the LLMS Problem
- 17.4 Remarks on the LLMS Solution and on the Error Variance
- 17.5 LLMS Example
- 17.6 LLMS for Inferring the Parameter of a Coin
- 17.7 LLMS with Multiple Observations
- 17.8 The Simplest LLMS Example with Multiple Observations
- 17.9 The Representation of the Data Matters in LLMS
- 18.1 Lecture Overview
- 18.2 The Markov Inequality
- 18.3 The Chebyshev Inequality
- 18.4 The Weak Law of Large Numbers
- 18.5 Polling
- 18.6 Convergence in Probability
- 18.7 Convergence in Probability Examples
- 18.8 Related Topics
- 18.1 Convergence in Probability of the Sum of Two Random Variables
- S18.2 Jensen's Inequality
- S18.3 Hoeffding's Inequality
- 19.1 Lecture Overview
- 19.2 The Central Limit Theorem
- 19.3 Discussion of the CLT
- 19.4 Illustration of the CLT
- 19.5 CLT Examples
- 19.6 Normal Approximation to the Binomial
- 19.7 Polling Revisited
- 20.1 Lecture Overview
- 20.2 Overview of the Classical Statistical Framework
- 20.3 The Sample Mean and Some Terminology
- 20.4 On the Mean Squared Error of an Estimator
- 20.5 Confidence Intervals
- 20.6 Confidence Intervals for the Estimation of the Mean
- 20.7 Confidence Intervals for the Mean, When the Variance is Unknown
- 20.9 Other Natural Estimators
- 20.9 Maximum Likelihood Estimation
- 20.10 Maximum Likelihood Estimation Examples
Part III: Random Processes
- 21.1 Lecture Overview
- 21.2 The Bernoulli Process
- 21.3 Stochastic Processes
- 21.4 Review of Known Properties of the Bernoulli Process
- 21.5 The Fresh Start Property
- 21.6 Example: The Distribution of a Busy Period
- 21.7 The Time of the K-th Arrival
- 21.8 Merging of Bernoulli Processes
- 21.9 Splitting a Bernoulli Process
- 21.10 The Poisson Approximation to the Binomial
- 22.1 Lecture Overview
- 22.2 Definition of the Poisson Process
- 22.3 Applications of the Poisson Process
- 22.4 The Poisson PMF for the Number of Arrivals
- 22.5 The Mean and Variance of the Number of Arrivals
- 22.6 A Simple Example
- 22.7 Time of the K–th Arrival
- 22.8 The Fresh Start Property and Its Implications
- 22.9 Summary of Results
- 22.10 An Example
- 23.1 Lecture Overview
- 23.2 The Sum of Independent Poisson Random Variables
- 23.3 Merging Independent Poisson Processes
- 23.4 Where is an Arrival of the Merged Process Coming From?
- 23.5 The Time Until the First (or Last) Lightbulb Burns Out
- 23.6 Splitting a Poisson Process
- 23.7 Random Incidence in the Poisson Process
- 23.8 Random Incidence in a Non–Poisson Process
- 23.9 Different Sampling Methods Can Give Different Results
- S23.1 Poisson Versus Normal Approximations to the Binomial
- S23.2 Poisson Arrivals During an Exponential Interval
- 24.1 Lecture Overview
- 24.2 Introduction to Markov Processes
- 24.3 Checkout Counter Example
- 24.4 Discrete-Time Finite-State Markov Chains
- 24.5 N–Step Transition Probabilities
- 24.6 A Numerical Example — Part I
- 24.7 Generic Convergence Questions
- 24.8 Recurrent and Transient States
- Brief Introduction
- 25.2 Lecture Overview
- 25.3 Markov Chain Review
- 25.4 The Probability of a Path
- 25.5 Recurrent and Transient States: Review
- 25.6 Periodic States
- 25.7 Steady-State Probabilities and Convergence
- 25.8 A Numerical Example — Part II
- 25.9 Visit Frequency Interpretation of Steady-State Probabilities
- 25.10 Birth-Death Processes — Part I
- 25.11 Birth-Death Processes — Part II
- Brief Introduction
- 26.2 Lecture Overview
- 26.3 Review of Steady-State Behavior
- 24.6 A Numerical Example — Part III
- 26.6 Design of a Phone System
- 26.6 Absorption Probabilities
- 26.7 Expected Time to Absorption
- 26.8 Mean First Passage Time
- 26.9 Gambler's Ruin
Syllabus and materials
At MIT OpenCourseWare, as the course publishes them.
Credits and licence
- Institution
- Massachusetts Institute of Technology
- Department
- Electrical Engineering and Computer Science
- Taught by
- Prof. John Tsitsiklis, Prof. Patrick Jaillet
- Term
- Spring 2018
- Level
- Undergraduate
- Language
- English
- Subjects
- Mathematics, Computer science, Engineering
- Source
- MIT OpenCourseWare
- Licence
- CC BY-NC-SA 4.0, as MIT OpenCourseWare states it. Non-commercial use only. MIT OpenCourseWare's terms
- Attribution
- Prof. John Tsitsiklis, Prof. Patrick Jaillet. Introduction to Probability. Spring 2018. Massachusetts Institute of Technology: MIT OpenCourseWare, https://ocw.mit.edu/. License: Creative Commons BY-NC-SA.
- Who can watch
- MIT OpenCourseWare gives no rating. Invisible Network shows it to Teens profiles and older, judged from what the source says about it. How courses are placed
More like this
Course
The Analytics Edge
35 lectures · Video
Course
Single Variable Calculus
35 lectures · Video
Course
Multivariable Calculus
35 lectures · Video
Course
Matrix Methods in Data Analysis, Signal Processing, and Machine Learning
34 lectures · Video
Course
Linear Algebra
35 lectures · Video
Course
Linear Algebra
158 lectures · Video
Course
Computational Science and Engineering I
50 lectures · Video
Course
Engineering Dynamics
223 lectures · Video
Course
Becoming the Next Bill Nye: Writing and Hosting the Educational Show
132 lectures · Video
Course
Introduction to Nuclear Engineering and Ionizing Radiation
34 lectures · Video
Course
Thermodynamics of Materials
38 lectures · Video
Course
Introduction to Solid-State Chemistry
38 lectures · Video