Learn · Introduction to Probability
Lecture 244 of 258Part III: Random Processes7 min
25.6 Periodic States
Prof. John Tsitsiklis and Prof. Patrick Jaillet · Massachusetts Institute of Technology
Credits and licence
- From
- Introduction to Probability, Massachusetts Institute of Technology (at MIT OpenCourseWare)
- Licence
- CC BY-NC-SA 4.0, as MIT OpenCourseWare states it. Non-commercial use only.
- Attribution
- Prof. John Tsitsiklis, Prof. Patrick Jaillet. Introduction to Probability. Spring 2018. Massachusetts Institute of Technology: MIT OpenCourseWare, https://ocw.mit.edu/. License: Creative Commons BY-NC-SA.
- Who can watch
- Teens profiles and older. How courses are placed
All the lectures
- 01.1 Lecture Overview
- 01.2 Sample Space
- 01.3 Sample Space Examples
- 01.4 Probability Axioms
- 01.5 Simple Properties of Probabilities
- 01.6 More Properties of Probabilities
- 01.7 A Discrete Example
- 01.8 A Continuous Example
- 01.9 Countable Additivity
- 01.10 Interpretations & Uses of Probabilities
- S01.0 Mathematical Background Overview
- S01.1 Sets
- S01.2 De Morgan's Laws
- S01.3 Sequences and their Limits
- S01.4 When Does a Sequence Converge?
- S01.5 Infinite Series
- S01.6 The Geometric Series
- S01.7 About the Order of Summation in Series with Multiple Indices
- S01.8 Countable and Uncountable Sets
- S01.9 Proof that a Set of Real Numbers is Uncountable
- S01.10 Bonferroni's Inequality
- 02.1 Lecture Overview
- 02.2 Conditional Probabilities
- 02.3 A Die Roll Example
- 02.4 Conditional Probabilities Obey the Same Axioms
- 02.5 A Radar Example and Three Basic Tools
- 02.6 The Multiplication Rule
- 02.7 Total Probability Theorem
- 02.8 Bayes' Rule
- 03.1 Lecture Overview
- 03.2 A Coin Tossing Example
- 03.3 Independence of Two Events
- 03.4 Independence of Event Complements
- 03.5 Conditional Independence
- 03.6 Independence Versus Conditional Independence
- 03.7 Independence of a Collection of Events
- 03.8 Independence Versus Pairwise Independence
- 03.9 Reliability
- 03.10 The King's Sibling
- 04.1 Lecture Overview
- 04.2 The Counting Principle
- 04.3 Die Roll Example
- 04.4 Combinations
- 04.5 Binomial Probabilities
- 04.6 A Coin Tossing Example
- 04.7 Partitions
- 04.8 Each Person Gets An Ace
- 04.9 Multinomial Probabilities
- 05.1 Lecture Overview
- 05.2 Definition of Random Variables
- 05.3 Probability Mass Functions
- 05.4 Bernoulli & Indicator Random Variables
- 05.5 Uniform Random Variables
- 05.6 Binomial Random Variables
- 05.7 Geometric Random Variables
- 05.8 Expectation
- 05.9 Elementary Properties of Expectation
- 05.10 The Expected Value Rule
- 05.11 Linearity of Expectations
- S05.1 Supplement: Functions
- 06.1 Lecture Overview
- 06.2 Variance
- 06.3 The Variance of the Bernoulli & the Uniform
- 06.4 Conditional PMFs & Expectations Given an Event
- 06.5 Total Expectation Theorem
- 06.6 Geometric PMF Memorylessness & Expectation
- 06.7 Joint PMFs and the Expected Value Rule
- 06.8 Linearity of Expectations & the Mean of the Binomial
- 07.1 Lecture Overview
- 07.2 Conditional PMFs
- 07.3 Conditional Expectation & the Total Expectation Theorem
- 07.4 Independence of Random Variables
- 07.5 Example
- 07.6 Independence & Expectations
- 07.7 Independence, Variances & the Binomial Variance
- 07.8 The Hat Problem
- S07.1 The Inclusion-Exclusion Formula
- 07.2 The Variance of the Geometric
- 07.3 Independence of Random Variables Versus Independence of Events
- 08.1 Lecture Overview
- 08.2 Probability Density Functions
- 08.4 Means & Variances
- 08.5 Mean & Variance of the Uniform
- 08.6 Exponential Random Variables
- 08.7 Cumulative Distribution Functions
- 08.8 Normal Random Variables
- 08.9 Calculation of Normal Probabilities
- 09.1 Lecture Overview
- 09.2 Conditioning A Continuous Random Variable on an Event
- 09.3 Conditioning Example
- 09.5 Total Probability & Expectation Theorems
- 09.6 Mixed Random Variables
- 09.8 From The Joint to the Marginal
- 09.9 Continuous Analogs of Various Properties
- 09.10 Joint CDFs
- S09.1 Buffon's Needle & Monte Carlo Simulation
- 10.1 Lecture Overview
- 18.10 Total Probability & Total Expectation Theorems
- 10.5 Independence
- 10.6 Stick-Breaking Example
- 10.7 Independent Normals
- 10.8 Bayes Rule Variations
- 10.9 Mixed Bayes Rule
- 10.10 Detection of a Binary Signal
- 10.11 Inference of the Bias of a Coin
- 11.1 Lecture Overview
- 11.2 The PMF of a Function of a Discrete Random Variable
- 11.3 A Linear Function of a Continuous Random Variable
- 11.4 A Linear Function of a Normal Random Variable
- 11.6 The Monotonic Case
- 11.7 The Intuition for the Monotonic Case
- 11.8 A Nonmonotonic Example
- S11.1 Simulation
- 12.1 Lecture Overview
- 12.2 The Sum of Independent Discrete Random Variables
- 12.3 The Sum of Independent Continuous Random Variables
- 12.4 The Sum of Independent Normal Random Variables
- 12.5 Covariance
- 12.6 Covariance Properties
- 12.7 The Variance of the Sum of Random Variables
- 12.8 The Correlation Coefficient
- 12.9 Proof of Key Properties of the Correlation Coefficient
- 12.10 Interpreting the Correlation Coefficient
- 12.11 Correlations Matter
- 13.1 Lecture Overview
- 13.2 Conditional Expectation as a Random Variable
- 13.3 The Law of Iterated Expectations
- 13.4 Stick-Breaking Revisited
- 13.5 Forecast Revisions
- 13.6 The Conditional Variance
- 13.7 Derivation of the Law of Total Variance
- 13.8 A Simple Example
- 13.9 Section Means and Variances
- 13.10 Mean of the Sum of a Random Number of Random Variables
- 13.11 Variance of the Sum of a Random Number of Random Variables
- 13.1 Conditional Expectation Properties
- 14.1 Lecture Overview
- 14.2 Overview of some Application Domains
- 14.3 Types of Inference Problems
- 14.4 The Bayesian Inference Framework
- 14.5 Discrete Parameter, Discrete Observation
- 14.6 Discrete Parameter, Continuous Observation
- 14.7 Continuous Parameter, Continuous Observation
- 14.8 Inferring the Unknown Bias of a Coin and the Beta Distribution
- 14.9 Inferring the Unknown Bias of a Coin—Point Estimates
- 14.10 Summary
- S14.1 The Beta Formula
- 15.1 Lecture Overview
- 15.3 Estimating a Normal Random Variable in the Presence of Additive Noise
- 15.4 The Case of Multiple Observations
- 15.5 The Mean Squared Error
- 15.6 Multiple Parameters; Trajectory Estimation
- 15.7 Linear Normal Models
- 15.8 Trajectory Estimation Illustration
- 16.1 Lecture Overview
- 16.2 LMS Estimation in the Absence of Observations
- 16.3 LMS Estimation of One Random Variable Based on Another
- 16.4 LMS Performance Evaluation
- 16.5 Example: The LMS Estimate
- 16.6 Example Continued: LMS Performance Evaluation
- 16.7 LMS Estimation with Multiple Observations or Unknowns
- 16.8 Properties of the LMS Estimation Error
- 17.1 Lecture Overview
- 18.2 LLMS Formulation
- 17.3 Solution to the LLMS Problem
- 17.4 Remarks on the LLMS Solution and on the Error Variance
- 17.5 LLMS Example
- 17.6 LLMS for Inferring the Parameter of a Coin
- 17.7 LLMS with Multiple Observations
- 17.8 The Simplest LLMS Example with Multiple Observations
- 17.9 The Representation of the Data Matters in LLMS
- 18.1 Lecture Overview
- 18.2 The Markov Inequality
- 18.3 The Chebyshev Inequality
- 18.4 The Weak Law of Large Numbers
- 18.5 Polling
- 18.6 Convergence in Probability
- 18.7 Convergence in Probability Examples
- 18.8 Related Topics
- 18.1 Convergence in Probability of the Sum of Two Random Variables
- S18.2 Jensen's Inequality
- S18.3 Hoeffding's Inequality
- 19.1 Lecture Overview
- 19.2 The Central Limit Theorem
- 19.3 Discussion of the CLT
- 19.4 Illustration of the CLT
- 19.5 CLT Examples
- 19.6 Normal Approximation to the Binomial
- 19.7 Polling Revisited
- 20.1 Lecture Overview
- 20.2 Overview of the Classical Statistical Framework
- 20.3 The Sample Mean and Some Terminology
- 20.4 On the Mean Squared Error of an Estimator
- 20.5 Confidence Intervals
- 20.6 Confidence Intervals for the Estimation of the Mean
- 20.7 Confidence Intervals for the Mean, When the Variance is Unknown
- 20.9 Other Natural Estimators
- 20.9 Maximum Likelihood Estimation
- 20.10 Maximum Likelihood Estimation Examples
- 21.1 Lecture Overview
- 21.2 The Bernoulli Process
- 21.3 Stochastic Processes
- 21.4 Review of Known Properties of the Bernoulli Process
- 21.5 The Fresh Start Property
- 21.6 Example: The Distribution of a Busy Period
- 21.7 The Time of the K-th Arrival
- 21.8 Merging of Bernoulli Processes
- 21.9 Splitting a Bernoulli Process
- 21.10 The Poisson Approximation to the Binomial
- 22.1 Lecture Overview
- 22.2 Definition of the Poisson Process
- 22.3 Applications of the Poisson Process
- 22.4 The Poisson PMF for the Number of Arrivals
- 22.5 The Mean and Variance of the Number of Arrivals
- 22.6 A Simple Example
- 22.7 Time of the K–th Arrival
- 22.8 The Fresh Start Property and Its Implications
- 22.9 Summary of Results
- 22.10 An Example
- 23.1 Lecture Overview
- 23.2 The Sum of Independent Poisson Random Variables
- 23.3 Merging Independent Poisson Processes
- 23.4 Where is an Arrival of the Merged Process Coming From?
- 23.5 The Time Until the First (or Last) Lightbulb Burns Out
- 23.6 Splitting a Poisson Process
- 23.7 Random Incidence in the Poisson Process
- 23.8 Random Incidence in a Non–Poisson Process
- 23.9 Different Sampling Methods Can Give Different Results
- S23.1 Poisson Versus Normal Approximations to the Binomial
- S23.2 Poisson Arrivals During an Exponential Interval
- 24.1 Lecture Overview
- 24.2 Introduction to Markov Processes
- 24.3 Checkout Counter Example
- 24.4 Discrete-Time Finite-State Markov Chains
- 24.5 N–Step Transition Probabilities
- 24.6 A Numerical Example — Part I
- 24.7 Generic Convergence Questions
- 24.8 Recurrent and Transient States
- Brief Introduction
- 25.2 Lecture Overview
- 25.3 Markov Chain Review
- 25.4 The Probability of a Path
- 25.5 Recurrent and Transient States: Review
- 25.6 Periodic States
- 25.7 Steady-State Probabilities and Convergence
- 25.8 A Numerical Example — Part II
- 25.9 Visit Frequency Interpretation of Steady-State Probabilities
- 25.10 Birth-Death Processes — Part I
- 25.11 Birth-Death Processes — Part II
- Brief Introduction
- 26.2 Lecture Overview
- 26.3 Review of Steady-State Behavior
- 24.6 A Numerical Example — Part III
- 26.6 Design of a Phone System
- 26.6 Absorption Probabilities
- 26.7 Expected Time to Absorption
- 26.8 Mean First Passage Time
- 26.9 Gambler's Ruin