1.2: Master Equations

Motivation and Derivation

The techniques developed in the basic theory of Markov processes are widely applicable, but there are of course many instances in which the discretization of time is either inconvenient or completely unphysical. In such instances, a master equation (more humbly referred to as a rate equation) may provide a continuous-time description of the system that is in keeping with all of our results about stochastic processes.

Eq. \((1.22)\) is a master equation. As the derivation suggests, \(\mathbf{W}\) plays the role of a transition probability matrix in this formulation. You may notice that the master equation looks structurally very similar to rate equations in elementary kinetics; in fact, the master equation is a generalization of such rate equations, and the derivation above provides some formal justification for the rules we learn in kinetics for writing them down. The matrix \(\mathbf{W}\) is analogous to the set of rate constants indicating the relative rates of reaction between species in the system, and the probabilities \(P_{n}\) are analogous to the relative concentrations of these species.

Example: Consider a random walk on a one-dimensional infinite lattice (see Figure 1.4). As indicated in the figure, the transition probability between a lattice point and either adjacent lattice point is \(k\), and all other transition probabilities are zero (in other words, the system cannot "hop" over a lattice point without first occupying it). We can write down a master equation to describe the flow of probability among the lattice sites in a manner analogous to writing down a rate law. For any given site \(n\) on the lattice, probability can flow into \(n\) from either site \(n-1\) or site \(n+1\), and both of these occur at rate \(k\); likewise, probability can flow out of state \(n\) to either site \(n+1\) or site \(n-1\), both of which also happen at rate \(k\). Hence, the master equation for all sites \(n\) on the lattice is

\[\dot{P}_{n}=k\left(P_{n+1}+P_{n-1}-2 P_{n}\right)\]

Now we define the average site of occupation as a sum over all sites, weighted by the probability of occupation at each site,

\[\bar{n}=\sum_{n=-\infty}^{\infty} n P_{n}(t)\]

Then we can compute, for example, how this average site evolves with time,

\[\sum_{n=-\infty}^{\infty} n \dot{P}_{n}(t)=\dot{\bar{n}}=0\]

Hence the average site of occupation does not change over time in this model, so if we choose the initial distribution to satisfy \(\bar{n}=0\), then this will always be the average site of occupation.

However, the mean square displacement \(\bar{n}^{2}\) is not constant; in keeping with our physical interpretation of the model, the mean square displacement increases with time. In particular,

\[\sum_{n=-\infty}^{\infty} n^{2} \dot{P}_{n}(t)=\dot{\bar{n}}^{2}=2 k\]

If the initial probability distribution is a delta function on site \(0, P_{n}(0)=\delta_{0}\), then it turns out that Fourier analysis provides a route towards a closed-form expression for the long-time limit of \(P_{n}(t)\) :

\[\begin{aligned} P_{n}(t)=\frac{1}{2 \pi} \int_{0}^{2 \pi} e^{i n z} e^{-2 k(1-\cos z) t} d z \\ \lim _{t \rightarrow \infty} P_{n}(t)=\frac{1}{2 \pi} \int_{-\infty}^{\infty} e^{i n z} e^{-2 D\left(\frac{z^{2}}{2}\right) t} d z \end{aligned}\]

\[\lim _{t \rightarrow \infty} P_{n}(t)=\sqrt{\frac{1}{4 \pi D t}} e^{-\frac{n^{2}}{4 D t}}\]

In the above manipulations, we have replaced \(k\) with the diffusion constant \(D\), the long-time limit of the rate constant (in this case, the two are identical). Thus the probability distribution for occupying the various sites becomes Gaussian at long times.

Mean First Passage Time

One of the most useful quantities we can determine from the master equation for a random walk is the average time it takes for the random walk to reach a particular site \(n_{s}\) for the first time. This quantity, called the mean first passage time, can be determined via the following trick: we place an absorbing boundary condition at \(n_{s}, P_{n_{s}}(t)=0\). Whenever the walk reaches site \(n_{s}\), it stays there for all later times. One then calculates the survival probability \(S(t)\), that is, the probability that the walker has not yet visited \(n_{s}\) at time \(t\),

\[S(t)=\sum_{n \neq n_{s}} P_{n}(t)\]

The mean first passage time \(\langle t\rangle\) then corresponds to the time-averaged survival probability,

\[\langle t\rangle=\int_{0}^{\infty} S(t) d t\]

Sometimes it is more convenient to write the mean first passage time in terms of the probability density of reaching site \(n_{s}\) at time \(t\). This quantity is denoted by \(f(t)\) and satisfies

\[f(t)=-\frac{d S(t)}{d t}=\sum_{n \neq n_{s}} P_{n}(t) W_{n n_{s}}\]

In terms of \(f(t)\), the mean first passage time is given by

\[\langle t\rangle=\int_{0}^{\infty} t f(t) d t\]

The mean first passage time is a quantity of interest in a number of current research applications. Rates of fluorescence quenching, electron transfer, and exciton quenching can all be formulated in terms of the mean first passage time of a stochastic process.

Example: Let’s calculate the mean first passage time of the three-site model introduced in Figure \(1.1\), with all transition rates having the same value \(k\). Suppose the system is prepared in state 1 , and we’re interested in knowing the mean first passage time for site 3 . Applying the absorbing boundary condition at site 3 , we derive the following master equations:

\[\left\{\begin{array}{l} \dot{P}_{1}=-2 k P_{1}+k P_{2} \\ \dot{P}_{2}=k P_{1}-2 k P_{2} \\ \dot{P}_{3}=k P_{1}+k P_{2} \end{array}\right.\]

The transition matrix \(\mathbf{W}\) corresponding to this system would have a zero column since \(P_{3}\) does not occur on the right hand side of any of these equations; hence the sink leads to a zero eigenvalue that we can ignore. The relevant submatrix

\[\mathbf{W}_{1,2}=\left(\begin{array}{cc} -2 k & k \\ k & -2 k \end{array}\right)\]

has eigenvalues \(\lambda_{1}=-k, \lambda_{2}=-3 k\). Using the spectral decomposition formula, we find that the survival probability is

\[S(t)=\sum_{i, n}\left\langle 1 \mid \psi_{i}\right\rangle e^{\lambda_{i} t}\left\langle\psi_{i} \mid n\right\rangle=e^{-k t}\]

Hence, the previously defined probability density \(f(t)\) is given by \(f(t)=k e^{-k t}\), and the mean first passage time for site 3 is

\[\langle t\rangle=\frac{1}{k}\]