Chapter 7 Decomposition of States, Branching Process (Lecture on 01/26/2021)

\(i\leftrightarrow j\) if \(p_{ij}(m)>0\) for some \(m\) and \(p_{ji}(n)>0\) for some \(n\).

Definition 7.1 (Close and Irreducible) A set \(C\) of states is called

  1. Closed, if \(p_{ij}=0\) for all \(i\in C\) and \(j\notin C\).

  2. Irreducible, if \(i\leftrightarrow j\) for all \(i,j\in C\).

Condition (a) says that once a chain takes a value in a closed set \(C\) of states, then it never leaves \(C\) subsequently.

Regarding condition (b), we have already proved that for intercommunicating states, we just need to study properties of one of the states, and all the other states in \(C\) will have the same properties.

Definition 7.2 (Absorbing State) A closed set containing exactly one state is called absorbing.

Theorem 7.1 (Decomposition Theorem) The state space \(S\) can be partitioned uniquely as \(S=T\cup C_1\cup C_2\cup\cdots\), where \(T\) is the set of transient states and \(C_i\) irreducible closed set of persistent states.

Example 7.1 (Partition of States) Let \(S=\{1,2,3,4,5,6\}\) and the transition probability matrix is given by \[\begin{equation} \begin{pmatrix} \frac{1}{2} & \frac{1}{2} & 0 & 0 & 0 & 0 \\ \frac{1}{4} & \frac{3}{4} & 0 & 0 & 0 & 0 \\ \frac{1}{4} & \frac{1}{4} & \frac{1}{4} & \frac{1}{4} & 0 & 0 \\ \frac{1}{4} & 0 & \frac{1}{4} & \frac{1}{4} & 0 & \frac{1}{4}\\ 0 & 0 & 0 & 0 & \frac{1}{2} & \frac{1}{2} \\ 0 & 0 & 0 & 0 & \frac{1}{2} & \frac{1}{2} \end{pmatrix} \tag{7.1} \end{equation}\] We will study properties of each of these states.

Let \(C_1=\{1,2\}\), then \(C_1\) is a closed and irreducible set. Then let \(C_2=\{5,6\}\), then \(C_2\) is also a closed and irreducible set.

Question: Properties of the states in \(C_1\), are they null recurrent? Whar are their period? etc.

Let us investigate state 1, we have \(f_{11}(n)=\left\{\begin{aligned} & \frac{1}{2} & n=1\\ &\frac{1}{2}(\frac{3}{4})^{n-2}\frac{1}{4} & n\geq 2\end{aligned}\right.\) Therefore, \[\begin{equation} \begin{split} f_{11}&=\sum_{n=1}^{\infty}f_{11}(n)\\ &=\frac{1}{2}+\frac{1}{2}\sum_{n=2}^{\infty}(\frac{3}{4})^{n-2}\frac{1}{4}\quad (\text{geometric sum})\\ &=\frac{1}{2}+\frac{1}{2}\cdot\frac{1}{4}\cdot\frac{1}{1-\frac{3}{4}}=1 \end{split} \tag{7.2} \end{equation}\] Therefore, state 1 is persistent.

To see if state 1 is null persistent, we need to compute \[\begin{equation} \begin{split} \mu_1&=\sum_{n=1}^{\infty}nf_{11}(n)\\ &=1\cdot\frac{1}{2}+\frac{1}{2}\frac{1}{4}\sum_{n=2}^{\infty} n(\frac{3}{4})^{n-2}\\ &=1\cdot\frac{1}{2}+\frac{1}{2}\cdot\frac{1}{4}\cdot 20 =3 \end{split} \tag{7.3} \end{equation}\] Therefore, state 1 is non-null persistent.

Note also that \(p_{11}(1)>0\), \(p_{11}(2)>0\) and \(gcd{m:p_{11}(m)>0}=1\). Thus, state 1 is an aperiodic, non-null persistent state.

Now note that state 2 is intercommunicate with state 1, by Theorem 6.2, state 2 is also anaperiodic, non-null persistent state.

Using similar calculations, we will be able to show that states in \(C_2=\{5,6\}\) are aperiodic and non-null persistent.

Now consider state 3 and 4. One can travel from state \(3\to 1\), but cannot come back. Therefore, the probability of eventual coming back is less than 1, i.e. \(f_{33}<1\). Therefore, state 3 is a transient state. Similarly, for state 4, one can go \(4\to 6\) but cannot come back. State 4 is also transient because \(f_{44}<1\).
Lemma 7.1 If \(S\) is finite, then at least one state is persistent and all persistent states are non-null.
This is a very powerful result. Because if from the transition matrix we see that all states are intercommunicating, then invoking this lemma, we can say that all states are non-null persistent.

Branching Process

In 18th century, British royal families were concerned about the family name. They wanted to know

  1. What will be the expected number of people carrying their family names in different generations?

  2. What is the probabilities that their family will be extinct?

Francis Galton with the help of Watson solved this problem, which famously designated as the Galton-Watson branching process.

Suppose each individual has a probability \(p_k\) of having \(k\) sons at any generation. Assume that the initial population starts with one individual, i.e. \(X_0=1\). Let \(X_{n+1}\) denote the number of sons in the family at the \((n+1)\)th generation. We define \(\xi_r\) as the number of sons for the \(r\)th individual, then we can write \(X_{n+1}=\sum_{r=1}^{X_n}\xi_r\) where \(P(\xi_r=k)=p_k\) for \(k=0,1,2,\cdots\). We want to answer \(E(X_n)\) and \(P(X_n=0,\exists n)\).

Let us define a generating function \[\begin{equation} \varphi(s)=\sum_{k=0}^{\infty}p_ks^k \tag{7.4} \end{equation}\] and \[\begin{equation} \varphi_n(s)=\sum_{k=0}^{\infty}P(X_n=k)s^k \tag{7.5} \end{equation}\] for \(n=0,1,2,\cdots\) and \(s\) is a variable taking values in \((0,1]\).

We have \[\begin{equation} \begin{split} \varphi_{n+1}(s)&=\sum_{k=0}^{\infty}P(X_{n+1}=k)s^k\\ &=\sum_{k=0}^{\infty}(\sum_{j=0}^{\infty}P(X_{n+1}=k|X_n=j)P(X_n=j))s^k\\ &=\sum_{j=0}^{\infty}P(x_n=j)[\sum_{k=0}^{\infty}P(\xi_1+\cdots+\xi_j=k)s^k] \end{split} \tag{7.6} \end{equation}\]

We claim that \[\begin{equation} \sum_{k=0}^{\infty}P(\xi_1+\cdots+\xi_j=k)s^k=[\varphi(s)]^j \tag{7.7} \end{equation}\] This is because \([\varphi(s)]^j=[\sum_{k=0}^{\infty}p_ks^k]^j\) and \[\begin{equation} \begin{split} P(\xi_1+\cdots+\xi_j=k)&=\sum_{h_1=0}^k\cdots\sum_{h_j=0}^kP(\xi_1=h_1)\cdots P(\xi_j=h_j)\quad (\text{subject to}\, h_1+\cdots+h_j=k)\\ &=\sum_{h_1=0}^k\cdots\sum_{h_j=0}^kp_{h_1}\cdots p_{h_j}\quad (\text{subject to}\, h_1+\cdots+h_j=k) \end{split} \end{equation}\] Look at the coefficient of \(s^k\) from \([\varphi(s)]^j=[\sum_{k=0}^{\infty}p_ks^k]^j\), it is exaactly \(\sum_{h_1=0}^k\cdots\sum_{h_j=0}^kp_{h_1}\cdots p_{h_j}\quad (\text{subject to}\, h_1+\cdots+h_j=k)\). Therefore, we have the claim \(\sum_{k=0}^{\infty}P(\xi_1+\cdots+\xi_j=k)s^k=[\varphi(s)]^j\).

From (7.6) and (7.7) we have \[\begin{equation} \varphi_{n+1}(s)=\sum_{j=0}^{\infty}P(x_n=j)[\varphi(s)]^j=\varphi_{n}(\varphi(s)) \tag{7.8} \end{equation}\] Iterating this relationship, we obtain \[\begin{equation} \varphi_{n+1}(s)=\varphi_{n}(\varphi(s))=\varphi_{n-1}(\varphi(\varphi(s)))=\cdots \tag{7.9} \end{equation}\] Generally, we obtain that \(\varphi_{n+1}(s)=\varphi_{n-k}(\varphi_{k+1}(s)))\) for any \(k=0,\cdots,n\). In particular, \[\begin{equation} \varphi_{n+1}(s)=\varphi(\varphi_n(s)) \tag{7.10} \end{equation}\]