Chapter 7 Decomposition of States, Branching Process (Lecture on 01/26/2021)
\(i\leftrightarrow j\) if \(p_{ij}(m)>0\) for some \(m\) and \(p_{ji}(n)>0\) for some \(n\).
Definition 7.1 (Close and Irreducible) A set \(C\) of states is called
Closed, if \(p_{ij}=0\) for all \(i\in C\) and \(j\notin C\).
- Irreducible, if \(i\leftrightarrow j\) for all \(i,j\in C\).
Condition (a) says that once a chain takes a value in a closed set \(C\) of states, then it never leaves \(C\) subsequently.
Regarding condition (b), we have already proved that for intercommunicating states, we just need to study properties of one of the states, and all the other states in \(C\) will have the same properties.
Theorem 7.1 (Decomposition Theorem) The state space \(S\) can be partitioned uniquely as \(S=T\cup C_1\cup C_2\cup\cdots\), where \(T\) is the set of transient states and \(C_i\) irreducible closed set of persistent states.
Example 7.1 (Partition of States) Let \(S=\{1,2,3,4,5,6\}\) and the transition probability matrix is given by \[\begin{equation} \begin{pmatrix} \frac{1}{2} & \frac{1}{2} & 0 & 0 & 0 & 0 \\ \frac{1}{4} & \frac{3}{4} & 0 & 0 & 0 & 0 \\ \frac{1}{4} & \frac{1}{4} & \frac{1}{4} & \frac{1}{4} & 0 & 0 \\ \frac{1}{4} & 0 & \frac{1}{4} & \frac{1}{4} & 0 & \frac{1}{4}\\ 0 & 0 & 0 & 0 & \frac{1}{2} & \frac{1}{2} \\ 0 & 0 & 0 & 0 & \frac{1}{2} & \frac{1}{2} \end{pmatrix} \tag{7.1} \end{equation}\] We will study properties of each of these states.
Let \(C_1=\{1,2\}\), then \(C_1\) is a closed and irreducible set. Then let \(C_2=\{5,6\}\), then \(C_2\) is also a closed and irreducible set.
Question: Properties of the states in \(C_1\), are they null recurrent? Whar are their period? etc.
Let us investigate state 1, we have \(f_{11}(n)=\left\{\begin{aligned} & \frac{1}{2} & n=1\\ &\frac{1}{2}(\frac{3}{4})^{n-2}\frac{1}{4} & n\geq 2\end{aligned}\right.\) Therefore, \[\begin{equation} \begin{split} f_{11}&=\sum_{n=1}^{\infty}f_{11}(n)\\ &=\frac{1}{2}+\frac{1}{2}\sum_{n=2}^{\infty}(\frac{3}{4})^{n-2}\frac{1}{4}\quad (\text{geometric sum})\\ &=\frac{1}{2}+\frac{1}{2}\cdot\frac{1}{4}\cdot\frac{1}{1-\frac{3}{4}}=1 \end{split} \tag{7.2} \end{equation}\] Therefore, state 1 is persistent.
To see if state 1 is null persistent, we need to compute \[\begin{equation} \begin{split} \mu_1&=\sum_{n=1}^{\infty}nf_{11}(n)\\ &=1\cdot\frac{1}{2}+\frac{1}{2}\frac{1}{4}\sum_{n=2}^{\infty} n(\frac{3}{4})^{n-2}\\ &=1\cdot\frac{1}{2}+\frac{1}{2}\cdot\frac{1}{4}\cdot 20 =3 \end{split} \tag{7.3} \end{equation}\] Therefore, state 1 is non-null persistent.
Note also that \(p_{11}(1)>0\), \(p_{11}(2)>0\) and \(gcd{m:p_{11}(m)>0}=1\). Thus, state 1 is an aperiodic, non-null persistent state.
Now note that state 2 is intercommunicate with state 1, by Theorem 6.2, state 2 is also anaperiodic, non-null persistent state.
Using similar calculations, we will be able to show that states in \(C_2=\{5,6\}\) are aperiodic and non-null persistent.
Now consider state 3 and 4. One can travel from state \(3\to 1\), but cannot come back. Therefore, the probability of eventual coming back is less than 1, i.e. \(f_{33}<1\). Therefore, state 3 is a transient state. Similarly, for state 4, one can go \(4\to 6\) but cannot come back. State 4 is also transient because \(f_{44}<1\).Branching Process
In 18th century, British royal families were concerned about the family name. They wanted to know
What will be the expected number of people carrying their family names in different generations?
What is the probabilities that their family will be extinct?
Francis Galton with the help of Watson solved this problem, which famously designated as the Galton-Watson branching process.
Suppose each individual has a probability \(p_k\) of having \(k\) sons at any generation. Assume that the initial population starts with one individual, i.e. \(X_0=1\). Let \(X_{n+1}\) denote the number of sons in the family at the \((n+1)\)th generation. We define \(\xi_r\) as the number of sons for the \(r\)th individual, then we can write \(X_{n+1}=\sum_{r=1}^{X_n}\xi_r\) where \(P(\xi_r=k)=p_k\) for \(k=0,1,2,\cdots\). We want to answer \(E(X_n)\) and \(P(X_n=0,\exists n)\).
Let us define a generating function \[\begin{equation} \varphi(s)=\sum_{k=0}^{\infty}p_ks^k \tag{7.4} \end{equation}\] and \[\begin{equation} \varphi_n(s)=\sum_{k=0}^{\infty}P(X_n=k)s^k \tag{7.5} \end{equation}\] for \(n=0,1,2,\cdots\) and \(s\) is a variable taking values in \((0,1]\).
We have \[\begin{equation} \begin{split} \varphi_{n+1}(s)&=\sum_{k=0}^{\infty}P(X_{n+1}=k)s^k\\ &=\sum_{k=0}^{\infty}(\sum_{j=0}^{\infty}P(X_{n+1}=k|X_n=j)P(X_n=j))s^k\\ &=\sum_{j=0}^{\infty}P(x_n=j)[\sum_{k=0}^{\infty}P(\xi_1+\cdots+\xi_j=k)s^k] \end{split} \tag{7.6} \end{equation}\]
We claim that \[\begin{equation} \sum_{k=0}^{\infty}P(\xi_1+\cdots+\xi_j=k)s^k=[\varphi(s)]^j \tag{7.7} \end{equation}\] This is because \([\varphi(s)]^j=[\sum_{k=0}^{\infty}p_ks^k]^j\) and \[\begin{equation} \begin{split} P(\xi_1+\cdots+\xi_j=k)&=\sum_{h_1=0}^k\cdots\sum_{h_j=0}^kP(\xi_1=h_1)\cdots P(\xi_j=h_j)\quad (\text{subject to}\, h_1+\cdots+h_j=k)\\ &=\sum_{h_1=0}^k\cdots\sum_{h_j=0}^kp_{h_1}\cdots p_{h_j}\quad (\text{subject to}\, h_1+\cdots+h_j=k) \end{split} \end{equation}\] Look at the coefficient of \(s^k\) from \([\varphi(s)]^j=[\sum_{k=0}^{\infty}p_ks^k]^j\), it is exaactly \(\sum_{h_1=0}^k\cdots\sum_{h_j=0}^kp_{h_1}\cdots p_{h_j}\quad (\text{subject to}\, h_1+\cdots+h_j=k)\). Therefore, we have the claim \(\sum_{k=0}^{\infty}P(\xi_1+\cdots+\xi_j=k)s^k=[\varphi(s)]^j\).
From (7.6) and (7.7) we have \[\begin{equation} \varphi_{n+1}(s)=\sum_{j=0}^{\infty}P(x_n=j)[\varphi(s)]^j=\varphi_{n}(\varphi(s)) \tag{7.8} \end{equation}\] Iterating this relationship, we obtain \[\begin{equation} \varphi_{n+1}(s)=\varphi_{n}(\varphi(s))=\varphi_{n-1}(\varphi(\varphi(s)))=\cdots \tag{7.9} \end{equation}\] Generally, we obtain that \(\varphi_{n+1}(s)=\varphi_{n-k}(\varphi_{k+1}(s)))\) for any \(k=0,\cdots,n\). In particular, \[\begin{equation} \varphi_{n+1}(s)=\varphi(\varphi_n(s)) \tag{7.10} \end{equation}\]