Chapter 7 Decomposition of States, Branching Process (Lecture on 01/26/2021)
i↔j if pij(m)>0 for some m and pji(n)>0 for some n.
Definition 7.1 (Close and Irreducible) A set C of states is called
Closed, if pij=0 for all i∈C and j∉C.
- Irreducible, if i↔j for all i,j∈C.
Condition (a) says that once a chain takes a value in a closed set C of states, then it never leaves C subsequently.
Regarding condition (b), we have already proved that for intercommunicating states, we just need to study properties of one of the states, and all the other states in C will have the same properties.
Theorem 7.1 (Decomposition Theorem) The state space S can be partitioned uniquely as S=T∪C1∪C2∪⋯, where T is the set of transient states and Ci irreducible closed set of persistent states.
Example 7.1 (Partition of States) Let S={1,2,3,4,5,6} and the transition probability matrix is given by (1212000014340000141414140014014140140000121200001212) We will study properties of each of these states.
Let C1={1,2}, then C1 is a closed and irreducible set. Then let C2={5,6}, then C2 is also a closed and irreducible set.
Question: Properties of the states in C1, are they null recurrent? Whar are their period? etc.
Let us investigate state 1, we have f11(n)={12n=112(34)n−214n≥2 Therefore, f11=∞∑n=1f11(n)=12+12∞∑n=2(34)n−214(geometric sum)=12+12⋅14⋅11−34=1 Therefore, state 1 is persistent.
To see if state 1 is null persistent, we need to compute μ1=∞∑n=1nf11(n)=1⋅12+1214∞∑n=2n(34)n−2=1⋅12+12⋅14⋅20=3 Therefore, state 1 is non-null persistent.
Note also that p11(1)>0, p11(2)>0 and gcdm:p11(m)>0=1. Thus, state 1 is an aperiodic, non-null persistent state.
Now note that state 2 is intercommunicate with state 1, by Theorem 6.2, state 2 is also anaperiodic, non-null persistent state.
Using similar calculations, we will be able to show that states in C2={5,6} are aperiodic and non-null persistent.
Now consider state 3 and 4. One can travel from state 3→1, but cannot come back. Therefore, the probability of eventual coming back is less than 1, i.e. f33<1. Therefore, state 3 is a transient state. Similarly, for state 4, one can go 4→6 but cannot come back. State 4 is also transient because f44<1.Branching Process
In 18th century, British royal families were concerned about the family name. They wanted to know
What will be the expected number of people carrying their family names in different generations?
What is the probabilities that their family will be extinct?
Francis Galton with the help of Watson solved this problem, which famously designated as the Galton-Watson branching process.
Suppose each individual has a probability pk of having k sons at any generation. Assume that the initial population starts with one individual, i.e. X0=1. Let Xn+1 denote the number of sons in the family at the (n+1)th generation. We define ξr as the number of sons for the rth individual, then we can write Xn+1=∑Xnr=1ξr where P(ξr=k)=pk for k=0,1,2,⋯. We want to answer E(Xn) and P(Xn=0,∃n).
Let us define a generating function φ(s)=∞∑k=0pksk and φn(s)=∞∑k=0P(Xn=k)sk for n=0,1,2,⋯ and s is a variable taking values in (0,1].
We have φn+1(s)=∞∑k=0P(Xn+1=k)sk=∞∑k=0(∞∑j=0P(Xn+1=k|Xn=j)P(Xn=j))sk=∞∑j=0P(xn=j)[∞∑k=0P(ξ1+⋯+ξj=k)sk]
We claim that ∞∑k=0P(ξ1+⋯+ξj=k)sk=[φ(s)]j This is because [φ(s)]j=[∑∞k=0pksk]j and P(ξ1+⋯+ξj=k)=k∑h1=0⋯k∑hj=0P(ξ1=h1)⋯P(ξj=hj)(subject toh1+⋯+hj=k)=k∑h1=0⋯k∑hj=0ph1⋯phj(subject toh1+⋯+hj=k) Look at the coefficient of sk from [φ(s)]j=[∑∞k=0pksk]j, it is exaactly ∑kh1=0⋯∑khj=0ph1⋯phj(subject toh1+⋯+hj=k). Therefore, we have the claim ∑∞k=0P(ξ1+⋯+ξj=k)sk=[φ(s)]j.
From (7.6) and (7.7) we have φn+1(s)=∞∑j=0P(xn=j)[φ(s)]j=φn(φ(s)) Iterating this relationship, we obtain φn+1(s)=φn(φ(s))=φn−1(φ(φ(s)))=⋯ Generally, we obtain that φn+1(s)=φn−k(φk+1(s))) for any k=0,⋯,n. In particular, φn+1(s)=φ(φn(s))