3.2 Homework Problems (Winter 2020)

Exercise 3.12 (Homework 1, Problem 1) Let Xexp(λ), where E(X)=1λ. What is the p.m.f. of Y=X (the floor of X)? Do you recognize it as a distribution that you have studied in the past?

Proof. For nonnegative integer a, and becasue X is a continuous random variable, we have Pr(Y=a)=Pr(X<a+1)Pr(X<a)=(1eλ(a+1))(1eλa)=eλa(1eλ) and if a is negative integer, then Pr(Y=a)=0. Thus, we have the pmf of Y as pY(y)={eλy(1eλ)y{0,1,2,}0otherwise Form (3.44) we can also recognize that Y has a geometric distribution with parameter p=eλ.

Exercise 3.13 (Homework 1, Problem 2) Let X1 and X2 be two independent random variables such that XiGamma(ai,b) for any a1,a2,b>0. Define Y=X1X1+X2 and Z=X1+X2.

  1. Find the joint pdf for Y and Z and show that these two random variables are independent.

  2. Find the marginal pdf of Z. Do you recognize this pdf as belonging to some family that you know?

  3. Find the marginal pdf of Y . Do you recognize this pdf as belonging to some family that you know?

  4. Compute E(Yk) for any k>0.

  5. What does this result imply if a1=a2=b=1?

Proof. (a) Since X1 and X2 are independent Ga(ai,b), the joint pdf is then f(x1,x2)=ba1Γ(a1)xa111exp(bx1)×ba2Γ(a2)xa212exp(bx2)=ba1+a2Γ(a1)Γ(a2)xa111exp(bx1)xa212exp(bx2) for x1>0 and x2>0. Define variable transformation as (3.46), {y=x1x1+x2z=x1+x2 Then we have {x1=yzx2=(1y)z The Jacobian corresponding to (3.47) has determinant |J|=|zyz1y|=z Thus, the joint distribution of Y and Z can be written as f(y,z)=ba1+a2Γ(a1)Γ(a2)(yz)a11exp(byz)((1y)z)a21exp(b(1y)z)z=ya11(1y)a21ba1+a2Γ(a1)Γ(a2)za1+a21exp(bz)=f(y)f(z) for 0<y<1 and z>0.Since the joint pdf can be factorized as a function of y only and a function of z only, random variables Y and Z are independent.

  1. To get the marginal pdf of Z, we need to integrate out Y from the joint pdf. Therefore fZ(z)=10ya11(1y)a21ba1+a2Γ(a1)Γ(a2)za1+a21exp(bz)dy=ba1+a2Γ(a1)Γ(a2)za1+a21exp(bz)Γa1Γ(a2)Γ(a1+a2)=ba1+a2Γ(a1+a2)za1+a21exp(bz) for z>0. The second equation in (3.50) uses the fact that this integral is the kernel of a beta distributed random variabel with parameters a1 and a2. From (3.50) it is obvious that the marginal distribution of Z is Ga(a1+a2,b).

  2. Similarily, integrate out Z from the joint distribution will give us the marginal distribution of Y. fZ(z)=0ya11(1y)a21ba1+a2Γ(a1)Γ(a2)za1+a21exp(bz)dz=ya11(1y)a21ba1+a2Γ(a1)Γ(a2)Γ(a1+a2)ba1+a2=ya11(1y)a21B(a1,a2) for y(0,1). The second equation in (3.51) uses the fact that this integral is the kernel of a gamma distributed random variabel with parameters a1+a2 and b. From (3.51) it is obvious that the marginal distribution of Y is Beta(a1,a2).

  3. By definition E(Yk)=10ykya11(1y)a21B(a1,a2)dy=1B(a1,a2)10yk+a11(1y)a21dy=B(a1+k,a2)B(a1,a2) for k>0. The second equation uses the fact that the integral is the kernel of a beta distributed random variable with parameters a1+k and a2.

  4. If a1=a2=b=1, then for each Xi, the gamma distribution collapse to a exponential distribution with parameter λ=1. Thus, the marginal distribution of Y is Beta(1,1) which is just a uniform distribution on (0,1). The marginal distribution of Z is the sum of two independent exponential random variables, which is an Erlang distributed random variable with parameter k=2 and λ=1. The result can be generate to n independent random variables Xi, XiGa(1,1)=Exp(1). The random variable that denote the proportion of random variable Xi to the sum of all random variables ni=1Xi is a Beta(1,n1) and the sum ni=1Xi has an Erlang distribution with parameters k=n and λ=1 (same as Gamma(n,1)).

Exercise 3.14 (Homework 1, Problem 3) Consider three independent random variables X1,X2, and X3 such that XiindGamma(ai,b), i=1,2,3. Let Y=(Y1,Y2,Y3)=(X1X1+X2+X3,X2X1+X2+X3,X3X1+X2+X3)

  1. Show that YDirichlet(a1,a2,a3), a Dirichlet distribution.

  2. How can this result be used to generate random variables according to a Dirichlet distribution? Write a simple function in R that takes as inputs n, the number of trivariate vectors to be generated, and a=(a1,a2,a3) generates a matrix of size n×3 whose rows correspond to independent samples from a Dirichlet distribution with parameter (a1,a2,a3).

Use each of a=(0.01,0.01,0.01),(100,100,100) and (3,5,10) and comment how the density of Y changes over a.

Proof. (a) Since Xiind.Ga(ai,b), the joint pdf is then f(x1,x2,x3)=3i=1baiΓ(ai)xai1iexp(bxi) Define variable transformation in (3.54) {y1=x1x1+x2+x3y2=x2x1+x2+x3z=x1+x2+x3 Then we have {x1=y1zx2=y2zx3=(1y1y2)z The Jacobian corresponding to (3.55) has determinant |J|=|z0y10zy2zz1y1y2|=z2(1y1y2)+z2y1+z2y2=z2 Thus, the joint distribution of Y1, Y2 and Z can be written as f(y1,y2,z)=b3i=1ai3i=1Γ(ai)[(y1z)a11exp(by1z)(y2z)a21exp(by2z)((1y1y2)z)a31exp(b(1y1y2)z)]z2 for 0<y1<1,0<y2<1,0<y1+y2<1 and z>0.Denote y3=1y1y2 and integrate w.r.t. Z, then we have f(y1,y2,y3)=z=0b3i=1ai3i=1Γ(ai)[(y1)a11(y2)a21(y3)a31za1+a2+a31exp(bz)]dz=b3i=1ai3i=1Γ(ai)Γ(3i=1ai)b3i=1ai(y1)a11(y2)a21(y3)a31=Γ(3i=1ai)3i=1Γ(ai)(y1)a11(y2)a21(y3)a31 with 0<yi<1 and 3i=1yi=1. Therefore, Y=(y1,y2,y3)Dirichlet(a1,a2,a3)

  1. The R function to generate such samples is shown below.

To compare the density of samples generated using this function with different choice of a, we plot 500 samples using this function and a using (0.01,0.01,0.01), (100,100,100) and (3,5,10) in Figure 3.1, Figure 3.2, and Figure 3.3, respectively. From the plot, we notice that when a=(0.01,0.01,0.01), all samples are on the axises, meaning that in each sample, either 1 or two Yi are really close to 0. When a=(100,100,100), the variance of samples are quite small, which makes them centered at the theoretical mean (1/3,1/3,1/3). As for a=(3,5,10), samples are around their theoretical mean (1/6,5/18,5/9) with larger variance comparing to the previous case.

FIGURE 3.1: Samples using a=(0.01,0.01,0.01)

FIGURE 3.2: Samples using a=(100,100,100)

FIGURE 3.3: Samples using a=(3,5,10)

Exercise 3.15 (Homework 1, Problem 4) Y follows an inverse Gamma distribution with shape parameter a and scale parameter b (YIG(a,b)) if Y=1/X with XGamma(a,b), (assume the Gamma distribution is parameterized so that E(X)=ab).

  1. Find the density of Y.

  2. Compute E(Yk). Do you need to impose any constrain on the problem for this expectation to exists?

  3. Compare E(Yk) to 1/E(Xk).

Proof. (a) Since XGamma(a,b), the pdf is f(x)=xa1exbbaΓ(a) with x>0. For Y=1X, define variable transformation y=1x then x=1y with corresponding Jacobian satisfies |J|=1y2. Thus, the pdf of y is given by f(y)=y(a+1)exp(1by)baΓ(a) for y>0.

  1. By definition, we have E(yk)=0yky(a+1)exp(1by)baΓ(a)dy=1baΓ(a)0y(a+1k)exp(1by)dy=1baΓ(a)0tak1exp(tb)dt=bakΓ(ak)baΓ(a)=Γ(ak)bkΓ(a) The forth equation in (3.62) uses the fact that the intergral, by transforming variable y=1t, is the kernel of a gamma distributed random variabel with parameters ak and b. Therefore, the constrain for E(Yk) to exist is ak>0.

  2. For XGamma(a,b), by definition we have E(Xk)=0xkxa1exbbaΓ(a)dx=Γ(a+k)ba+kbaΓ(a)=bkΓ(a+k)Γ(a) Therefore, consider the ratio γ=1/E(Xk)E(Yk)=Γ(a)Γ(a)Γ(a+k)Γ(ak) Since from (3.64) γ=1k=0, we know that expectation is not invariant to non-linear transformation such as y=1x since every moment of Y and X are different.

Exercise 3.16 (Homework 1, Problem 5) Y follows a log normal distribution with parameters μ and σ2 (denotes as YLogN(μ,σ2)) if Y=exp(X) and XN(μ,σ2).

  1. Find the denisty of Y.

  2. Compute the mean and the variance of Y.

Proof. (a) Since XN(μ,σ2), the pdf of X is f(x)=1σ2πexp((xμ)22σ2) with x(,+) For Y=exp(X), consider variable transformation y=exp(x) then x=log(y) with corresponding Jacobian |J|=1y. Therefore, the pdf of Y is f(y)=1yσ2πexp((log(y)μ)22σ2) for y>0.

  1. We have E(Yk)=E(ekX)=MX(k)=exp(μk+σ2k22) Therefore, E(Y)=exp(μ+σ22)Var(Y)=E(Y2)(E(Y))2=eσ2+2μ(eσ21)
Exercise 3.17 (Homework 1, Problem 6) Let X=(X1,,Xp) with \mathbf{X}\sim N_p(\boldsymbol{\mu},\Sigma) and set \mathbf{Z}_1=(X_1,\cdots,X_q) and \mathbf{Z}_2=(X_{q+1},\cdots,X_p) with 1<q<p. Show that \begin{equation} \mathbf{Z}_1|\mathbf{Z}_2\sim N_q(\boldsymbol{\mu}_1+\Sigma_{12}\Sigma_{22}^{-1}(\mathbf{Z}_2-\boldsymbol{\mu}_2),\Sigma_{11}-\Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21}) \tag{3.68} \end{equation} where \boldsymbol{\mu}_k and \Sigma_{k\ell} denote the block of \boldsymbol{\mu} and \Sigma where the rows correspond to the variables in Z_k and the columns correspond to the variables in Z_{\ell}.

Proof. Using the inverse of block matrix formula from Wiki we have \begin{equation} \begin{split} \Sigma^{-1}&=\begin{pmatrix} \Sigma_{11} & \Sigma_{12} \\ \Sigma_{21} & \Sigma_{22} \end{pmatrix}^{-1}\\ &=\begin{pmatrix} \Sigma_1^{-1} & -\Sigma_{11}^{-1}\Sigma_{12}\Sigma_{2}^{-1}\\ -\Sigma_2^{-1}\Sigma_{21}\Sigma_{11}^{-1}& \Sigma_2^{-1} \end{pmatrix} \end{split} \tag{3.69} \end{equation} with \Sigma_1=\Sigma_{11}-\Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21} and \Sigma_2=\Sigma_{22}-\Sigma_{21}\Sigma_{11}^{-1}\Sigma_{12}. We have \begin{equation} \begin{split} f(\mathbf{z}_1|\mathbf{z}_2)&\propto f(\mathbf{z}_1,\mathbf{z}_2)\\ &\propto exp\{-\frac{1}{2}\begin{pmatrix} \mathbf{z}_1-\boldsymbol{\mu}_1\\ \mathbf{z}_2-\boldsymbol{\mu}_2 \end{pmatrix}^T\begin{pmatrix} \Sigma_{11} & \Sigma_{12} \\ \Sigma_{21} & \Sigma_{22} \end{pmatrix}^{-1}\begin{pmatrix} \mathbf{z}_1-\boldsymbol{\mu}_1\\ \mathbf{z}_2-\boldsymbol{\mu}_2 \end{pmatrix} \}\\ &\propto exp\{-\frac{1}{2}[(\mathbf{z}_1-\boldsymbol{\mu}_1)^T\Sigma_{1}^{-1}(\mathbf{z}_1-\boldsymbol{\mu}_1)-(\mathbf{z}_2-\boldsymbol{\mu}_2)^T\Sigma_2^{-1}\Sigma_{21}\Sigma_{11}^{-1}(\mathbf{z}_1-\boldsymbol{\mu}_1)\\ &-(\mathbf{z}_1-\boldsymbol{\mu}_1)^T\Sigma_{11}^{-1}\Sigma_{12}\Sigma_{2}^{-1}(\mathbf{z}_2-\boldsymbol{\mu}_2)]\}\\ &\propto exp\{-\frac{1}{2}[\mathbf{z}_1^T\Sigma_1^{-1}\mathbf{z}_1-\mathbf{z}_1^T(\Sigma_{1}^{-1}\boldsymbol{\mu}_1+\Sigma_{11}^{-1}\Sigma_{12}\Sigma_{2}^{-1}(\mathbf{z}_2-\boldsymbol{\mu}_2)\\ &-(\boldsymbol{\mu}_1^T\Sigma_{1}^{-1}+(\mathbf{z}_2-\boldsymbol{\mu}_2)^T\Sigma_2^{-1}\Sigma_{21}\Sigma_{11}^{-1})\mathbf{z}_1]\}\\ &=exp\{-\frac{1}{2}[(\mathbf{z}_1-(\boldsymbol{\mu}_1+\Sigma_{12}\Sigma_{22}^{-1}(\mathbf{z}_2-\boldsymbol{\mu}_2)))^T\Sigma_1^{-1}\\ &(\mathbf{z}_1-(\boldsymbol{\mu}_1+\Sigma_{12}\Sigma_{22}^{-1}(\mathbf{z}_2-\boldsymbol{\mu}_2)))]\} \end{split} \tag{3.70} \end{equation} Therefore, by recoginzing the kernel we have \mathbf{z}_1|\mathbf{z}_2\sim N(\boldsymbol{\mu}_1+\Sigma_{12}\Sigma_{22}^{-1}(\mathbf{z}_2-\boldsymbol{\mu}_2),\Sigma_{11}-\Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21}) as we desired.

Exercise 3.18 (Homework 1, Problem 7) Show that if X\sim exp(\beta), then

  1. Y=X^{1/\gamma} has a Weibull distribution with parameters \gamma and \beta with \gamma>0 a constant.

  2. Y=(2X/\beta)^{1/2} has the Rayleigh distribution.

For both parts, derive the form of the p.d.f., verify that is a p.d.f., and calculate the mean and the variance.

Proof. (a) Since X\sim Exp(\beta), the pdf of X is \begin{equation} f(x)=\beta exp(-\beta x) \tag{3.71} \end{equation} for x>0. Consider variable transformation y=x^{1/\gamma} then x=y^{\gamma} and hence |J|=\gamma y^{\gamma-1}, we have the pdf of y as \begin{equation} f(y)=\beta\gamma y^{\gamma-1}exp(-\beta y^{\gamma}) \tag{3.72} \end{equation} for y>0, which is a Weibull distribution with parameters \gamma>0 and \beta>0.

To verify the pdf, consider \begin{equation} \begin{split} \int_0^{\infty}f(y)dy&=\int_0^{\infty}\beta\gamma y^{\gamma-1}exp(-\beta y^{\gamma})dy\\ &=\int_0^{\infty}\beta exp(-\beta y^{\gamma})dy^{\gamma}=1 \end{split} \tag{3.73} \end{equation} Thus, it is a proper pdf.

To compute mean and variance, we have \begin{equation} \begin{split} E(Y^k)&=E(X^{k/\gamma})=\beta\int_0^{\infty}x^{k/\gamma}exp(-\beta x)dx\\ &=\frac{\Gamma(\frac{k}{\gamma}+1)}{\beta^{k/\gamma}} \end{split} \tag{3.74} \end{equation} where the second equation uses the fact that the integral is the kernel of a gamma distributed random variable with parameter \frac{k}{\gamma}+1 and \beta. Therefore \begin{equation} \begin{split} &E(Y)=\frac{\Gamma(\frac{1}{\gamma}+1)}{\beta^{1/\gamma}}\\ &Var(Y)=E(Y^2)-(E(Y))^2=\frac{\Gamma(\frac{2}{\gamma}+1)-(\Gamma(\frac{1}{\gamma}+1))^2}{\beta^{2/\gamma}} \end{split} \tag{3.75} \end{equation}

  1. Similarily, consider variable transformation y=(2x/\beta)^{1/2}, then x=\frac{\beta y^2}{2} with Jacobian |J|=\beta y and thus Y has the pdf in (3.76). \begin{equation} f(y)=\beta^2yexp(-\frac{\beta^2y^2}{2}) \tag{3.76} \end{equation} for y>0. It has the form as Rayleigh distribution with parameter \beta>0.

To see it is a legal pdf, consider \begin{equation} \begin{split} \int_0^{\infty}f(y)dy&=\int_0^{\infty}\beta^2yexp(-\frac{\beta^2y^2}{2})dy\\ &=\int_0^{\infty}\frac{1}{2}exp(-\frac{\beta^2y^2}{2})d\beta^2y^2=1 \end{split} \tag{3.77} \end{equation}

Finally, for mean and variance, we have \begin{equation} \begin{split} E(Y^k)&=E((\frac{2X}{\beta})^{k/2})=\frac{2^{k/2}}{\beta^{\frac{k-2}{2}}}\int_0^{\infty}x^{k/2}exp(-\beta x)dx\\ &=\frac{2^{k/2}\Gamma(\frac{k}{2}+1)}{\beta^k} \end{split} \tag{3.78} \end{equation} Therefore \begin{equation} \begin{split} &E(Y)=\frac{\sqrt{2}\Gamma(1.5)}{\beta}=\frac{\sqrt{2\pi}}{2\beta}\\ &Var(Y)=E(Y^2)-(E(Y))^2=\frac{2\Gamma(2)}{\beta^2}-(\frac{\sqrt{2\pi}}{2\beta})^2=\frac{4-\pi}{2\beta^2} \end{split} \tag{3.79} \end{equation}