Chapter 2 Model

Define the following model. Suppose we have the observation of a tree node as: \[y_{ij}, i = 1,\ldots,n_j, \; j = 1\ldots, m\] where \(y_{ij}\) is observation \(i\) in group \(j\). There are different numbers of observations \(n_j\) in each group, so for example \(n_1\) is the number of observations in group 1, etc. There are \(m\) groups. The total number of observations is \(n = \sum_{j=1}^m n_j\)

Then, for each tree node, suppose we have the likelihood: \[y_{ij} \sim N(\mu_j, \tau^{-1})\]

so each group has an overall mean \(\mu_j\), with an overall precision term \(\tau\).

We then have a hierarchical prior distribution:

\[\mu_j \sim N(\mu, k_1 (\tau^{-1}))\]

where \(k_1\) will be taken as a constant for simplicity, and the hyper-parameter prior distributions are:

\[\mu \sim N(0, \tau_{\mu} = k_2 (\tau^{-1}))\] \[\tau \sim Ga(\alpha, \beta)\]

Where the values \(k_1, k_2, \alpha, \beta\) are all fixed.

2.1 Maths

• The likelihood of each tree node will be:

\[\begin{equation} L = \prod_{j = 1}^{m} \prod_{i = 1}^{n_j} p(y_{ij} | \mu_{j}, \tau) \\ L \propto \tau^{n/2} exp \{ -\frac{\tau}{2} \sum_{j = 1}^{m} \sum_{i = 1}^{n_j} (y_{ij} - \mu_j)^2 \} \end{equation}\]

with prior distributions:

• \(\mu_j | \mu, \tau, k_1\)

\[\begin{equation} p(\mu_1, \dots, \mu_m | \mu, \tau) \propto (\tau/k_1)^{m/2} exp\{ - \frac{\tau}{2k_1} \sum_{j = 1}^{m} (\mu_j - \mu)^2 \} \end{equation}\]

• \(\tau | \alpha, \beta\)

\[p(\tau | \alpha, \beta) \propto \tau^{\alpha - 1} exp\{ - \beta \tau \}\]

• \(\mu | \tau_{\mu} = k_2 (\tau^{-1})\)

\[\begin{equation} p(\mu | k_2, \tau) \propto (\tau/k_2)^{1/2} exp\{ - \frac{ \tau}{2 k_2} \mu^2 \} \} \end{equation}\]

and their joint distribution as:

• \(p(\tau, \mu_1, \dots, \mu_m, \mu| y, k_1, k_2, \tau, \alpha, \beta)\)

\[\begin{equation} p(\tau, \mu_1, \dots, \mu_m, \mu| y, k_1, k_2, \tau, \alpha, \beta) \propto \tau^{\alpha - 1} exp\{ - \beta \tau \} \times \\ (\tau/k_1)^{m/2} exp\{ - \frac{\tau}{2k_1} \sum_{j = 1}^{m} (\mu_j - \mu)^2 \} \\ \times (\tau/k_2)^{1/2} exp\{ - \frac{ \tau}{2 k_2} \mu^2 \} \times \tau^{n/2} exp \{ -\frac{\tau}{2} \sum_{j = 1}^{m} \sum_{i = 1}^{n_j} (y_{ij} - \mu_j)^2 \} \end{equation}\]