21.4 Randomized Block Designs
To improve the precision of treatment comparisons, we can reduce variability among the experimental units. We can group experimental units into blocks so that each block contains relatively homogeneous units.
- Within each block, random assignment treatments to units (separate random assignment for each block)
- The number of units per block is a multiple of the number of factor combinations.
- Commonly, use each treatment once in each block.
Benefits of Blocking
Reduction in variability of estimators for treatment means
- Improved power for t-tests and F-tests
- Narrower confidence intervals
- Smaller MSE
Compare treatments under different conditions (related to different blocks).
Loss from Blocking (little to lose)
- If you don’t do blocking well, you waste df on negligible block effects that could have been used to estimate \(\sigma^2\)
- Hence, the df for \(t\)-tests and denominator df for \(F\)-tests will be reduced without reducing MSE and small loss of power for both tests.
Consider
\[ Y_{ij} = \mu_{..} + \rho_i + \tau_j + \epsilon_{ij} \]
where
- \(i = 1, 2, \dots, n\)
- \(j = 1, 2, \dots, r\)
- \(\mu_{..}\): overall mean response, averaging across all blocks and treatments
- \(\rho_i\): block effect, average difference in response for i-th block (\(\sum \rho_i =0\))
- \(\tau_j\) treatment effect, average across blocks (\(\sum \tau_j = 0\))
- \(\epsilon_{ij} \sim iid N(0,\sigma^2)\): random experimental error.
Here, we assume that the block and treatment effects are additive. The difference in average response for any pair of treatments i the same within each block
\[ (\mu_{..} + \rho_i + \tau_j) - (\mu_{..} + \rho_i + \tau_j') = \tau_j - \tau_j' \]
for all \(i=1,..,n\) blocks
\[ \begin{aligned} \hat{\mu} &= \bar{Y}_{..} \\ \hat{\rho}_i &= \bar{Y}_{i.} - \bar{Y}_{..} \\ \hat{\tau}_j &= \bar{Y}_{.j} - \bar{Y}_{..} \end{aligned} \]
Hence,
\[ \begin{aligned} \hat{Y}_{ij} &= \bar{Y}_{..} + (\bar{Y}_{i.} - \bar{Y}_{..}) + (\bar{Y}_{.j}- \bar{Y}_{..}) = \bar{Y}_{i.} + \bar{Y}_{.j} - \bar{Y}_{..} \\ e_{ij} &= Y_{ij} - \hat{Y}_{ij} = Y_{ij}- \bar{Y}_{i.} - \bar{Y}_{.j} + \bar{Y}_{..} \end{aligned} \]
ANOVA table
| Source of Variation | SS | df | Fixed Treatments E(MS) |
Random Treatments E(MS) |
|---|---|---|---|---|
| Blocks | \(r \sum_i(\bar{Y}_{i.}-\bar{Y}_{..})^2\) | \(n - 1\) | \(\sigma^2 +r \frac{\sum \rho^2_i}{n-1}\) | \(\sigma^2 + r \frac{\sum \rho^2_i}{n-1}\) |
| Treatments | \(n\sum_ j (\bar{Y} _ {.j}-\bar{ Y}_{..})^2\) | \(r - 1\) | \(\sigma^2 + n \frac{\sum \tau^2_j}{r-1}\) | \(\sigma^2 + n \sigma^2_\tau\) |
| Error | \(\sum_i \sum _j ( Y_{ ij } - \bar { Y}_{i.} - \bar{Y}_{.j} + \bar{ Y}_{..})^2\) | \((n-1)(r-1)\) | \(\sigma^2\) | \(\sigma^2\) |
| Total | \(SSTO\) | \(nr-1\) |
F-tests
\[ \begin{aligned} H_0: \tau_1 = \tau_2 = ... = \tau_r = 0 && \text{Fixed Treatment Effects} \\ H_a: \text{not all } \tau_j = 0 \\ \\ H_0: \sigma^2_{\tau} = 0 && \text{Random Treatment Effects} \\ H_a: \sigma^2_{\tau} \neq 0 \end{aligned} \]
In both cases \(F = \frac{MSTR}{MSE}\), reject \(H_0\) if \(F > f_{(1-\alpha; r-1,(n-1)(r-1))}\)
we don’t use F-test to compare blocks, because
- We have a priori that blocs are different
- Randomization is done “within” block.
To estimate the efficiency that was gained by blocking (relative to completely randomized design).
\[ \begin{aligned} \hat{\sigma}^2_{CR} &= \frac{(n-1)MSBL + n(r-1)MSE}{nr-1} \\ \hat{\sigma}^2_{RB} &= MSE \\ \frac{\hat{\sigma}^2_{CR}}{\hat{\sigma}^2_{RB}} &= \text{above 1} \\ \end{aligned} \]
then a completely randomized experiment would
\[ (\frac{\hat{\sigma}^2_{CR}}{\hat{\sigma}^2_{RB}}-1)\%% \]
more observations than the randomized block design to get the same MSE
If batches are randomly selected then they are random effects. That is , if the experiment was repeated, a new sample of i batches would be selected,d yielding new values for \(\rho_1, \rho_2,...,\rho_i\) then.
\[ \rho_1, \rho_2,...,\rho_j \sim N(0,\sigma^2_\rho) \]
Then,
\[ Y_{ij} = \mu_{..} + \rho_i + \tau_j + \epsilon_{ij} \]
where
- \(\mu_{..}\) fixed
- \(\rho_i\): random iid \(N(0,\sigma^2_p)\)
- \(\tau_j\) fixed (or random) \(\sum \tau_j = 0\)
- \(\epsilon_{ij} \sim iid N(0,\sigma^2)\)
Fixed Treatment
\[ \begin{aligned} E(Y_{ij}) &= \mu_{..} + \tau_j \\ var(Y_{ij}) &= \sigma^2_{\rho} + \sigma^2 \end{aligned} \]
\[ \begin{aligned} cov(Y_{ij},Y_{ij'}) &= \sigma^2 , j \neq j' \text{ treatments within same block are correlated} \\ cov(Y_{ij},Y_{i'j'}) &= 0 , i \neq i' , j \neq j' \end{aligned} \]
Correlation between 2 observations in the same block
\[ \frac{\sigma^2_{\rho}}{\sigma^2 + \sigma^2_{\rho}} \]
The expected MS for the additive fixed treatment effect, random block effect is
| Source | SS | E(MS) |
|---|---|---|
| Blocks | SSBL | \(\sigma^2 + r \sigma^2_\rho\) |
| Treatment | SSTR | \(\sigma^2 + n \frac{\sum \tau^2_j}{r-1}\) |
| Error | SSE | \(\sigma^2\) |
Interactions and Blocks
without replications within each block for each treatment, we can’t consider interaction between block and treatment when the block effect is fixed. Hence, only in the random block effect, we have
\[ Y_{ij} = \mu_{..} + \rho_i + \tau_j + (\rho \tau)_{ij} + \epsilon_{ij} \]
where
- \(\mu_{..}\) constant
- \(\rho_i \sim idd N(0,\sigma^2_{\rho})\) random
- \(\tau_j\) fixed (\(\sum \tau_j = 0\))
- \((\rho \tau)_{ij} \sim N(0,\frac{r-1}{r}\sigma^2_{\rho \tau})\) with \(\sum_j (\rho \tau)_{ij}=0\) for all i
- \(cov((\rho \tau)_{ij},(\rho \tau)_{ij'})= -\frac{1}{r} \sigma^2_{\rho \tau}\) for \(j \neq j'\)
- \(\epsilon_{ij} \sim iid N(0,\sigma^2)\) random
Note: a special case of mixed 2-factor model with 1 observation per “cell”
\[ \begin{aligned} E(Y_{ij}) &= \mu_{..} + \tau_j \\ var(Y_{ij}) &= \sigma^2_\rho + \frac{r-1}{r} \sigma^2_{\rho \tau} + \sigma^2 \end{aligned} \]
\[ \begin{aligned} cov(Y_{ij},Y_{ij'}) &= \sigma^2_\rho - \frac{1}{r} \sigma^2_{\rho \tau}, j \neq j' \text{ obs from the same block are correlated} \\ cov(Y_{ij},Y_{i'j'}) &= 0, i \neq i', j \neq j' \text{ obs from different blocks are independent} \end{aligned} \]
The sum of squares and degrees of freedom for interaction model are the same as those for the additive model. The difference exists only with the expected mean squares
| Source | SS | df | E(MS) |
|---|---|---|---|
| Blocks | \(SSBL\) | \(n-1\) | \(\sigma^2 + r \sigma^2_\rho\) |
| Treatment | \(SSTR\) | \(r -1\) | \(\sigma^2 + \sigma ^2_{\rho \tau} + n \frac{\sum \tau_j^2}{r-1}\) |
| Error | \(SSE\) | \((n-1)(r-1)\) | \(\sigma^2 + \sigma ^2_{\rho \tau}\) |
- No exact test is possible for block effects when interaction is present (Not important if blocks are used primarily to reduce experimental error variability)
- \(E(MSE) = \sigma^2 + \sigma^2_{\rho \tau}\) the error term variance and interaction variance \(\sigma^2_{\rho \tau}\). We can’t estimate these components separately with this model. The two are confounded.
- If more than 1 observation per treatment block combination, one can consider interaction with fixed block effects, which is called generalized randomized block designs (multifactor analysis).
21.4.1 Tukey Test of Additivity
(Tukey’s 1 df test for additivity)
formal test of interaction effects between blocks and treatments for a randomized block design. can also considered for testing additivity in 2-way analyses when there is only one observation per cell.
we consider a less restricted interaction term
\[ (\rho \tau)_{ij} = D\rho_i \tau_j \text{(D: Constant)} \]
So,
\[ Y_{ij} = \mu_{..} + \rho_i + \tau_j + D\rho_i \tau_j + \epsilon_{ij} \]
the least square estimate or MLE for D
\[ \hat{D} = \frac{\sum_i \sum_j \rho_i \tau_j Y_{ij}}{\sum_i \rho_i^2 \sum_j \tau^2_j} \]
replacing the parameters by their estimates
\[ \hat{D} = \frac{\sum_i \sum_j (\bar{Y}_{i.}- \bar{Y}_{..})(\bar{Y}_{.j}- \bar{Y}_{..})Y_{ij}}{\sum_i (\bar{Y}_{i.}- \bar{Y}_{..})^2 \sum_j(\bar{Y}_{.j}- \bar{Y}_{..})^2} \]
Thus, the interaction sum of squares
\[ SSint = \sum_i \sum_j \hat{D}^2(\bar{Y}_{i.}- \bar{Y}_{..})^2(\bar{Y}_{.j}- \bar{Y}_{..})^2 \]
The ANOVA decomposition
\[ SSTO = SSBL + SSTR + SSint + SSRem \]
where \(SSRem\): remainder sum of squares
\[ SSRem = SSTO - SSBL - SSTR - SSint \]
if \(D = 0\) (i.e., no interactions of the type \(D \rho_i \tau_j\)). \(SSint\) and \(SSRem\) are independent \(\chi^2_{1,rn-r-n}\).
If \(D = 0\),
\[ F = \frac{SSint/1}{SSRem/(rn-r-n)} \sim f_{(1-\alpha;rn-r-n)} \]
if
\[ \begin{aligned} &H_0: D = 0 \text{ no interaction present} \\ &H_a: D \neq 0 \text{ interaction of form $D \rho_i \tau_j$ present} \end{aligned} \]
we reject \(H_0\) if \(F > f_{(1-\alpha;1,nr-r-n)}\)