4.4 Chapter summary

Statistical tests are procedures for deciding whether to accept or reject a hypothesis about the population given the data from the sample. You can test whether a parameter differs from some value (often zero), and also whether it is equivalent to some value (again often zero) to within a pre-defined region of practical equivalence. The most common tests of difference are null hypothesis significance tests, in which a p-value less than 0.05 is taken as grounds for rejecting the null hypothesis. Significant p-values indicate the result is worthy of further attention. However, they often represent false positives, just as non-significant ones often reflect false negatives. The likelihood of returning a non-significant p-value is strongly affected by sample size. An effect or association that is not significantly different from zero may not be equivalent to zero. It may be that the evidence is simply inconclusive.