Chapter 7 Central Limit Theorem and law of large numbers
7.1 Introduction
In this Section we will show why the Normal distribution, introduced in Section 5.7, is so important in probability and statistics. The central limit theorem states that under very weak conditions (almost all probability distributions you will see will satisfy them) the sum of i.i.d. random variables, , will converge, appropriately normalised to a standard Normal distribution as . For finite, but large , we can approximate by a normal distribution and the normal distribution approximation can be used to answer questions concerning . In Section 7.2 we present the Central Limit Theorem and apply it to an example using exponential random variables. In Section 7.3 we explore how a continuous distribution (the Normal distribution) can be used to approximate sums of discrete distributions. Finally, in Section 7.4, we present the Law of Large Numbers which states that the uncertainty in the sample mean of observations decreases as increases and converges to the population mean . Both the Central Limit Theorem and the Law of Large Numbers will be important moving forward when considering statistical questions.
7.2 Statement of Central Limit Theorem
Before stating the Central Limit Theorem, we introduce some notation.
We write as .
Let be independent and identically distributed random variables (i.e. a random sample) with finite mean and variance . Let . Then
where is the mean of the distributions .
Therefore, we have that for large ,Suppose are i.i.d. exponential random variables with parameter .
- Find .
- Find limits within which will lie with probability .
Since are i.i.d. exponential random variables with parameter , and . Hence,
Since is sufficiently big, is approximately normally distributed by the central limit theorem (CLT). Therefore,
Given that , we can compute the exactly , which shows that the central limit theorem gives a reasonable approximation.
Since are i.i.d. exponential random variables with parameter , and . Therefore, and .
Since , will be approximately normally distributed by the CLT, henceThere are infinitely many choices for and but a natural choice is . That is, we choose and such that there is equal chance that is less than or greater than . Thus if for , satisfies , we have that
Hence,
7.3 Central limit theorem for discrete random variables
The central limit theorem can be applied to sums of discrete random variables as well as continuous random variables. Let be i.i.d. copies of a discrete random variable with and . Further suppose that the support of is in the non-negative integers . (This covers all the discrete distributions, we have seen, binomial, negative binomial, Poisson and discrete uniform.)
Let . Then the central limit theorem states that for large , . However, there will exist such thatThis is known as the continuity correction.
Suppose that is a Bernoulli random variable with , so and . Then
For , can be approximated by , see Figure 7.1.

Figure 7.1: Central limit theorem approximation for the binomial.
We can see the approximation in Figure 7.1 in close-up for to in Figure 7.2.

Figure 7.2: Central limit theorem approximation for the binomial for x=54 to 56.
7.4 Law of Large Numbers
We observed that and the variance is decreasing as increases.
Given that we have in general that
A random variable which has and is the constant, , that is, . This suggests that as , converges in some sense to . We can make this convergence rigorous.
A sequence of random variables are said to converge in probability to a random variable , if for any , We write as .
We will often be interested in convergence in probability where is a constant.
A useful result for proving convergence in probability to a constant is Chebychev’s inequality. Chebychev’s inequality is a special case of the Markov inequality which is helpful in bounding probabilities in terms of expectations.
Chebychev’s inequality.
Let be a random variable with and . Then for any ,
Law of Large Numbers.
Let be i.i.d. according to a random variable with and . Then

Figure 7.3: Dice picture.
Central limit theorem for dice
Let denote the outcomes of successive rolls of a fair six-sided dice.
Let denote the total score from rolls of the dice and let denote the mean score from rolls of the dice.
- What is the approximate distribution of ?
- What is the approximate probability that lies between and , inclusive?
- How large does need to be such that ?
Attempt Exercise 1 and then watch Video 15 for the solutions.
Video 15: Central limit theorem for dice
Alternatively the solutions are available:
Solution to Exercise 1
Then and .
Since the rolls of the dice are independent,
and
Thus by the central limit theorem, .
Using the CLT approximation above, and the continuity correction
If using Normal tables, we have that
and
Using the Central Limit Theorem, .
We know by the law of large numbers that as , but how large does need to be such that there is a (or greater) chance of being within of ?
Using the approximation , we want:
Now
Consider .
Note that
We have , and using
qnorm
function in Rqnorm(0.995)
gives 2.5758293.
Thereforeor equivalently
Given that we require , we have that .
Task: Lab 4
Attempt the R Markdown file for Lab 4:
Lab 4: Convergence and the Central Limit Theorem
Student Exercises
Attempt the exercises below.
Question 1.
Let be independent Poisson random variables each having mean 1. Use the central limit theorem to approximateSolution to Question 1.
Note that the final step comes from the symmetry of the normal distribution.
For comparison, since , we have that .
Question 2.
The lifetime of a Brand X TV (in years) is an exponential random variable with mean 10. By using the central limit theorem, find the approximate probability that the average lifetime of a random sample of 36 TVs is at least 10.5.
Solution to Question 2.
Let denote the lifetime of the TV in the sample, . Then (from the lecture notes) we know that , .
The sample mean is .
Using the central limit theorem, , sowhere . Thus the required probability is approximately .
Question 3.
Prior to October 2015, in the UK National Lottery gamblers bought a ticket on which they mark six different numbers from . Six balls were drawn uniformly at random without replacement from a set of similarly numbered balls. A ticket won the jackpot if the six numbers marked are the same as the six numbers drawn.
- Show that the probability a given ticket won the jackpot is .
- In Week 9 of the UK National Lottery tickets were sold and there were jackpot winners. If all gamblers chose their numbers independently and uniformly at random, use the central limit theorem to determine the approximate distribution of the number of jackpot winners that week. Comment on this in the light of the actual number of jackpot winners.
Solution to Question 3.
- There are different ways of choosing 6 distinct numbers , so the probability a ticket wins the jackpot if .
- Let be the number of jackpot winners in Week 9 if gamblers chose their numbers independently and uniformly at random. Then
Hence, by the central limit theorem,
where . This probability is small, so there is very strong evidence that the gamblers did not choose their numbers independently and uniformly at random.