Explain Central Limit Theorem in detail.

Easy Last updated on April 26, 2022, 11:39 p.m.

Answer In Progress

The central limit theorem states that the expected value of a number of (independent and identically distributed) i.i.d random variables with finite variances, will tend to a normal distribution as the number of random variables grows. More formally, if $X_{1}, X_{2}…$ are a sequence of i.i.d random variables with $E[X_{i}]=\mu$ and $Var[X_{i}]=\sigma^{2}<\infty$, then

$$\frac{\sqrt{n}\bar{X_{n}} - \mu}{\sigma} \rightarrow N(0,1)$$

Markdown Monster icon
the central limit theorem indicates that the sample mean will equal the population mean.

A $\alpha$-confidence interval on the mean can then be computed as
$$[\bar{X_{n}} - z_{(1-\alpha)/2}\frac{\sigma}{n}, \bar{X_{n}} + z_{(1-\alpha)/2}\frac{\sigma}{n}]$$

Here $z$ denotes the z-score of the standard normal distribution. Note that we do not have access to $\sigma$ of the unknown distribution. In this case, we can use the standard deviation of the sample $\sigma_{est}$ and the z-score of the student t distribution to compute confidence intervals

$$[\bar{X_{n}} - z_{(1+\alpha)/2}\frac{\sigma_{est}}{\sqrt{n}}, \bar{X_{n}} + z_{(1+\alpha)/2}\frac{\sigma_{est}}{\sqrt{n}}]$$

where $z$ is the z-score of the student t distribution having degrees of freedom set to $n-1$

Note: Bounds computed using the central limit theorem only holds for either a large number of samples or small samples sizes from Gaussian distributions. Sample sizes equal to or greater than 30 are often considered sufficient for the Central Limit Theorem to hold.