The Normal Distribution

Random Variables

A random variable is a function that assigns a real number to each outcome of an experiment, making it possible to attach probabilities to numerical values. Capital letters such as X, Y, or Z denote the random variable itself; lower-case letters x, y, z denote a specific observed value it takes.

Random variables fall into two categories. A discrete random variable takes countable values, such as the score on a die or the number of machine breakdowns in a week. A continuous random variable takes any value within a range, such as height, weight, or temperature, and is typically the result of measurement rather than counting.

To illustrate, suppose a coin is tossed. The sample space S = {H, T} is non-numerical, so define X = 1 for heads and X = 0 for tails. The random variable X now takes values in {0, 1}, making it a discrete random variable with two possible outcomes.

The Population Mean

The population mean μ is the probability-weighted average of all values the random variable can take. Unlike the sample mean x̄, it is a theoretical quantity derived from the full probability distribution:

where x_i are the possible values and p_i = P(X = x_i) are their corresponding probabilities. The probabilities must always sum to 1.

Suppose X takes values 3, 6, and 8 with probabilities 0.20, 0.70, and 0.10 respectively. Computing each weighted term gives 3(0.20) = 0.60, 6(0.70) = 4.20, and 8(0.10) = 0.80. Summing these yields μ = 0.60 + 4.20 + 0.80 = 5.60.

Population Variance and Standard Deviation

The population variance σ² measures the spread of a distribution around its mean. By definition, it is the probability-weighted average of squared deviations from μ:

A computationally more convenient form that produces an identical result is the shortcut formula:

The standard deviation σ is the positive square root of the variance.

For a fair die where X takes values 1 through 6 each with probability 1/6, the mean is μ = 3.5. Applying the shortcut formula, E(X²) is the average of the squared face values:

The variance is therefore σ² = 91/6 − (3.5)² = 15.167 − 12.25 = 2.917, and the standard deviation is σ = √2.917 ≈ 1.71.

The Normal Distribution

The normal distribution is a continuous, bell-shaped, symmetric distribution described entirely by two parameters: its mean μ and variance σ². A random variable following this distribution is written:

It is symmetric about μ, so the mean, median, and mode all coincide. The mean can take any real value, while the variance must be strictly positive. The total area under the curve equals 1. A larger variance produces a wider, flatter bell; a smaller variance produces a narrower, taller one.

The standard normal distribution is the special case where μ = 0 and σ² = 1, written Z ∼ N(0, 1). All other normal distributions can be converted to this form, which is what makes published tables usable in general.

To see the role of the variance in practice, compare X ∼ N(10, 25) and Y ∼ N(10, 4). Both distributions are centred at μ = 10, but X has standard deviation σ = 5 while Y has σ = 2. The X curve is wider and flatter; the Y curve is narrower and taller. The two curves share a centre but cross each other because of their different spreads.

The Central Limit Theorem

The central limit theorem is one of the most important results in statistics. It states that the sample mean X̄ of any population with finite mean and variance follows an approximately normal distribution for sufficiently large sample sizes, regardless of the shape of the underlying population.

Formally, if X₁, X₂, …, Xₙ are drawn from a population with mean μ and variance σ², then as n grows:

A sample size of n ≥ 30 is generally considered sufficient. If the population is itself normally distributed, the result holds exactly for any n, however small.

Consider a country where heights have μ = 170 cm and σ = 10 cm but the population distribution is not normal. For samples of size n = 49, the theorem applies because 49 ≥ 30. The expected value of the sample mean is 170, its variance is σ²/n = 100/49, and the sampling distribution is:

Working with Standard Normal Tables

Standard normal tables give the lower-tail probability for Z ∼ N(0, 1):

Any required probability can be derived from this function. For an upper-tail probability P(Z > z) with z ≥ 0, subtract from 1 to get 1 − Φ(z). For a negative argument P(Z ≤ z) where z < 0, apply the symmetry of the standard normal: Φ(−z) = 1 − Φ(z). For an interval probability P(a < Z < b), compute Φ(b) − Φ(a), handling any negative bounds using symmetry first.

As a direct example, to find P(Z > 1.2): the table gives Φ(1.2) = 0.8849, so P(Z > 1.2) = 1 − 0.8849 = 0.1151.

For a two-sided probability spanning a negative bound, consider P(−1.24 < Z < 1.86). This equals Φ(1.86) − Φ(−1.24). The table gives Φ(1.86) = 0.9686. The key symmetry rule for negative arguments is:

Applying it gives Φ(−1.24) = 1 − Φ(1.24) = 1 − 0.8925 = 0.1075. The final probability is therefore 0.9686 − 0.1075 = 0.8611.

Standardisation

Standard normal tables only cover Z ∼ N(0, 1), so any general normal random variable X ∼ N(μ, σ²) must be converted before using them. The conversion subtracts the mean and divides by the standard deviation:

Once standardised, all table lookups proceed in the usual way.

To find P(5.8 < X < 7.0) when X ∼ N(5, 4), note that μ = 5 and σ = √4 = 2. Standardising both bounds:

From tables, Φ(1.0) = 0.8413 and Φ(0.4) = 0.6554. The probability is 0.8413 − 0.6554 = 0.1859.

Estimators and Estimates

When the full population cannot be observed, sample data is used to approximate unknown population parameters. Two terms need to be distinguished carefully.

An estimator is a formula, a function of the sample data that is itself a random variable. The sample mean X̄ = (1/n) ΣXᵢ is an estimator of the population mean μ. An estimate is the specific numerical value the estimator produces when applied to a particular sample. Two different samples drawn from the same population will generally yield two different estimates from the same estimator.

For example, if one sample gives the values 4, 8, 2, 6, the estimate is x̄ = 20/4 = 5. A second sample of 2, 9, 1, 4 gives x̄ = 16/4 = 4. The estimator rule is unchanged; the estimates differ because of sampling variability.

The Sampling Distribution of the Sample Mean

Because X̄ is a random variable, it has its own probability distribution, known as the sampling distribution. When the underlying population is normal, this result holds exactly for any sample size: for a population N(μ, σ²) with sample size n:

For any non-normal population the CLT provides the same result approximately when n is large. The standard error of X̄ is the standard deviation of its sampling distribution:

A larger sample size reduces the standard error and yields more precise estimates. This is the formal basis for the intuition that larger samples are better.

As a worked example, consider a random sample of n = 16 drawn from N(25, 9). Here μ = 25 and σ = 3, so the sampling distribution is X̄ ∼ N(25, 9/16) with standard error 3/4 = 0.75. To find P(X̄ > 26), standardise:

From tables, Φ(1.33) = 0.9082, so P(Z > 1.33) = 1 − 0.9082 = 0.0918.

A workbook for you to practice here: https://datalad.co.uk/the-normal-distribution-workbook-10-exercises-with-full-solutions/

See you soon.