Point and Interval Estimation

From Point Estimates to Confidence Intervals

Statistical inference begins with a point estimate: a single number computed from sample data as the best available guess for an unknown population parameter. The sample mean x̄ estimates the population mean μ; a sample proportion p estimates the population proportion π. Point estimates are useful but incomplete, because different samples produce different estimates, and a single number carries no information about how far it might be from the truth.

A confidence interval (CI) addresses this by attaching a measure of uncertainty to the estimate. A 95% confidence interval is constructed so that if the same procedure were repeated across many independent samples, 95% of the resulting intervals would contain the true parameter. It is best understood as a point estimate plus or minus a margin of error, where the margin reflects both the variability of the estimator and the desired level of confidence.

A wider interval signals lower precision and higher uncertainty; a narrower interval signals greater precision. The margin of error is always the product of two components: a multiplier determined by the confidence level, and the standard error of the estimator.

The General CI Formula When the Standard Error Is Known

When the estimator θ̂ is normally distributed and its standard error SE(θ̂) is known exactly, standardising produces a standard normal variable. Rearranging the resulting probability statement gives the CI endpoints. For a 95% interval:

For a general 100(1−α)% interval:

The multiplier z_{α/2} is the value that cuts off α/2 in the upper tail of the standard normal distribution. The three most commonly used confidence levels and their multipliers are:

Confidence level	α	z_{α/2}
90%	0.10	1.6449
95%	0.05	1.9600
99%	0.01	2.5758

To illustrate, suppose an estimator gives θ̂ = 12.4 with SE(θ̂) = 0.8. For a 90% interval, the multiplier is 1.6449, the margin of error is 1.6449 × 0.8 = 1.316, and the interval is (11.08, 13.72).

The Student’s t Distribution

In practice the standard error is rarely known exactly. When it must be estimated from the data using s in place of the unknown σ, the standardised ratio no longer follows a standard normal distribution. Instead it follows a Student’s t distribution with ν degrees of freedom.

The t distribution is bell-shaped and symmetric about zero, like the standard normal, but has fatter tails that assign more probability to extreme values. Each value of ν defines a different member of the family. As ν increases the tails thin and the distribution converges to N(0, 1). The critical value t_{α/2, ν} cuts off α/2 in the upper tail.

The practical consequence of the fatter tails is that t multipliers are larger than their z counterparts for the same confidence level. At the 95% level, t_{0.025, 5} = 2.571 for a small sample of six observations, compared with t_{0.025, 120} ≈ 1.980 for a much larger sample and z_{0.025} = 1.960 for the known-variance case. As a rule of thumb, when n > 30 the z approximation is generally acceptable.

The General CI Formula When the Standard Error Is Estimated

When the standard error must be estimated, replace SE(θ̂) with the estimated standard error ESE(θ̂) and replace the z multiplier with the appropriate t critical value:

The degrees of freedom ν depend on how the variance was estimated and differ between the one-sample and two-sample settings covered in the sections that follow.

Confidence Interval for a Single Mean: σ Known

When the population standard deviation σ is known, the standard error of the sample mean is σ/√n exactly and a z multiplier applies:

Suppose a sample of n = 200 ball bearings yields a mean diameter of x̄ = 0.824 mm and the population standard deviation is σ = 0.042 mm. The standard error is 0.042/√200 = 0.002970. For a 95% interval: 0.824 ± 1.96 × 0.002970 = 0.824 ± 0.006, giving (0.818, 0.830). For a 99% interval: 0.824 ± 2.5758 × 0.002970 = 0.824 ± 0.008, giving (0.816, 0.832). The 99% interval is wider because higher confidence requires a broader net.

Confidence Interval for a Single Mean: σ Unknown

When σ is unknown it is replaced by the sample standard deviation s, giving the estimated standard error s/√n. The t distribution with n − 1 degrees of freedom then applies:

Consider a sample of n = 12 items with a mean price of x̄ = £41.625 and sample standard deviation s = £7.84. The estimated standard error is 7.84/√12 = 2.263. With 11 degrees of freedom, t_{0.025, 11} = 2.201, the margin of error is 2.201 × 2.263 = 4.982, and the 95% confidence interval is (£36.64, £46.61).

Confidence Interval for a Single Proportion

For a survey of n respondents in which r answer yes, the sample proportion is p = r/n, estimating the population proportion π. The estimated standard error is √(p(1−p)/n), and for large n the normal approximation holds, so a z multiplier is used:

Suppose 68 out of 150 customers favour a new service. Then p = 68/150 = 0.453 and ESE(p) = √(0.453 × 0.547 / 150) = 0.04065. The 95% confidence interval is 0.453 ± 1.96 × 0.04065 = 0.453 ± 0.080, giving (0.37, 0.53).

Determining the Required Sample Size

Before data collection, the required sample size can be calculated to estimate a parameter within a specified tolerance e at a given confidence level.

For a population mean, set the margin of error equal to the tolerance and solve for n:

Always round up to the next integer. If reaction times have σ = 0.05 seconds and the tolerance is e = 0.01 seconds, then for 95% confidence n ≥ (1.96)² × (0.05)² / (0.01)² = 96.04, giving n = 97. For 99% confidence n ≥ (2.5758)² × (0.05)² / (0.01)² = 165.87, giving n = 166. Greater confidence demands a larger sample.

For a population proportion the same approach applies:

If no prior estimate of π is available, use p = 0.5, which maximises p(1−p) and produces the most conservative sample size. If a pilot study gives p = 0.40 and the tolerance is e = 0.03 at 90% confidence, then n ≥ (1.645)² × 0.40 × 0.60 / (0.03)² = 721.6, giving n = 722.

Confidence Interval for the Difference Between Two Proportions

When comparing two populations on a binary outcome, the difference p₁ − p₂ estimates π₁ − π₂. The estimated standard error combines variability from both samples:

The confidence interval is:

If the interval includes zero, there is no statistically significant difference between the two proportions.

Consider advertising awareness measured before a campaign (p₂ = 68/150 = 0.4533) and after (p₁ = 65/120 = 0.5417). The point estimate is p₁ − p₂ = 0.0884 and the estimated standard error is √(0.4533 × 0.5467 / 150 + 0.5417 × 0.4583 / 120) = √0.003720 = 0.06099. The 95% CI is 0.0884 ± 1.96 × 0.06099 = 0.0884 ± 0.1195, giving (−0.031, 0.208). Because the interval contains zero, there is no significant evidence of a change in awareness.

Confidence Interval for the Difference Between Two Means: Variances Known

For two independent populations with known variances σ₁² and σ₂², the confidence interval for μ₁ − μ₂ uses a z multiplier:

With n₁ = 40, x̄₁ = 52, σ₁ = 6 and n₂ = 50, x̄₂ = 48, σ₂ = 4, the point estimate is 4 and the standard error is √(36/40 + 16/50) = √1.220 = 1.105. The 95% CI is 4 ± 1.96 × 1.105 = 4 ± 2.166, giving (1.83, 6.17). Zero is not in the interval, supporting the conclusion that μ₁ > μ₂.

When the variances are unknown but both sample sizes exceed 30, replace σ₁² and σ₂² with the sample variances s₁² and s₂² and continue using a z multiplier. The central limit theorem justifies the normal approximation for large samples.

Confidence Interval for the Difference Between Two Means: Equal Variances (Pooled t)

When population variances are unknown but can be assumed equal (σ₁² = σ₂² = σ²), the two samples are combined into a single pooled variance estimate that uses all available information:

This estimate has n₁ + n₂ − 2 degrees of freedom. The confidence interval is:

For Company A with n₁ = 12, x̄₁ = 8.5, s₁ = 3.6 and Company B with n₂ = 10, x̄₂ = 4.8, s₂ = 2.1, the pooled variance on 20 degrees of freedom is:

The estimated standard error is √(9.113 × (1/12 + 1/10)) = √1.670 = 1.292. With t_{0.025, 20} = 2.086 and a point estimate of 3.7, the 95% CI is 3.7 ± 2.086 × 1.292 = 3.7 ± 2.697, giving (1.01, 6.39). Zero is not in the interval, so Company B is significantly faster on average.

Confidence Interval for the Difference Between Two Means: Paired Samples

When the same individuals are measured twice — before and after a treatment, for example — the two samples are not independent and the unpaired methods above do not apply. The correct approach is to reduce the problem to a single sample by working with the individual differences:

Compute the mean x̄_d and standard deviation s_d of the differences, then apply the single-mean t formula with n − 1 degrees of freedom:

If zero does not appear in the interval, there is evidence of a genuine change.

In a diet study with eight participants, the before-minus-after weight differences in pounds were 5, 10, −2, 7, 6, 9, 12, 1. The mean difference is x̄_d = 48/8 = 6 and the standard deviation is s_d = 4.66. The estimated standard error is 4.66/√8 = 1.648. With t_{0.025, 7} = 2.365, the 95% CI is 6 ± 2.365 × 1.648 = 6 ± 3.90, giving (2.1, 9.9). Zero is not in the interval, so the diet does reduce weight on average.

Quick Reference

Situation	Variance known?	Test statistic
Single mean	Yes	z
Single mean	No	t_{n−1}
Single proportion	— (large n)	z
Difference of means, unpaired	Known	z
Difference of means, unpaired	Unknown, unequal, large n	z
Difference of means, unpaired	Unknown, equal	t_{n₁+n₂−2} (pooled)
Difference of means, paired	—	t_{n−1} on differences
Difference of proportions	— (large n)	z

An article with 10 exercises for you to practice here: https://datalad.co.uk/point-and-interval-estimation-workbook-10-exercises-with-full-solutions/

See you soon.