This workbook accompanies the Simple Regression Model article and is built for practice with pen, paper, and a calculator. The ten exercises follow the lesson’s arc: the model and its assumptions, computing the OLS estimates by hand, fitted values and residuals, the algebraic properties, R-squared, and the statistical results of unbiasedness, error variance, and standard errors. Several exercises build on a single small dataset, so once you compute the line you can reuse it.
Attempt every exercise before reading the solutions, and write out each step. A reliable self-check throughout is that the residuals must sum to zero and the regression line must pass through the point of sample means, so if either fails, an arithmetic slip has crept in.
Part One: The Exercises
Exercise 1 (Model components). For the model wage = β₀ + β₁ educ + u, name the dependent variable, the explanatory variable, and the error term, and state in one sentence what β₁ measures.
Exercise 2 (Zero conditional mean). State whether the zero conditional mean assumption is likely to hold in each case. (a) Crop yield regressed on a randomly assigned amount of fertiliser. (b) Wage regressed on years of education, where higher-ability people tend to choose more schooling.
Exercise 3 (Computing the OLS estimates). For the five observations (x, y) = (1, 1), (2, 3), (3, 4), (4, 5), (5, 7), compute the OLS slope and intercept and write the fitted regression line.
Exercise 4 (Fitted values and residuals). Using the line from Exercise 3, compute the fitted value and residual for each observation, and verify that the residuals sum to zero.
Exercise 5 (Algebraic property). For the same data, verify that the sum of the regressor times the residuals is zero, and confirm that the point of sample means lies on the regression line.
Exercise 6 (R-squared). For the same data, compute the total, explained, and residual sums of squares, then compute R-squared and interpret it.
Exercise 7 (The assumptions). State which of the SLR assumptions guarantees that OLS is unbiased, and which one is needed to derive the variance of the estimators. Briefly say what that second assumption requires.
Exercise 8 (Unbiasedness). Explain what it means for the OLS slope estimator to be unbiased, whether a single estimate of 1.4 must equal the true β₁, and the single most common reason OLS becomes biased.
Exercise 9 (Estimating the error variance). Using the residual sum of squares from Exercise 6, estimate the error variance and the standard error of the regression for this five-observation sample.
Exercise 10 (Standard error of the slope). Using the results from Exercises 6 and 9, compute the standard error of the slope estimate, and explain in one sentence what it tells you.
Part Two: Worked Solutions
Solution 1. The dependent variable, the one being explained, is wage. The explanatory variable, doing the explaining, is educ. The error term u collects every other factor affecting wage, such as ability, motivation, and experience. The slope β₁ measures the change in wage for a one-year increase in education, holding all the factors in u fixed.
Solution 2. In part (a), the assumption holds, because random assignment of fertiliser makes it independent of land quality and every other determinant of yield, so knowing the fertiliser amount tells you nothing about the expected error. In part (b), the assumption almost certainly fails, because people partly choose their education based on their own ability, and ability sits in the error, so education and the error are correlated.
Solution 3. The means are x-bar equal to 3 and y-bar equal to 4. The deviations of x are -2, -1, 0, 1, 2 and of y are -3, -1, 0, 1, 3, so the cross-products sum to 14 and the squared x-deviations sum to 10.
The fitted line is y-hat = -0.2 + 1.4x.
Solution 4. Substituting each x into the line gives fitted values of 1.2, 2.6, 4.0, 5.4, and 6.8, so the residuals, actual minus fitted, are -0.2, 0.4, 0.0, -0.4, and 0.2.
The residuals sum to zero, as they must by construction.
Solution 5. Weighting each residual by its x value gives the products -0.2, 0.8, 0.0, -1.6, and 1.0.
This confirms the residuals are uncorrelated with the regressor. The point of sample means lies on the line, since plugging x-bar into the line returns y-bar.
Solution 6. With y-bar equal to 4, the total sum of squares is the sum of squared deviations of y, which is 9 + 1 + 0 + 1 + 9 = 20. The explained sum of squares, the squared deviations of the fitted values from y-bar, is 7.84 + 1.96 + 0 + 1.96 + 7.84 = 19.6. The residual sum of squares, the squared residuals, is 0.04 + 0.16 + 0 + 0.16 + 0.04 = 0.40, and 19.6 plus 0.40 returns the total of 20 as a check.
The regressor explains 98% of the variation in y in this sample.
Solution 7. The zero conditional mean assumption, SLR.4, that the expected error given x is zero, is what guarantees OLS is unbiased. The homoskedasticity assumption, SLR.5, is the one needed to derive the variance of the estimators, and it requires that the error has the same variance for every value of x. Homoskedasticity plays no role in unbiasedness; it is needed only for the variance formulas.
Solution 8. Unbiasedness means that the estimator, the rule for computing the slope, gives the true β₁ on average across many repeated samples, so its expected value equals β₁. It does not mean any single estimate equals the truth, so an estimate of 1.4 may sit above or below β₁ in any one sample, because unbiasedness is a statement about the long-run average, not about one draw. The most common reason OLS becomes biased is a violation of SLR.4, when an omitted factor in the error is correlated with x.
Solution 9. The unbiased estimate of the error variance divides the residual sum of squares by n minus 2, here 5 minus 2.
The standard error of the regression is its square root.
Solution 10. The standard error of the slope divides the standard error of the regression by the square root of the total variation in x, which is 10.
It measures how much the slope estimate would typically vary from sample to sample, and so quantifies the precision of the 1.4 estimate and forms the basis for confidence intervals and hypothesis tests.
How to Get the Most From This Workbook
The computational exercises all run through the same short pipeline: find the means, build the deviations from those means, and from them get the slope as the cross-product sum over the squared x-deviations, then the intercept from the through-the-means relationship. Everything else, fitted values, residuals, the sums of squares, R-squared, and the standard errors, follows from those pieces. The conceptual exercises drill the one thing that matters most for trusting a regression: the zero conditional mean assumption is what makes the slope a genuine effect and OLS unbiased, and it is the assumption that fails whenever an omitted factor is correlated with the regressor. Compute a few of these by hand until the pipeline is automatic, and keep asking of every real regression whether SLR.4 is plausible, because that question decides whether your tidy number means anything at all.
[…] The Simple Regression Model: 10 Exercises with Full Solutions […]