Contingency Tables and the Chi-Squared Test

Some of the most important questions in statistics are about categories rather than measurements. Does hair colour relate to eye colour? Does a person’s region predict the kind of crime they fall victim to? Does gender relate to political affiliation? You cannot answer these with correlation, because correlation needs numbers. The tool for categorical relationships is the chi-squared test, built on a humble device called the contingency table, and this article works through both the test of association and its close cousin, the goodness-of-fit test.

Association Is Not Correlation

It helps to be precise about what we are measuring. Correlation describes the relationship between two measurable, numerical variables, and it tells you both whether a relationship exists and how strong it is. Association is the categorical counterpart, and it answers only the first of those questions: whether two categorical variables are related at all, not how strongly. The choice of method follows directly from the variable types. Two measurable variables call for correlation or regression. Two categorical variables call for a chi-squared test of association. When one variable is measurable and the other categorical, you demote the measurable one into categories and then run the chi-squared test.

The hypotheses for a test of association are always the same in spirit. The null hypothesis states that there is no association between the two factors, which is equivalent to saying the two factors are independent. The alternative states that an association does exist. So if a researcher records hair colour and eye colour for 200 people, both variables are categorical with no natural measurement scale, which points straight to a chi-squared test of association, with a null hypothesis of no association between hair colour and eye colour.

Contingency Tables and Observed Frequencies

A contingency table, also called a cross-tabulation, organises counts by two categorical factors at once. The rows represent the levels of one factor, the columns the levels of the other, and each cell holds the count of observations falling into that particular row-and-column combination. These counts are the observed frequencies, written as O for cell i, j. The first thing to do with any contingency table is complete the row totals, the column totals, and the grand total, because every later calculation depends on them.

Consider crimes recorded by area and type, with 150 cases in total.

The cell counts are entered as given, each row is summed across, each column is summed down, and a quick check confirms the row totals and column totals both add to the grand total of 150.

Expected Frequencies

The heart of the test is comparing what we observed against what we would expect if the two factors were genuinely independent. Under the null hypothesis of independence, the expected frequency for a cell is the product of its row total and column total divided by the grand total.

This comes directly from the independence rule for probabilities, scaled up to the sample size.

Applying the formula to the first row of the crimes table gives an expected burglary count in area A of 55 times 50 over 150, which is 18.33, an expected robbery count of 55 times 60 over 150, which is 22.00, and an expected car theft count of 55 times 40 over 150, which is 14.67. A useful sanity check is that these expected values sum back to the row total of 55. Completing every cell produces the full table of expected frequencies.

The Chi-Squared Test Statistic

The test statistic measures how far the observed counts stray from the expected ones, summing the standardised squared differences across every cell.

Under the null hypothesis this statistic follows a chi-squared distribution with degrees of freedom equal to (r minus 1) times (c minus 1), where r and c are the numbers of rows and columns. The logic of the comparison makes this an upper-tailed test in every case. When observed and expected counts are close, supporting independence, each term is small and the total is small. When they diverge, supporting association, the squared differences grow and the total balloons. You reject the null hypothesis only when the statistic exceeds the upper-tail critical value.

Working through the crimes table cell by cell gives the following contributions.

Summing the contributions gives the test statistic.

The degrees of freedom are (3 minus 1) times (3 minus 1), which is 4. The 5% critical value on 4 degrees of freedom is 9.488, and since 23.16 comfortably exceeds it, the null hypothesis is rejected. The 1% critical value is 13.28, and 23.16 exceeds that too, so the result is highly significant. There is strong evidence of an association between area and crime type. Minor rounding differences are normal here, so a value of 23.13 from a slightly different rounding path leads to exactly the same conclusion.

The Chi-Squared Distribution

A few properties of the chi-squared distribution explain why the test behaves as it does. It is defined only for positive values, because it is built from squared quantities. It is positively skewed at every degree of freedom, though it grows more symmetric as the degrees of freedom increase. Its shape depends entirely on the single parameter, the degrees of freedom, which also fixes its centre and spread, since the distribution has mean equal to the degrees of freedom and variance equal to twice the degrees of freedom. Critical values come from the upper-tail probabilities in the statistical tables.

Degrees of Freedom for a Table

For a table with r rows and c columns, the degrees of freedom are given by a simple product.

The intuition is about constraints. You begin with r times c cells, but computing the expected frequencies uses up information: one degree of freedom goes to the grand total, (r minus 1) to the row totals, and (c minus 1) to the column totals, and what remains is the product above. A two-by-two table therefore has 1 degree of freedom, a two-by-three table has 2, and a three-by-three table has 4.

Interpreting the Association

Rejecting the null hypothesis tells you that an association exists, but not where it lives. To describe it, look at which cells contribute most to the test statistic, since the largest values of the squared standardised difference are the ones driving the result. A cell where the observed count exceeds the expected is over-represented, and one where the observed falls short is under-represented. In the crimes example, the biggest contributors are area A burglary at 7.43 with more burglaries than expected, area A car theft at 5.12 with fewer than expected, and area C car theft at 4.87 with more than expected. The story the numbers tell is that burglary is disproportionately concentrated in area A while car theft is disproportionately concentrated in area C.

Goodness-of-Fit Tests

The same machinery answers a related question: does a sample come from a particular distribution? The most common case tests against the discrete uniform distribution, where each of k categories is assumed equally likely. The null hypothesis is that the data follow the specified distribution, with all categories equally likely, and the alternative is that they do not. For the uniform case the expected frequency in each category is the sample size spread evenly across the categories.

One condition matters before you trust the result. The chi-squared approximation is only valid when every expected frequency is at least 5, and if any expected count falls below that, you merge adjacent categories until the condition holds. The test statistic has the familiar form, now summed over the k categories.

Here the degrees of freedom are k minus 1, with the single lost degree coming from the constraint that the observed and expected counts both sum to n, and as before the test is upper-tailed.

Suppose 33 consumers each choose between three sweet wrappers, with counts of 8 for wrapper A, 10 for B, and 15 for C, and we want to test whether the three are equally preferred. The null hypothesis is equal preference across all three wrappers. With k equal to 3 and n equal to 33, the expected frequency for each wrapper is 33 divided by 3, which is 11, comfortably above the threshold of 5. The test statistic follows.

The degrees of freedom are k minus 1, which is 2. The 5% critical value on 2 degrees of freedom is 5.991, and since 2.364 falls well below it, the null hypothesis is not rejected. Even at the more lenient 10% level, where the critical value is 4.605, the statistic does not reach significance. The conclusion is that there is no evidence of any preference between the wrappers.

A Full Worked Example

To bring the test of association together, consider gender against party identification for 795 people.

The null hypothesis is no association between gender and party identification, against an alternative that an association exists. Computing the expected frequencies from the row and column totals divided by 795 gives the following.

The cell contributions then follow.

Summing them gives the test statistic.

With (2 minus 1) times (3 minus 1), or 2 degrees of freedom, the 5% critical value is 5.991 and the 1% critical value is 9.210. The statistic of 44.70 dwarfs both, so the result is highly significant. Reading the largest contributors, males are disproportionately Republican while females are disproportionately Democrat.

I prepared an article with 10 exercises for this lesson here: https://datalad.co.uk/chi-squared-test-workbook-10-exercises-with-full-solutions/

Conclusion

The chi-squared test turns a question about categorical relationships into a comparison between what you observed and what independence would predict. Build the contingency table and total it, compute each expected frequency as the row total times the column total over the grand total, and sum the standardised squared differences to get the statistic. For a test of association the degrees of freedom are (r minus 1) times (c minus 1), while for a goodness-of-fit test against a uniform distribution they are k minus 1, and either way the test is upper-tailed against a value from the tables. Keep an eye on the rule that every expected frequency should be at least 5, and once you reject the null, read the largest cell contributions to say not just that an association exists but exactly where it lies.