Chi-Square Test: Complete Guide

What it answers

The chi-square test (written as χ² or chi-squared) is a non-parametric statistical test used for categorical data — data that falls into distinct categories or groups. Unlike t-tests and ANOVA, which compare means of continuous variables, chi-square tests work with counts or frequencies. This makes chi-square the go-to test when your data consists of categories: yes/no, red/blue/green, Democrat/Republican/Independent, etc.

There are two main types of chi-square tests, each answering a different question:

• Chi-square test of independence: "Are two categorical variables related?" For example: Is gender associated with voting preference? Is treatment type related to recovery outcome? Is smoking status linked to lung cancer diagnosis?
• Chi-square goodness-of-fit test: "Does a single categorical variable follow a specific distribution?" For example: Do M&M color proportions match the manufacturer's claim? Is a die fair? Does a genetic cross produce offspring in the expected Mendelian ratio?

A concrete example — test of independence: A researcher surveys 200 people about their preferred exercise type (running, swimming, cycling) and age group (under 30, 30-50, over 50). The question is: does exercise preference depend on age? The data is organized into a contingency table:

• Under 30: 30 running, 20 swimming, 10 cycling
• 30-50: 15 running, 35 swimming, 20 cycling
• Over 50: 10 running, 25 swimming, 35 cycling

The chi-square test will determine whether the pattern of exercise preferences differs across age groups, or whether the differences are small enough to be explained by random chance. If significant, it suggests age and exercise preference are associated — older people tend to prefer different exercises than younger people.

A concrete example — goodness-of-fit: A candy company claims their bag contains 30% chocolate, 30% caramel, and 40% nuts. You buy a bag, count 100 pieces, and find: 25 chocolate, 35 caramel, 40 nuts. Is the distribution consistent with the claim? The chi-square goodness-of-fit test compares your observed counts to the expected counts (30, 30, 40) to determine if the deviation is statistically significant.

Why chi-square matters: Many real-world questions involve categorical data. Medical research often classifies patients as recovered/not recovered. Marketing studies classify customers as responders/non-responders. Social science research classifies survey responses by category. Chi-square provides a rigorous way to test hypotheses about these categorical variables without assuming normal distributions or equal variances. It is one of the most versatile and widely applicable statistical tests.

The chi-square distribution: The chi-square statistic follows a chi-square distribution, which is right-skewed and depends on degrees of freedom. For a goodness-of-fit test with k categories, df = k − 1. For a test of independence with an r × c contingency table, df = (r − 1) × (c − 1). As degrees of freedom increase, the chi-square distribution becomes more symmetric and approaches a normal distribution.

The formula

The chi-square statistic measures the total discrepancy between observed and expected counts across all categories:

Key properties of the formula:

• Squaring the differences: (O − E)² ensures that both over-represented and under-represented categories contribute positively to the statistic. Without squaring, positive and negative deviations would cancel out.
• Dividing by expected counts: This standardizes the contribution of each category. A deviation of 5 matters more when you expected 10 than when you expected 100. Dividing by E gives appropriate weight to each category.
• Summing across all categories: The total chi-square statistic aggregates evidence from every category into a single number.

How expected counts are calculated:

For a goodness-of-fit test:

For a test of independence:

Worked example — test of independence: A study examines whether smoking status (smoker/non-smoker) is related to lung disease (disease/no disease):

Step 1 — Calculate expected counts:

• E(Smoker, Disease) = (100 × 60) / 200 = 30
• E(Smoker, No Disease) = (100 × 140) / 200 = 70
• E(Non-smoker, Disease) = (100 × 60) / 200 = 30
• E(Non-smoker, No Disease) = (100 × 140) / 200 = 70

Step 2 — Calculate chi-square:

• χ² = (40−30)²/30 + (60−70)²/70 + (20−30)²/30 + (80−70)²/70
• χ² = 100/30 + 100/70 + 100/30 + 100/70
• χ² = 3.33 + 1.43 + 3.33 + 1.43 = 9.52

Step 3 — Find degrees of freedom and p-value:

• df = (2 − 1) × (2 − 1) = 1
• p-value = 0.002 (using chi-square distribution with 1 df)

Since p = 0.002 < 0.05, we reject the null hypothesis of independence and conclude that smoking status is significantly associated with lung disease.

Worked example — goodness-of-fit: A geneticist predicts a dihybrid cross will produce offspring in a 9:3:3:1 ratio. Observed: 926, 288, 293, 104 (total = 1311). Expected: 737.3, 245.8, 245.8, 81.9. Chi-square = (926−737.3)²/737.3 + (288−245.8)²/245.8 + (293−245.8)²/245.8 + (104−81.9)²/81.9 = 48.1 + 7.2 + 9.1 + 6.0 = 70.4. With df = 3, p < 0.001. The observed ratio does not match 9:3:3:1, suggesting the genes may not assort independently.

Reading the result

Interpreting a chi-square test involves examining the statistic, degrees of freedom, p-value, and effect size:

• p < 0.05: The observed distribution differs significantly from the expected distribution. For a test of independence, this means the two variables are associated (not independent). For goodness-of-fit, the data does not match the hypothesized distribution.
• p ≥ 0.05: No significant evidence of an association or deviation from the expected distribution. This does not prove independence — it means the study did not find sufficient evidence to conclude an association exists.
• Chi-square statistic: Larger values indicate greater discrepancy between observed and expected. However, the absolute value depends on sample size and degrees of freedom, so always look at the p-value and effect size.

Post-hoc analysis — where are the differences? A significant chi-square tells you that an association exists, but not where. To identify which cells contribute most to the significance:

• Standardized residuals: Calculate (O − E) / √E for each cell. Values beyond ±2 indicate cells that deviate substantially from expectations. These are the cells driving the significant result.
• Adjusted standardized residuals: A refined version that accounts for the table structure. Values beyond ±2.58 (or ±1.96 for α = 0.05) are significantly different from expected.
• Examine cell-by-cell contributions: Calculate (O − E)²/E for each cell. The cells with the largest contributions are the ones most responsible for the significant result.

Effect size — how strong is the association? A significant p-value tells you an association exists but not how strong it is. Report effect size alongside the chi-square statistic:

• Phi (φ) for 2×2 tables: φ = √(χ² / n). Ranges from 0 (no association) to 1 (perfect association). Benchmarks: 0.1 = small, 0.3 = medium, 0.5 = large. In the smoking example above, φ = √(9.52/200) = 0.22, a small-to-medium effect.
• Cramer's V for larger tables: V = √(χ² / [n × min(r−1, c−1)]). Ranges from 0 to 1, with the same benchmarks as phi. Use this for tables larger than 2×2.
• Odds ratio for 2×2 tables: OR = (a×d) / (b×c), where a,b,c,d are the four cells. An OR of 2.67 in our example means smokers are 2.67 times more likely to have lung disease than non-smokers.

How to report results (APA style): "A chi-square test of independence revealed a significant association between smoking status and lung disease, χ²(1, N = 200) = 9.52, p = .002, φ = .22. Smokers were more likely to have lung disease (40%) than non-smokers (20%)."

Yates' correction for continuity: For 2×2 tables, some statisticians apply Yates' correction, which subtracts 0.5 from the absolute value of each (O − E) difference before squaring. This makes the test more conservative (less likely to find significance). However, Yates' correction is controversial — many modern statisticians recommend against it because it can be overly conservative, especially with moderate sample sizes. Use the uncorrected chi-square unless your discipline specifically requires Yates' correction.

Assumptions

The chi-square test relies on several assumptions. Violating these can produce unreliable p-values and incorrect conclusions:

• Categorical data: The data must be counts (frequencies) in discrete categories, not continuous measurements. If your data is continuous (height, weight, time), use a t-test, ANOVA, or regression instead. Chi-square treats each observation as belonging to exactly one category.
• Independence of observations: Each observation must contribute to only one cell in the table. The same person cannot be counted twice. This is ensured through proper sampling design — for example, random sampling without replacement. If you have repeated measures or paired data, use McNemar's test instead of chi-square.
• Sufficient expected frequencies: The expected count in each cell should be at least 5. This is the most commonly cited rule. If expected counts are below 5, the chi-square approximation becomes unreliable. With very small expected counts (below 1), the test can be severely misleading.
• Adequate total sample size: Even if individual expected counts are low, the total sample size should generally be at least 20-40 for chi-square to be valid. Larger samples are always better.

What to do when assumptions are violated:

• Expected counts below 5: Use Fisher's exact test, which calculates exact probabilities rather than relying on the chi-square approximation. Fisher's test is valid for any sample size and is the default for small tables in most software. For 2×2 tables, Fisher's exact test is always available. For larger tables, use Fisher-Freeman-Halton exact test.
• Very small samples: Fisher's exact test is the appropriate choice. If even Fisher's test is questionable (very sparse tables), consider exact methods or combining categories (if theoretically justified).
• Dependent observations: Use McNemar's test for paired categorical data (e.g., before/after measurements on the same subjects). For repeated measures across multiple time points, use generalized estimating equations (GEE) or mixed models.
• Ordinal data: If your categorical variables have a natural order (rating scales, severity levels), consider the linear-by-linear association test (Mantel-Haenszel chi-square), which is more powerful than chi-square because it uses the ordering information.

The rule of expected counts — why it matters: The chi-square distribution is an approximation that becomes accurate when expected counts are large enough. With small expected counts, the actual distribution of the chi-square statistic deviates from the theoretical chi-square distribution, leading to incorrect p-values. The "expected count ≥ 5" rule is a guideline, not an absolute law. Some sources use a more stringent rule: no expected count below 1, and no more than 20% of expected counts below 5. Always check expected counts (most software reports them) before interpreting the chi-square result.

Pooling categories: When some categories have very low expected counts, you can sometimes combine adjacent categories to increase expected counts. For example, if a survey has "Strongly Agree," "Agree," "Neutral," "Disagree," "Strongly Disagree" and "Strongly Agree" has only 2 responses, you might combine it with "Agree." However, only do this if it makes theoretical sense — do not arbitrarily merge categories just to get significant results. Report the original and combined tables.