Non-parametric Tests

Non-parametric tests are statistical methods used when the assumptions required for parametric tests, such as normality and homogeneity of variance, are not met. These tests do not rely on the underlying distribution of the data, making them more flexible and robust for analyzing data that does not conform to the assumptions of traditional parametric tests. This article explores some of the most commonly used non-parametric tests, including the Mann-Whitney U Test, Wilcoxon Signed-Rank Test, and Kruskal-Wallis H Test. We will also discuss when and how to apply these tests, providing detailed examples for each.

Understanding Non-parametric Tests

What Are Non-parametric Tests?

Non-parametric tests are statistical methods that do not assume a specific distribution for the data. Unlike parametric tests (such as T-tests or ANOVA), which assume that the data follows a normal distribution, non-parametric tests can be used with data that is skewed, ordinal, or has outliers.

When to Use Non-parametric Tests

Non-parametric tests are appropriate in the following situations:

• Non-normal Distribution: When the data is not normally distributed and cannot be transformed to normality.
• Ordinal Data: When the data is ordinal (ranked) rather than interval or ratio.
• Small Sample Sizes: When the sample size is too small to reliably estimate the parameters of a normal distribution.
• Presence of Outliers: When the data contains outliers that cannot be removed or transformed.
• Heterogeneity of Variance: When the variances across groups are not equal.

Common Non-parametric Tests

Mann-Whitney U Test

Overview

The Mann-Whitney U Test (also known as the Wilcoxon rank-sum test) is a non-parametric test used to compare the medians of two independent groups. It is an alternative to the independent two-sample T-test when the assumption of normality is not met.

Example: Comparing Test Scores

Problem Setup:

Suppose you want to compare the test scores of two groups of students who used different study methods. The data is not normally distributed, so you decide to use the Mann-Whitney U Test. You have the following data:

• Group A: [85, 88, 78, 92, 85, 87, 90, 91]
• Group B: [78, 82, 85, 80, 88, 84, 86, 85]

Step 1: State the Hypotheses

• Null Hypothesis $H_0$: The distributions of the two groups are the same.
• Alternative Hypothesis $H_1$: The distributions of the two groups are different.

Step 2: Rank the Data

Rank all the scores from both groups together, from lowest to highest. If there are tied ranks, assign the average rank to each tied value.

Score	Group	Rank
78	A	1
78	B	1
80	B	3
82	B	4
84	B	5
85	A	6.5
85	A	6.5
85	B	6.5
85	B	6.5
86	B	10
87	A	11
88	A	12
88	B	12
90	A	14
91	A	15
92	A	16

Step 3: Calculate the U Statistic

First, calculate the sum of ranks for each group:

• $R_A = 1 + 6.5 + 6.5 + 11 + 12 + 14 + 15 + 16 = 82.0$
• $R_B = 1 + 3 + 4 + 5 + 6.5 + 6.5 + 10 + 12 = 48.0$

Calculate the U statistic for each group using the formula:

$$U_A = n_A \times n_B + \frac{n_A \times (n_A + 1)}{2} - R_A$$ $$U_A = 8 \times 8 + \frac{8 \times 9}{2} - 82 = 64 + 36 - 82 = 18$$ $$U_B = n_A \times n_B + \frac{n_B \times (n_B + 1)}{2} - R_B$$ $$U_B = 8 \times 8 + \frac{8 \times 9}{2} - 48 = 64 + 36 - 48 = 52$$

The Mann-Whitney U statistic is the smaller of $U_A$ and $U_B$:

$$U = 18$$

Step 4: Determine the P-value

Using a Mann-Whitney U distribution table or statistical software, find the p-value corresponding to $U = 18$ with $n_A = 8$ and $n_B = 8$.

• P-value: Approximately 0.12

Since $0.12 > 0.05$, we fail to reject the null hypothesis. There is no statistically significant difference in the distributions of the two groups at the 5% significance level.

Effect Size:

To complement the p-value, calculate the effect size using $r$:

$$r = \frac{Z}{\sqrt{N}}$$

Assuming the Z-score corresponding to $U = 18$ is approximately -1.31:

$$r = \frac{-1.31}{\sqrt{16}} = \frac{-1.31}{4} \approx -0.3275$$

This indicates a medium effect size.

When to Use the Mann-Whitney U Test

• Non-normal Data: When data is not normally distributed.
• Ordinal Data: When data is ranked or ordinal.
• Small Sample Sizes: When sample sizes are small and parametric assumptions are questionable.

Wilcoxon Signed-Rank Test

Overview

The Wilcoxon Signed-Rank Test is a non-parametric test used to compare the medians of two related samples or paired observations. It is an alternative to the paired T-test when the assumption of normality is not met.

Example: Pre-Test and Post-Test Scores

Problem Setup:

Suppose you want to test whether a new teaching method improves students' test scores. You measure the scores before and after the intervention for the same group of students. The data is not normally distributed, so you decide to use the Wilcoxon Signed-Rank Test. You have the following data:

• Pre-Test Scores: [78, 82, 88, 85, 90, 87, 84, 89]
• Post-Test Scores: [82, 85, 90, 88, 92, 90, 86, 91]

Step 1: State the Hypotheses

• Null Hypothesis $H_0$: The median difference between the pairs is zero.
• Alternative Hypothesis $H_1$: The median difference between the pairs is not zero.

Step 2: Calculate the Differences and Rank Them

Calculate the differences between the pre-test and post-test scores for each student. Then, rank the absolute values of these differences. Ignore the pairs with zero differences.

Student	Pre-Test $X$	Post-Test $Y$	Difference $D = Y - X$	$\vert D \vert$	Rank	Sign
1	78	82	4	4	1	+
2	82	85	3	3	2.5	+
4	85	88	3	3	2.5	+
6	87	90	3	3	2.5	+
3	88	90	2	2	6	+
5	90	92	2	2	6	+
7	84	86	2	2	6	+
8	89	91	2	2	6	+

Step 3: Calculate the Test Statistic

Sum the ranks for the positive differences $W^+$ and for the negative differences $W^-$. In this example, all differences are positive.

$$W^+ = 1 + 2 + 3 + 2.5 + 3 + 2.5 + 3 + 3 = 21$$

Since there are no negative differences, $W^- = 0$. The test statistic is the smaller of $W^+$ and $W^-$:

$$W = 0$$

Step 4: Determine the P-value

Using a Wilcoxon Signed-Rank distribution table or statistical software, find the p-value corresponding to $W = 0$ with $n = 8$.

• P-value: Approximately 0.031 (one-tailed), 0.062 (two-tailed)

Since $0.062 > 0.05$, we fail to reject the null hypothesis at the 5% significance level for a two-tailed test. However, for a one-tailed test (if you have a directional hypothesis), $0.031 < 0.05$, and you would reject the null hypothesis, suggesting that the teaching method has a significant effect.

Effect Size:

Calculate the effect size using $r$:

$$r = \frac{Z}{\sqrt{N}} \approx \frac{-2.45}{\sqrt{8}} \approx -0.867$$

This indicates a large effect size.

When to Use the Wilcoxon Signed-Rank Test

• Non-normal Data: When paired differences are not normally distributed.
• Ordinal Data: When dealing with ranked or ordinal paired data.
• Small Sample Sizes: When sample sizes are small and parametric assumptions are questionable.

Kruskal-Wallis H Test

Overview

The Kruskal-Wallis H Test is a non-parametric test used to compare the medians of three or more independent groups. It is an extension of the Mann-Whitney U Test to more than two groups and is an alternative to one-way ANOVA when the assumption of normality is not met.

Example: Comparing Exam Scores Across Three Teaching Methods

Problem Setup:

Suppose you want to compare the exam scores of students from three different teaching methods (A, B, C). The data is not normally distributed, so you decide to use the Kruskal-Wallis H Test. You have the following data:

• Method A: [85, 87, 90, 88, 86]
• Method B: [78, 82, 80, 85, 81]
• Method C: [92, 94, 89, 95, 93]

Step 1: State the Hypotheses

• Null Hypothesis $H_0$: The distributions of the three groups are the same.
• Alternative Hypothesis $H_1$: At least one group has a different distribution.

Step 2: Rank the Data

Rank all the scores from all groups together, from lowest to highest. If there are tied ranks, assign the average rank to each tied value.

Score	Group	Rank
78	B	1
80	B	2
81	B	3
82	B	4
85	A	5.5
85	B	5.5
86	A	7
87	A	8
88	A	9
89	C	10
90	A	11
92	C	12
93	C	13
94	C	14
95	C	15

Step 3: Calculate the H Statistic

First, compute the group ranks and overall statistics:

• Group A:

  • $R_A = 5.5 + 7 + 8 + 9 + 11 = 40.5$
  • $n_A = 5$

• Group B:

  • $R_B = 1 + 2 + 3 + 4 + 5.5 = 15.5$
  • $n_B = 5$

• Group C:

  • $R_C = 10 + 12 + 13 + 14 + 15 = 64$
  • $n_C = 5$

• Overall:

  • $N = 15$
  • $\bar{R} = \frac{R_A + R_B + R_C}{k} = \frac{40.5 + 15.5 + 64}{3} = \frac{120}{3} = 40$

Calculate the Kruskal-Wallis H statistic:

$$H = \frac{12}{N(N + 1)} \sum_{i=1}^k \frac{R_i^2}{n_i} - 3(N + 1)$$

Substitute the values:

$$H = \frac{12}{15 \times 16} \left( \frac{40.5^2}{5} + \frac{15.5^2}{5} + \frac{64^2}{5} \right) - 3 \times 16$$

Calculate each term:

• $\frac{40.5^2}{5} = \frac{1640.25}{5} = 328.05$
• $\frac{15.5^2}{5} = \frac{240.25}{5} = 48.05$
• $\frac{64^2}{5} = \frac{4096}{5} = 819.2$

Sum them:

$$328.05 + 48.05 + 819.2 = 1195.3$$

Now, calculate H:

$$H = \frac{12}{240} \times 1195.3 - 48 = \frac{14343.6}{240} - 48 \approx 59.765 - 48 = 11.765$$

Step 4: Determine the P-value

Using a chi-square distribution table or statistical software with $k - 1 = 2$ degrees of freedom:

• P-value: Approximately 0.002

Since $0.002 < 0.05$, we reject the null hypothesis. There is a statistically significant difference in the distributions of at least one teaching method.

Effect Size:

Calculate eta-squared $\eta^2$:

$$\eta^2 = \frac{H - k + 1}{N} = \frac{11.765 - 3}{15} \approx \frac{8.765}{15} \approx 0.5857$$

This indicates a large effect size.

When to Use the Kruskal-Wallis H Test

• Non-normal Data: When data is not normally distributed.
• Ordinal Data: When data is ranked or ordinal.
• Multiple Independent Groups: When comparing three or more independent groups.

Summary of Non-parametric Tests

Test	Use Case	Null Hypothesis	Assumptions	Parametric Alternative
Mann-Whitney U Test	Compare medians of two independent groups	The distributions of the two groups are the same	Independent observations	Independent Two-Sample T-test
Wilcoxon Signed-Rank Test	Compare medians of two related samples or paired observations	The median difference between pairs is zero	Paired observations	Paired T-test
Kruskal-Wallis H Test	Compare medians of three or more independent groups	The distributions of the groups are the same	Independent observations, similar shapes	One-Way ANOVA

Best Practices for Applying Non-parametric Tests

1. Check for Normality: Before choosing a non-parametric test, use diagnostic plots (e.g., Q-Q plots) or statistical tests (e.g., Shapiro-Wilk test) to assess whether your data meets the assumptions of normality. If not, proceed with a non-parametric test.

2. Rank Data Appropriately: For non-parametric tests like the Mann-Whitney U Test and Kruskal-Wallis H Test, ensure that the data is ranked correctly, and handle ties by assigning average ranks.

3. Consider Sample Size: While non-parametric tests are robust to violations of assumptions, they may have lower power than parametric tests, especially with small sample sizes. Consider the implications of this trade-off.

4. Use Non-parametric Tests as a Last Resort: Non-parametric tests should be used when parametric tests are not suitable due to violations of their assumptions. If possible, try to meet the assumptions of parametric tests through transformations or other data preprocessing techniques.

5. Effect Sizes: In addition to p-values, report effect sizes (e.g., Cohen’s d for Mann-Whitney U Test, eta-squared for Kruskal-Wallis H Test) to provide a measure of the magnitude of differences.

6. Interpret Results Carefully: Non-parametric tests often test for differences in distributions rather than just differences in medians. Be cautious in interpreting the results, especially in terms of what aspect of the distribution (e.g., median, shape) is being tested.

Conclusion

Non-parametric tests provide valuable tools for analyzing data that do not meet the assumptions required for parametric tests. By understanding when and how to apply methods like the Mann-Whitney U Test, Wilcoxon Signed-Rank Test, and Kruskal-Wallis H Test, you can handle a wide range of data analysis scenarios where traditional parametric approaches may fail.

These tests allow you to draw meaningful conclusions from data that is skewed, ordinal, or contains outliers, providing flexibility and robustness in your statistical analyses. By following best practices and carefully interpreting the results, you can ensure that your non-parametric tests yield reliable and insightful findings.