Non-parametric Tests

Non-parametric tests avoid specifying a full parametric family, but they are not assumption-free. Independence or correct pairing, the sampling design, exchangeability for permutation arguments, and test-specific conditions still matter. Rank tests often target equality of distributions; interpreting them purely as median tests requires additional shape assumptions. This article explores the Mann-Whitney U, Wilcoxon signed-rank, and Kruskal-Wallis H tests with those distinctions explicit.

Understanding Non-parametric Tests

What Are Non-parametric Tests?

Non-parametric tests are statistical methods that do not assume a specific distribution for the data. Unlike parametric tests (such as T-tests or ANOVA), which assume that the data follows a normal distribution, non-parametric tests can be used with data that is skewed, ordinal, or has outliers.

When to Use Non-parametric Tests

Non-parametric tests are appropriate in the following situations:

• Non-normal Distribution: When the data is not normally distributed and cannot be transformed to normality.
• Ordinal Data: When the data is ordinal (ranked) rather than interval or ratio.
• Small Sample Sizes: When the sample size is too small to reliably estimate the parameters of a normal distribution.
• Presence of Outliers: When the data contains outliers that cannot be removed or transformed.
• Heterogeneity of Variance: When the variances across groups are not equal.

Common Non-parametric Tests

Mann-Whitney U Test

Overview

The Mann-Whitney U Test (closely related to the Wilcoxon rank-sum test) compares two independent distributions. A location or median interpretation requires similarly shaped distributions; in general it tests stochastic equality rather than only medians.

Example: Comparing Test Scores

Problem Setup:

Suppose you want to compare the test scores of two groups of students who used different study methods. The data is not normally distributed, so you decide to use the Mann-Whitney U Test. You have the following data:

• Group A: [85, 88, 78, 92, 85, 87, 90, 91]
• Group B: [78, 82, 85, 80, 88, 84, 86, 85]

Step 1: State the Hypotheses

• Null Hypothesis $H_0$: The distributions of the two groups are the same.
• Alternative Hypothesis $H_1$: The distributions of the two groups are different.

Step 2: Rank the Data

Rank all the scores from both groups together, from lowest to highest. If there are tied ranks, assign the average rank to each tied value.

Score	Group	Rank
78	A	1.5
78	B	1.5
80	B	3
82	B	4
84	B	5
85	A	7.5
85	A	7.5
85	B	7.5
85	B	7.5
86	B	10
87	A	11
88	A	12.5
88	B	12.5
90	A	14
91	A	15
92	A	16

Step 3: Calculate the U Statistic

First, calculate the sum of ranks for each group:

• $R_A = 1.5 + 7.5 + 7.5 + 11 + 12.5 + 14 + 15 + 16 = 85$
• $R_B = 1.5 + 3 + 4 + 5 + 7.5 + 7.5 + 10 + 12.5 = 51$

Calculate the U statistic for each group using the formula:

$$U_A = n_A \times n_B + \frac{n_A \times (n_A + 1)}{2} - R_A$$ $$U_A = 8 \times 8 + \frac{8 \times 9}{2} - 85 = 15$$ $$U_B = n_A \times n_B + \frac{n_B \times (n_B + 1)}{2} - R_B$$ $$U_B = 8 \times 8 + \frac{8 \times 9}{2} - 51 = 49$$

The Mann-Whitney U statistic is the smaller of $U_A$ and $U_B$:

$$U = 15$$

Step 4: Determine the P-value

Using the tie-corrected asymptotic method with continuity correction gives the following two-sided result (software may report $U_A=49$ instead of the equivalent smaller statistic 15):

• P-value: Approximately 0.0804

Since $0.0804 > 0.05$, we fail to reject the null hypothesis. The data do not provide sufficient evidence of a distributional difference at the 5% level; this does not establish equality.

An effect estimate such as rank-biserial correlation should be reported with its direction, convention, and interval. Avoid translating an approximate standardized statistic into universal “small/medium/large” labels without domain context.

When to Use the Mann-Whitney U Test

• Non-normal Data: When data is not normally distributed.
• Ordinal Data: When data is ranked or ordinal.
• Small Sample Sizes: When sample sizes are small and parametric assumptions are questionable.

Wilcoxon Signed-Rank Test

Overview

The Wilcoxon Signed-Rank Test tests paired differences using their signs and ranks. Under a symmetric shift model it tests whether the center of the difference distribution is zero; without symmetry it should not be described solely as a median test.

Example: Pre-Test and Post-Test Scores

Problem Setup:

Suppose you want to test whether a new teaching method improves students' test scores. You measure the scores before and after the intervention for the same group of students. The data is not normally distributed, so you decide to use the Wilcoxon Signed-Rank Test. You have the following data:

• Pre-Test Scores: [78, 82, 88, 85, 90, 87, 84, 89]
• Post-Test Scores: [82, 85, 90, 88, 92, 90, 86, 91]

Step 1: State the Hypotheses

• Null Hypothesis $H_0$: The median difference between the pairs is zero.
• Alternative Hypothesis $H_1$: The median difference between the pairs is not zero.

Step 2: Calculate the Differences and Rank Them

Calculate the differences between the pre-test and post-test scores for each student. Then, rank the absolute values of these differences. Ignore the pairs with zero differences.

Student	Pre-Test $X$	Post-Test $Y$	Difference $D = Y - X$	$\vert D \vert$	Rank	Sign
1	78	82	4	4	8	+
2	82	85	3	3	6	+
4	85	88	3	3	6	+
6	87	90	3	3	6	+
3	88	90	2	2	2.5	+
5	90	92	2	2	2.5	+
7	84	86	2	2	2.5	+
8	89	91	2	2	2.5	+

Step 3: Calculate the Test Statistic

Sum the ranks for the positive differences $W^+$ and for the negative differences $W^-$. In this example, all differences are positive.

$$W^+ = 8 + 6 + 6 + 6 + 2.5 + 2.5 + 2.5 + 2.5 = 36$$

Since there are no negative differences, $W^- = 0$. The test statistic is the smaller of $W^+$ and $W^-$:

$$W = 0$$

Step 4: Determine the P-value

Using a Wilcoxon Signed-Rank distribution table or statistical software, find the p-value corresponding to $W = 0$ with $n = 8$.

• Exact p-value: $0.00390625$ one-sided for a positive shift; $0.0078125$ two-sided.

Both values are below 0.05. For a direction chosen before seeing the data, the one-sided test supports a positive shift; the two-sided test supports a nonzero shift. The causal phrase “method has an effect” additionally requires a design that identifies causality.

All eight observed differences are positive, which is directly interpretable alongside the p-value. An effect estimate should also describe the magnitude in score units, not only rank evidence.

When to Use the Wilcoxon Signed-Rank Test

• Non-normal Data: When paired differences are not normally distributed.
• Ordinal Data: When dealing with ranked or ordinal paired data.
• Small Sample Sizes: When sample sizes are small and parametric assumptions are questionable.

Kruskal-Wallis H Test

Overview

The Kruskal-Wallis H Test compares the rank distributions of three or more independent groups. With similarly shaped distributions it can be interpreted as a location comparison; otherwise rejection may reflect shape or spread as well as medians.

Example: Comparing Exam Scores Across Three Teaching Methods

Problem Setup:

Suppose you want to compare the exam scores of students from three different teaching methods (A, B, C). The data is not normally distributed, so you decide to use the Kruskal-Wallis H Test. You have the following data:

• Method A: [85, 87, 90, 88, 86]
• Method B: [78, 82, 80, 85, 81]
• Method C: [92, 94, 89, 95, 93]

Step 1: State the Hypotheses

• Null Hypothesis $H_0$: The distributions of the three groups are the same.
• Alternative Hypothesis $H_1$: At least one group has a different distribution.

Step 2: Rank the Data

Rank all the scores from all groups together, from lowest to highest. If there are tied ranks, assign the average rank to each tied value.

Score	Group	Rank
78	B	1
80	B	2
81	B	3
82	B	4
85	A	5.5
85	B	5.5
86	A	7
87	A	8
88	A	9
89	C	10
90	A	11
92	C	12
93	C	13
94	C	14
95	C	15

Step 3: Calculate the H Statistic

First, compute the group ranks and overall statistics:

• Group A:

  • $R_A = 5.5 + 7 + 8 + 9 + 11 = 40.5$
  • $n_A = 5$

• Group B:

  • $R_B = 1 + 2 + 3 + 4 + 5.5 = 15.5$
  • $n_B = 5$

• Group C:

  • $R_C = 10 + 12 + 13 + 14 + 15 = 64$
  • $n_C = 5$

• Overall:

  • $N = 15$
  • $\bar{R} = \frac{R_A + R_B + R_C}{k} = \frac{40.5 + 15.5 + 64}{3} = \frac{120}{3} = 40$

Calculate the Kruskal-Wallis H statistic:

$$H = \frac{12}{N(N + 1)} \sum_{i=1}^k \frac{R_i^2}{n_i} - 3(N + 1)$$

Substitute the values:

$$H = \frac{12}{15 \times 16} \left( \frac{40.5^2}{5} + \frac{15.5^2}{5} + \frac{64^2}{5} \right) - 3 \times 16$$

Calculate each term:

• $\frac{40.5^2}{5} = \frac{1640.25}{5} = 328.05$
• $\frac{15.5^2}{5} = \frac{240.25}{5} = 48.05$
• $\frac{64^2}{5} = \frac{4096}{5} = 819.2$

Sum them:

$$328.05 + 48.05 + 819.2 = 1195.3$$

Now, calculate H:

$$H_{\text{uncorrected}} = \frac{12}{240} \times 1195.3 - 48 = 11.765$$

Because score 85 is tied, apply the standard tie correction, giving $H\approx11.786$.

Step 4: Determine the P-value

Using a chi-square distribution table or statistical software with $k - 1 = 2$ degrees of freedom:

• P-value: Approximately 0.00276

Since $0.00276 < 0.05$, we reject the null hypothesis. At least one teaching-method distribution differs; adjusted post-hoc comparisons are required to locate the differences.

Effect Size:

Calculate eta-squared $\eta^2$:

$$\epsilon^2 = \frac{H-k+1}{N-k} = \frac{11.786-3+1}{15-3} \approx 0.816$$

This sample estimate is large, but it is uncertain with only five observations per group. A significant omnibus result does not identify which groups differ; adjusted post-hoc comparisons are needed.

When to Use the Kruskal-Wallis H Test

• Non-normal Data: When data is not normally distributed.
• Ordinal Data: When data is ranked or ordinal.
• Multiple Independent Groups: When comparing three or more independent groups.

Summary of Non-parametric Tests

Test	Use Case	Null Hypothesis	Assumptions	Parametric Alternative
Mann-Whitney U Test	Compare two independent distributions	The two distributions are the same	Independent observations; similar shapes for a pure location interpretation	Independent Two-Sample T-test
Wilcoxon Signed-Rank Test	Test a paired location shift	The distribution of paired differences is symmetric about zero	Independent pairs; symmetric differences for a median-shift interpretation	Paired T-test
Kruskal-Wallis H Test	Compare three or more independent distributions	The group distributions are the same	Independent observations; similar shapes for a location interpretation	One-Way ANOVA

Best Practices for Applying Non-parametric Tests

1. Check for Normality: Before choosing a non-parametric test, use diagnostic plots (e.g., Q-Q plots) or statistical tests (e.g., Shapiro-Wilk test) to assess whether your data meets the assumptions of normality. If not, proceed with a non-parametric test.

2. Rank Data Appropriately: For non-parametric tests like the Mann-Whitney U Test and Kruskal-Wallis H Test, ensure that the data is ranked correctly, and handle ties by assigning average ranks.

3. Consider Sample Size: While non-parametric tests are robust to violations of assumptions, they may have lower power than parametric tests, especially with small sample sizes. Consider the implications of this trade-off.

4. Choose the estimand first: Rank tests are not merely a last resort. Select a test whose null hypothesis and effect measure answer the scientific question; a transformation changes the estimand and should not be used only to chase normality.

5. Effect Sizes: In addition to p-values, report a compatible effect estimate and interval, such as a probability-of-superiority/rank-biserial measure for Mann-Whitney or an epsilon-squared measure for Kruskal-Wallis. Cohen's $d$ targets a standardized mean difference and is not the rank-test effect size.

6. Interpret Results Carefully: Non-parametric tests often test for differences in distributions rather than just differences in medians. Be cautious in interpreting the results, especially in terms of what aspect of the distribution (e.g., median, shape) is being tested.

Conclusion

Non-parametric tests provide valuable tools for analyzing data that do not meet the assumptions required for parametric tests. By understanding when and how to apply methods like the Mann-Whitney U Test, Wilcoxon Signed-Rank Test, and Kruskal-Wallis H Test, you can handle a wide range of data analysis scenarios where traditional parametric approaches may fail.

These tests allow you to draw meaningful conclusions from data that is skewed, ordinal, or contains outliers, providing flexibility and robustness in your statistical analyses. By following best practices and carefully interpreting the results, you can ensure that your non-parametric tests yield reliable and insightful findings.