What they actually assume, when they are the right choice, and why the Mann–Whitney test is not simply a comparison of medians.
Non-parametric tests do not assume the data follow a particular distribution, which makes them useful for ordinal data, small samples with clear non-normality, and variables with extreme values. They are not assumption-free, and that is the common misunderstanding. Most of them work on ranks, and interpreting the result correctly depends on conditions people rarely check — notably, the Mann–Whitney test only compares medians if the two distributions have the same shape.
The name refers to not assuming the data come from a distribution described by parameters such as a mean and standard deviation. It does not mean the tests have no assumptions, and treating them as a safe default whenever something looks non-normal is how they get misused.
Most still require:
And several carry a further condition about distribution shape that determines how the result may be interpreted — which is the subject of the next section, and the thing worth reading if you read nothing else here.
Almost every introductory source describes the Mann–Whitney U test as comparing the medians of two groups. That is only true under a condition that is rarely stated and frequently not met.
What the test actually assesses is whether values drawn from one group tend to be larger than values drawn from the other — whether one distribution is stochastically greater. It becomes a test of medians only when the two distributions have the same shape and spread, differing merely in location.
Where the shapes differ — one group tightly clustered and the other widely spread, or one skewed and the other symmetric — a significant result tells you the distributions differ in some way that makes one tend to produce larger values. It does not license a clean claim about medians, and the two groups can even have identical medians while the test is significant.
What to do about it is straightforward. Plot both distributions before running the test. If the shapes are reasonably similar, interpreting in terms of medians is defensible and you can say so. If they are not, word the conclusion in terms of one group tending to score higher, and describe both distributions so the reader can see what is going on. The same logic applies to the Kruskal–Wallis test across three or more groups.
This is exactly the kind of point an examiner in a quantitative discipline may probe, and handling it correctly in one sentence marks out a candidate who understands the test rather than having selected it from a decision table.
| Design | Parametric test | Non-parametric equivalent |
|---|---|---|
| Two independent groups | Independent-samples t-test | Mann–Whitney U |
| Same people, two occasions | Paired-samples t-test | Wilcoxon signed-rank |
| Three or more independent groups | One-way ANOVA | Kruskal–Wallis |
| Same people, three or more occasions | Repeated-measures ANOVA | Friedman |
| Association between two variables | Pearson correlation | Spearman’s rho, or Kendall’s tau |
| Two categorical variables, small counts | Chi-square | Fisher’s exact test |
Two notes. Kendall’s tau is often preferable to Spearman’s rho with small samples or many tied ranks. And as with parametric tests, a significant Kruskal–Wallis or Friedman result tells you the groups differ somewhere without saying where, so it needs following up with appropriate pairwise comparisons and a correction for the number of them.
Non-parametric tests are the right answer when:
They are not free, though, and the costs should inform the decision.
They usually have less power when parametric assumptions hold. Converting values to ranks discards information about how far apart they were, so if a parametric test was appropriate, using a non-parametric one instead makes you less likely to detect a real effect.
They do not extend easily to complex designs. There is no straightforward non-parametric equivalent of a factorial ANOVA with interactions, a multiple regression with covariates, or a multilevel model. Studies with anything beyond a simple design usually need another route.
Effect sizes are less familiar. They exist — rank-biserial correlation, or r derived from the test statistic and sample size — and should still be reported, but they are less widely recognised and worth defining briefly when you use them.
Because of those costs, three alternatives are often better than switching to a rank-based test: transforming the variable and reporting that you did; using a robust method such as a t-test that does not assume equal variances; or bootstrapping, which builds an empirical sampling distribution from your own data and keeps you within the parametric framework while making no distributional assumption. Bootstrapping in particular is available in standard software and is under-used in doctoral work.
Report the test statistic, the sample sizes, the exact p-value and an effect size — the same standard applied to any other analysis.
Two additions specific to these tests. Give medians and interquartile ranges rather than means and standard deviations, since the whole reason for using a rank-based test is usually that the mean was not a good summary. And say something about distribution shape, because that is what determines how your result may be interpreted — a sentence noting that the two distributions were similarly shaped, with a figure to support it, is what licenses a conclusion about medians.
Also state why you chose the non-parametric test. “The data were not normally distributed” is thin on its own, since the relevant assumption often concerns residuals or sampling distributions rather than the raw variable, and parametric tests tolerate a good deal. A stronger sentence names what you examined, what you found, and why it mattered at your sample size.
Tests that do not assume the data come from a specific distribution such as the normal. Most work by converting values to ranks, which makes them suitable for ordinal data and resistant to extreme values. They are not assumption-free: they still require independent observations, an appropriate level of measurement, and defensible sampling if you intend to generalise.
Only when the two distributions have the same shape and spread. What it actually tests is whether values from one group tend to be larger than values from the other. Where the shapes differ, a significant result means the distributions differ in a way that makes one tend to produce larger values — and the groups can even share a median while the test is significant. Plot both distributions, and word your conclusion accordingly.
When the outcome is genuinely ordinal, when the sample is small and clearly non-normal, or when genuine extreme values are distorting the means. Bear in mind the cost: ranking discards information, so if parametric assumptions did hold you are less likely to detect a real effect. Before switching, consider a transformation, a robust test that does not assume equal variances, or bootstrapping — all of which keep more information.
Usually, when the parametric assumptions actually hold, because converting values to ranks discards information about how far apart they were. When those assumptions are clearly violated the position reverses and the rank-based test can be the more trustworthy one. The practical implication is not to use them as a precaution — check what the assumption actually requires first, since many parametric tests are more tolerant than people assume.
Yes, and you should. The rank-biserial correlation works for Mann–Whitney and Wilcoxon tests, and an r value can be derived from the test statistic and sample size. Report medians and interquartile ranges alongside them rather than means and standard deviations, since the reason for using the test was usually that the mean was a poor summary. Define the effect size briefly, as these are less familiar to some readers.
Not in a straightforward form, and this is one of the main practical limits. There is no simple rank-based equivalent of a factorial design with interactions, a multiple regression with covariates, or a multilevel model. For designs of that kind the usual routes are transformation, robust estimation methods, bootstrapping, or a generalised model with a distribution suited to your outcome — ordinal regression, for instance, where the outcome is ordered categories.
Switching to ranks is often a precaution against an assumption the test never made, and it costs power. Describe your data and design, and a PhD in your field will tell you what the analysis genuinely requires.
Leave an email or a WhatsApp number — whichever you prefer — and tick how we should reach you. We reach out within 30 minutes.
