Skip to Main Content

Analyze Data: R and RStudio

Contributors: Lindsay Plater, Michelle Dollois, Angelica Nascimento de Oliveira and Riley Oremush

Chi-Square Test of Independence

The Chi-Square Test is used to test whether two categorical (nominal) variables are associated with each other. This test assumes that the observations are independent, and that the expected frequencies for each category should be at least 1 (Note: no more than 20% of the categories should have expected frequencies less than 5).

Note that this is a non-parametric test. There is no parametric version of a Chi-Square Test of Independence.

How to run a Chi-square test

To run a Chi-Square test of independence in R, start by creating a contingency table from your categorical data. If your data are in separate columns within the same dataframe, use the table() function to generate this table, summarizing the frequency distribution of the variables. Create the table: TABLE_NAME <- table(NAMEOFDATAFRAME$NAMEOFCOLUMN1, NAMEOFDATAFRAME$NAMEOFCOLUMN2). Next, print the contingency table to verify it looks correct using: print(TABLE_NAME). Then perform the Chi-Square Test of Independence using the chisq.test() function: CHI_TEST <- chisq.test(TABLE_NAME). Print the test result using print(CHI_TEST) to view the Chi-Square statistic, degrees of freedom, and p-value. Pro Tip: it is important to check both the actual and expected frequencies, as the Chi-Square Test assumes sufficient data in each cell.

Interpreting the Output

Running the above steps will generate the following output: a Pearson’s Chi-squared test table that tells you whether your categorical variables are independent (p > .05) or associated (p < .05), a crosstab table showing the actual combinations between the variables you selected (e.g., indicating how many observations of each combination were present in your data), and a crosstab table showing the expected combinations between the variables you selected (e.g., indicating how many of each combination would be expected in your data).

For the Pearson’s Chi-squared test table, the “X-squared” value is the test statistic (X2), and the p-value indicates statistical significance (p < .05 is generally considered statistically significant, which would indicate that the variables are associated).

The RStudio interface, showing code for a chi-square test using the chisq.test() function. The output is displayed in the console, showing a non-significant chi-square at p = .65, and the observed and expected frequency tables.

Effect size

The effect size for the Chi-square test is the Cramer’s V, which is a measure of association between two categorical variables.

To calculate Cramer’s V, use the cramersv() function from the confintr package to compute Cramer's V based on the result of your chi-square test: cramersv(CHI_TEST).

The RStudio interface, showing code for calculating the effect size for a chi-square test using the cramers_v_test() .

Spearman's Rank-Order Correlation

Spearman’s rank-order correlation is used to determine the strength and direction of a relationship of the rankings of two variables. The variables can be ordinal or continuous. This test does not assume the variables are normally distributed. However, the relationship between the ranked values should be monotonic (i.e., an increasing OR decreasing relationship; not increasing AND decreasing).

Note that this is a non-parametric test; you could / should use a Spearman’s rank-order correlation if the normality assumption has been violated for your Pearson correlation (i.e., the parametric equivalent). You can also use this test if you wish to conduct a correlation on ordinal data (note: Pearson’s would not be appropriate here).

How to run a Spearman correlation

To run a Spearman correlation, use the cor.test() function, passing it the two columns you wish to correlate. By default, this function will run a Pearson’s correlation, but you can specify a Spearman correlation by specifying "spearman" in the method argument. This test requires a dataframe with at least two columns of numeric data. Use: str(NAMEOFDATAFRAME) to verify the data type for your variables (the <num> label indicates a numeric variable).

There are two methods of providing the two columns: individually, or as a formula.

  1. When giving your variables individually, you provide the variable / column names separated by a comma: cor.test(NAMEOFDATAFRAME$NAMEOFCOLUMN1, NAMEOFDATAFRAME$NAMEOFCOLUMN2, method = "spearman"). The test will then pair up the entries in each column in the order they appear and run the test of correlation. This method is especially helpful when you want to correlate two sets of numbers that are not stored in the same variable.
  2. When using the formula method, following a tilde (~), provide the column names separated by a plus sign (+) and indicate in which dataframe they can be found: cor.test(~NAMEOFCOLUMN1 + NAMEOFCOLUMN2, data = NAMEOFDATAFRAME, method = "spearman"). This method is particularly helpful when the data you are correlating are in separate columns in the same dataframe.

The RStudio interface, showing code for a spearman correlation using the cor.test() function. The output is displayed in the console, showing a non-significant correlation at p = .92. A scatterplot is displayed, showing a weak positive correlation.

Interpreting the Output

Running the above steps will generate the following output:

A close-up of the spearman correlation output, indicating the name of the test, the test statistic S, the p-value, and the rho value.

  1. The name of the statistical test
  2. The test statistic (S) and its value
  3. A p-value indicating whether the test is significant (p < .05), or, said another way, whether the specified variables are correlated
  4. The correlation coefficient (rho; the Greek symbol is ρ) indicating the strength and direction of the relationship between the two variables.

Spearman’s rho can range from -1 (perfect negative) to +1 (perfect positive), and indicates the strength and direction of the relationship between the two variables. Here, we see a non-significant, weak positive correlation.
 

Wilcoxon signed-rank test

The Wilcoxon signed-rank test is used to determine whether the median of a single continuous variable differs from a specified constant (similar to a one-sample t-test) AND / OR whether the median of two continuous variables from the same group of participants differ (similar to a paired-samples t-test). Both versions of this test do not assume that the data are normally distributed.

Note that this is a non-parametric test; you could / should use the Wilcoxon signed-rank test if the normality assumption has been violated for your one-sample t-test or a paired-samples t-test (i.e., the parametric equivalents).

This test requires a dataframe with at least one ordinal or continuous variable column in it. Use glimpse(NAMEOFDATAFRAME) to see the structure of your data; the <dbl> label indicates a continuous variable, and the <fct> label indicates a categorical variable. If your column is categorical but isn't labeled as a factor (“<fct>”), use: NAMEOFDATAFRAME$NAMEOFCOLUMN <- as.factor(NAMEOFDATAFRAME$NAMEOFCOLUMN) to change it.

How to run a Wilcoxon signed-rank test (one sample t-test version)

To run a one-sample Wilcoxon signed-rank test, use the wilcox.test() function. This function requires two pieces of information: the data to be tested, and the constant to test them against. Provide the column that contains the data, and use the mu parameter to define the constant: wilcox.test(NAMEOFDATAFRAME$NAMEOFCOLUMN, mu = CONSTANT).

The code above will determine whether the median of the specified column of data statistically differs from the constant.

You can customize the test further by adding additional arguments. For instance, specify the alternative hypothesis using: alternative = “____”, selecting one of: "two.sided", "greater", or "less". Setting conf.int = TRUE and conf.level = 0.95 will include a confidence interval at the specified level.

The RStudio interface, showing code for a wilcoxon signed-rank (one sample) test using the wilcox.test() function. The output is displayed in the console, showing a significant test at p = .000000002.

Interpreting the Output (one sample t-test version)

Running the above steps will generate the following output:

A close-up of the output from the Wilcoxon signed rank test (one-sample), showing the V-statistic, significant p-value, 95% confidence interval, and sample estimate (i.e., the median of the group).

  1. The name of the statistical test
  2. The test statistic (V, i.e., sum of the ranks of the differences) and its value
  3. A p-value indicating whether the test is significant (p<.05), or, said another way, whether the specified variable differs from the specified constant
    1. Note: R often provides values in scientific notation. Here, p = 1.863e-09 = 0.000000001863
    2. Note: the “alternative hypothesis” line indicates the specified constant
  4. The median of the selected variable
  5. The 95% confidence interval (CI) on the median

Effect size

For non-parametric tests like the Wilcoxon signed-rank test, a commonly used effect size is r, which represents the strength of the association between the sample and the hypothesized median. Pro Tip: the effect size r for the Wilcoxon signed-rank test is not the same as Pearson's r.

The effect size r for the Wilcoxon signed-rank can be calculated in R by:

  1. writing out the formula
  2. using the wilcox_effsize() function from the rstatix package

The wilcox_effsize() function will also give a 95% confidence interval on the effect size. When using the wilcox_effsize() function with a one-sample comparison, provide the dataframe and column that contains the data, use the mu parameter to define the constant, use alternative to specify the alternative hypothesis, and use ci to request confidence intervals for the effect size: wilcox_effsize(NAMEOFDATAFRAME, NAMEOFCOLUMN ~ 1, mu = 0, alternative = “two.sided”, ci = TRUE).

The RStudio interface, showing code for the effect size calculation of a wilcoxon signed-rank (one sample) test using the wilcox_effsize() function. The output is displayed in the console, showing a large effect size (0.873).

How to run a Wilcoxon Signed-Rank Test (paired samples t-test version)

To perform a paired Wilcoxon Signed-Rank Test in R (equivalent to a paired-samples t-test), you can use the wilcox.test() function, which is included in base R (i.e., you do not need to install or load any packages). There are two ways of conducting this test in RStudio, and the most appropriate test depends on the way your data are organized.

Method 1: Separate Columns for Each Condition

If the dependent variable data for each condition are in separate columns or separate variables in the same dataframe (i.e., wide format):

wilcox.test(NAMEOFDATAFRAME$CONDITION1, NAMEOFDATAFRAME$CONDITION2, paired = TRUE

Note: This test requires a dataframe with at least two columns of numeric data. Use str(NAMEOFDATAFRAME) to see the structure of your data; the label <num> indicates a numeric variable.

Note: it is important that TRUE is written in all capital letters for this code to run properly.

Note: The order of groups is important. If you get zero as your V statistic, change the order of groups, e.g.: wilcox.test(NAMEOFDATAFRAME$CONDITION2, NAMEOFDATAFRAME$CONDITION1, paired = TRUE).

The RStudio interface, showing code for a wilcoxon signed-rank (two sample) test using the wilcox.test() function.

Interpreting the Output

Running the above steps will generate the output above:

A close-up of the output from the Wilcoxon signed rank test (two-sample), showing the V-statistic and significant p-value.

  1. The name of the statistical test
  2. The test statistic (V) and its value
  3. A p-value indicating whether the test is significant (p<.05), or, said another way, whether the median differed between condition 1 and condition 2
    1. Note: R often provides values in scientific notation. Here, p = 1.869e-09 = 0.000000001863

Method 2: Conditions are in the Same Column

If the dependent variable data for both conditions are in the same column, with the condition labels in another column of the same data frame (i.e., long format):

wilcox_test(NAMEOFDVCOLUMN ~ NAMEOFIVCOLUMN, data = NAMEOFDATAFRAME , paired = TRUE)

Note: This test requires a dataframe with at least one column of numeric data (your DV) and one column of categorical data (your IV). Use str(NAMEOFDATAFRAME) to see the structure of your data; the label <num> indicates a numeric variable, the label <fct> indicates a categorical variable (in this case, it should shows “Factor w/ 2 levels” because the factor has two levels).

Note: If you get zero as your V statistic, change the order of the groups using: NAMEOFDATAFRAME $NAMEOFIVCOLUMN <- relevel(NAMEOFDATAFRAME$NAMEOFIVCOLUMN, ref = "NAMEOFGROUP2").

The RStudio interface, showing code for a wilcoxon signed-rank (two sample) test using the wilcox.test() function.

Interpreting the Output

Running the above steps will generate the following output:

A close-up of the output from the Wilcoxon signed rank test (two-sample), showing the V-statistic and significant p-value.

In this tibble:

  1. y: This column indicates the dependent variable analyzed.
  2. group1: This column shows the first group.
  3. group2: This column shows the second group.
  4. n1: This column provides the number of observations in group1.
  5. n2: This column provides the number of observations in group2.
  6. statistic: This column reports the test statistic value.
  7. p: This column displays the p-value, indicating whether the test is significant (p<.05), or, said another way, whether the median differed between condition 1 and condition 2.

Effect size

To calculate the effect size, use the wilcox_effsize() function with the same formula and paired setting:

wilcox_effsize(NAMEOFDVCOLUMN ~ NAMEOFIVCOLUMN, data = NAMEOFDATAFRAME.long, paired = TRUE).

The RStudio interface, showing code for the effect size calculation of a wilcoxon signed-rank (two sample) test using the wilcox_effsize() function.

Running the above steps will generate the following output:

A close-up of the output from the effect size calculation of the Wilcoxon signed rank test (two-sample), showing a large effect size (0.873).

In this tibble:

  1. y: This column shows the dependent variable analyzed.
  2. group1: This column indicates the first group.
  3. group2: This column indicates the second group.
  4. effsize: This column reports the effect size, r. This value represents the magnitude of the difference between the two paired samples.
  5. n1: This column provides the number of observations in group1.
  6. n2: This column provides the number of observations in group2.
  7. magnitude: This column describes the magnitude of the effect size, according to common interpretation guidelines.

Mann-Whitney test

The Mann-Whitney U test is used to determine whether two groups’ medians on the same continuous variable differ (similar to an independent samples t-test). This test does not assume that the data are normally distributed, but is does assume that the distributions are the same shape.

Note that this is a non-parametric test; you could / should use the Mann-Whitney U test if the normality assumption has been violated for your independent samples t-test (i.e., the parametric equivalent).

This test requires a dataframe with one column of numeric data (your DV) and one column of categorical data (your IV). Use str(NAMEOFDATAFRAME) to see the structure of your data; the label <num> indicates a numeric variable, the label <fct> indicates a categorical variable (in this case, it should shows “Factor w/ 2 levels” because the factor has two levels). Your IV variable must be categorical; if your IV column isn’t a factor (“<fct>”), use: NAMEOFDATAFRAME$NAMEOFCOLUMN <- as.factor(NAMEOFDATAFRAME$NAMEOFCOLUMN) to change it.

How to run a Mann-Whitney U test

To run a Mann-Whitney U test, use the wilcox_test() function from the rstatix package (Reminder: the data must be in long format for this test):

wilcox_test(NAMEOFDVCOLUMN ~ NAMEOFIVCOLUMN, data = NAMEOFDATAFRAME, paired = FALSE)

Note: it is important that FALSE is written in all capital letters for this code to run properly.

The RStudio interface, with code for a Mann-Whitney U test (also called the Wilcoxon Rank Sum Test) in the console using the wilcox_test() function.

Interpreting the Output

Running the above steps will generate the following output:

The output of the Mann-Whitney U test, showing a trending p-value (p = 0.0975).

In this tibble:

  1. y: This column indicates the dependent variable analyzed.
  2. group1: This column shows the first group.
  3. group2: This column shows the second group.
  4. n1: This column provides the number of observations in group1.
  5. n2: This column provides the number of observations in group2.
  6. statistic: This column reports the value of the test statistic.
  7. p: This column displays the p-value, indicating whether the test is significant (p<.05), or, said another way, whether the median differed between group1 and group2.

Effect size

To calculate the effect size for the Mann-Whitney U test, use the wilcox_effsize() function with the same formula and non-paired setting:

wilcox_effsize(NAMEOFDVCOLUMN ~ NAMEOFIVCOLUMN, data = NAMEOFDATAFRAME, paired = FALSE)

The RStudio interface, with code for a Mann-Whitney U test (also called the Wilcoxon Rank Sum Test) effect size in the console using the wilcox_effsize() function.

Running the above steps will generate the following output:

The output of the Mann-Whitney U test effect size, showing a moderate effect size (r = 0.307).

In this tibble:

  1. y: This column shows the dependent variable analyzed.
  2. group1: This column indicates the first group.
  3. group2: This column indicates the second group.
  4. effsize: This column reports the effect size, r. This value represents the magnitude of the difference between the two non-paired samples.
  5. n1: This column provides the number of observations in group1.
  6. n2: This column provides the number of observations in group2.
  7. magnitude: This column describes the magnitude of the effect size, according to common interpretation guidelines.

Visualizing results

Next, we can plot the data. The best visualization for a Man-Whitney U test is a boxplot for each group. We can plot this using the boxplot() function from base R:

boxplot(NAMEOFDVCOLUMN ~ NAMEOFIVCOLUMN, data = NAMEOFDATAFRAME)

The RStudio interface with code for a boxplot in the source window and a boxplot showing fake_data1 divided based on gender in the plots window. The male group has a slightly higher median (approximately 87.75) than the female group (approximately 86.75).

The dark bar in the middle of the box is the median, the edges of the box indicate the interquartile range (IQR; quartile 1 to quartile 3), the whiskers indicate the minimum and maximum (Q1 - 1.5*IQR and Q3 + 1.5*IQR, respectively), and any circles outside the whiskers indicate outliers in that group / condition.

Kruskal-Wallis test

The Kruskal-Wallis H test is used to determine whether three or more groups’ medians on the same continuous variable differ (similar to a one-way ANOVA, with independent groups). This test does not assume that the data are normally distributed, but it does assume the distributions are the same shape.

Note that this is a non-parametric test; you could / should use the Kruskal-Wallis H test if the normality assumption has been violated for your one-way ANOVA with independent groups (i.e., the parametric equivalent).

This test requires a dataframe with at one column of numeric data (your DV) and one column of categorical data (your IV). Use str(NAMEOFDATAFRAME) to see the structure of your data; the label <num> indicates a numeric variable, the label <fct> indicates a categorical variable (in this case, it should shows “Factor w/ 4 levels” because the factor has four levels). Your IV variable must be categorical; If your IV column isn’t a factor (“<fct>”), use: NAMEOFDATAFRAME$NAMEOFCOLUMN <- as.factor(NAMEOFDATAFRAME$NAMEOFCOLUMN) to change it.

How to run a Kruskal-Wallis H test

To run a Kruskal-Wallis H test, use the kruskal_test() function from the rstatix package (Reminder: the data must be in long format for this test):

kruskal_test(NAMEOFDVCOLUMN ~ NAMEOFIVCOLUMN, data = NAMEOFDATAFRAME)

The RStudio interface, with code for a Kruskal-Wallis H test in the console using the kruskal_test() function.

Interpreting the Output

Running the above steps will generate the following output:

The output of the Kruskal-Wallis H test, showing a non-significant p-value (p = 0.91).

In this tibble:

  1. y: This column indicates the dependent variable analyzed.
  2. n: This column provides the number of observations.
  3. statistic: This column reports the value of the test statistic.
  4. sf: This column reports the degrees of freedom.
  5. p: This column displays the p-value, indicating whether the test is significant (p<.05), or, said another way, whether the median differed between the different groups.
  6. method: This column reports the test name.

Effect size

To calculate the effect size for the Kruskal-Wallis H test, use the kruskal_effsize() function:

kruskal_effsize(NAMEOFDVCOLUMN ~ NAMEOFIVCOLUMN, data = NAMEOFDATAFRAME)

The RStudio interface, with code for a Kruskal-Wallis H test effect size in the console using the kruskal_effsize() function.

Running the above steps will generate the following output:

The output of the Mann-Whitney U effect size test, showing a moderate effect size (eta-squared = -0.0947).

  1. y: This column shows the dependent variable analyzed.
  2. n: This represents the number of observations (sample size) in the data used for the effect size calculation.
  3. effsize: This is the calculated effect size for the Kruskal-Wallis test, eta-squared. This value represents the magnitude of the difference between the groups.
  4. method: This column shows the type of effect size measure used.
  5. magnitude: This column describes the magnitude of the effect size, according to common interpretation guidelines.

Post-hoc tests

Typically, when a Kruskal-Wallis H test is non-significant, you would not run post hoc tests. If, however, the Kruskal-Wallis H test returned p < .05, this indicates a difference between groups somewhere. To determine where the difference(s) lies, we must run follow-up tests (generally post hoc tests). To perform post-hoc tests, use the wilcox_test() function to run pairwise comparisons across all levels of your independent variable:

wilcox_test(NAMEOFDVCOLUMN ~ NAMEOFIVCOLUMN, data = NAMEOFDATAFRAME, p.adjust.method = "bonferroni")

The RStudio interface, with code for Mann-Whitney U test (also called a Wilcoxon Rank Sum Test) post-hoc tests following a significant Kruskal-Wallis H test in the console. The Mann-Whitney U test is calculated using the wilcoxon_test() function.

The output from the wilcox_test() function presents the results of all combinations of Wilcoxon rank-sum tests (also called Mann-Whitney U tests).

The output of the post-hoc Mann-Whitney U test (following the Kruskal-Wallis H test), showing non-significant differences between all condition pairs (all ps > 0.529).

In this tibble:

  1. y: This column indicates the dependent variable analyzed.
  2. group1: This column shows the first group.
  3. group2: This column shows the second group.
  4. n1: This column provides the number of observations in group1.
  5. n2: This column provides the number of observations in group2.
  6. statistic: This column reports the value of the test statistic.
  7. p: The unadjusted p-value for the test statistic, indicating whether the test is significant (p<.05), or, said another way, whether the median differed between group1 and group2.
  8. p.adj: The adjusted p-value after applying the Bonferroni correction to account for multiple comparisons.
  9. p.adj.signif: This column indicates the significance of the adjusted p-value. One or more stars (*) indicate significance, and “ns” stands for “not significant”.

Effect size for the post-hoc tests

To calculate the effect sizes for pairwise comparisons following a Kruskal-Wallis test, use the wilcox_effsize() function from the rstatix package:

wilcox_effsize(NAMEOFDVCOLUMN ~ NAMEOFIVCOLUMN, data = NAMEOFDATAFRAME)

The RStudio interface, with code for the post-hoc Mann-Whitney U test (also called the Wilcoxon Rank Sum Test) effect size in the console using the wilcoxon_effsize() function.

Running the above steps will generate the following output:

The output of the post-hoc Mann-Whitney U test (following the Kruskal-Wallis H test) effect sizes, showing small effect sizes for all pairs (all rs < 0.200).

In this tibble:

  1. y: This column indicates the dependent variable analyzed.
  2. group1: This column shows the first group.
  3. group2: This column shows the second group.
  4. effsize: This column provides the effect size, r, for each pairwise comparison.
  5. n1: This column provides the number of observations in group1.
  6. n2: This column provides the number of observations in group2.
  7. magnitude: This column describes the magnitude of the effect size, according to common interpretation guidelines.

Visualizing results

Next, we can plot the data. The best visualization for a Kruskal-Wallis H test is a boxplot for each group. We can plot this using the boxplot() function from base R:

boxplot(NAMEOFDVCOLUMN ~ NAMEOFIVCOLUMN, data = NAMEOFDATAFRAME)

The RStudio interface, showing the code for a boxplot for the Kruskal-Wallis H test in the source window and the boxplot in the plots window. There do not appear to be differences in the medians between the four colour groups (all medians range between 86.5 - 87.5).

The dark bar in the middle of the box is the median, the edges of the box indicate the interquartile range (IQR; quartile 1 to quartile 3), the whiskers indicate the minimum and maximum (Q1 - 1.5*IQR and Q3 + 1.5*IQR, respectively), and any circles outside the whiskers indicate outliers in that group / condition.

Friedman Test

The Friedman test is used to determine whether one groups’ ranking on three or more continuous or ordinal variables differ (similar to a repeated measures one-way ANOVA). This test does not assume that the data are normally distributed, but it does assume the distributions are the same shape.

Note that this is a non-parametric test; you could / should use the Friedman test if the normality assumption has been violated for your repeated measures one-way ANOVA (i.e., the parametric equivalent).

This test requires a dataframe with one column of numeric data (your DV), one column of categorical data (your IV), and one column of identification (ID). The ID column is used to match observations across conditions; for example, if the data belong to participants, the ID column would label which observations belong to participant 1, participant 2, et cetera. There should be one value for each participant in each condition. Use str(NAMEOFDATAFRAME) to see the structure of your data; the label <num> indicates a numeric variable, the label <fct> indicates a categorical variable (in this case, the IV should show “Factor w/ 3 levels” because the factor has three levels). Your IV and ID variables must be categorical variables; If your IV and ID column aren’t factors (“<fct>”), use: NAMEOFDATAFRAME$NAMEOFCOLUMN <- as.factor(NAMEOFDATAFRAME$NAMEOFCOLUMN) to change them.

How to run a Friedman test

To run a Friedman test, use the friedman_test() function from the rstatix package (REMINDER: the data must be in long format for this test):

friedman_test(NAMEOFDVCOLUMN ~ NAMEOFIVCOLUMN | ID, data = NAMEOFDATAFRAME)

The RStudio interface, with code for a Friedman's test in the console using the friedman_test() function.

Interpreting the Output

Running the above steps will generate the following output:

The output of the Friedman test, showing a significant main effect (p = 0.0000000000000936).

In this tibble:

  1. y: This column indicates the dependent variable analyzed.
  2. n: This column provides the number of observations.
  3. statistic: This column reports the value of the test statistic.
  4. df: This column represents the degrees of freedom associated with the test.
  5. p: This column displays the p-value, indicating whether the test is significant (p<.05), or, said another way, whether the median differed between conditions.
  6. method: This column reports the test name.

Effect size

To calculate the effect size for the Friedman test, use the friedman_effsize() function:

friedman_effsize(NAMEOFDVCOLUMN ~ NAMEOFIVCOLUMN | ID, data = NAMEOFDATAFRAME)

The RStudio interface, with code for a Friedman test effect size in the console using the friedman_effsize() function.

Running the above steps will generate the following output:

The output of the Friedman test effect size, showing a large effect size (Kendall's W = 1).

  1. y: This column shows the dependent variable analyzed.
  2. n: This represents the number of observations (sample size) in the data used for the effect size calculation.
  3. effsize: This is the calculated effect size for the Friedman test, Kendall’s W. This value represents the magnitude of the difference between the conditions.
  4. method: This column shows the type of effect size measure used.
  5. magnitude: This column describes the magnitude of the effect size, according to common interpretation guidelines.

Post-hoc tests

Typically, when a Friedman’s test is non-significant, you would not run post hoc tests. If, however, the Friedman’s test returned p < .05, this indicates a difference between conditions somewhere. To determine where the difference(s) lies, we must run follow-up tests (generally post hoc tests). To perform post-hoc tests, use the wilcox_test() function to run pairwise comparisons across all levels of your independent variable:

wilcox_test(NAMEOFDVCOLUMN ~ NAMEOFIVCOLUMN, data = NAMEOFDATAFRAME, paired = TRUE, p.adjust.method = "bonferroni")

The RStudio interface, with code for a Wilcoxon Signed-Rank post-hoc test (following a significant Friedman test) in the console using the wilcox_test() function.

The output from the wilcox_test() function presents the results of all combinations of Wilcoxon signed-rank tests:

The output of the post-hoc Wilcoxon Signed Rank test (following the Friedman test), showing significant differences between all pairs of conditions (all ps < 0.001).

In this tibble:

  1. y: This column indicates the dependent variable analyzed.
  2. group1: This column shows the first group.
  3. group2: This column shows the second group.
  4. n1: This column provides the number of observations in group1.
  5. n2: This column provides the number of observations in group2.
  6. statistic: This column reports the value of the test statistic.
  7. p: This column displays the p-value, indicating whether the test is significant (p<.05), or, said another way, whether the median differed between group1 and group2.
  8. p.adj: The adjusted p-value after applying the Bonferroni correction to account for multiple comparisons.
  9. p.signif: This column indicates the significance of the adjusted p-value. One or more stars (*) indicate significance, and “ns” stands for “not significant”.

Visualizing results

Next, we can plot the data. The best visualization for a Kruskal-Wallis H test is a boxplot for each group. We can plot this using the boxplot() function from base R:

boxplot(NAMEOFDVCOLUMN ~ NAMEOFIVCOLUMN, data = NAMEOFDATAFRAME)

The RStudio interface, showing the code for a boxplot for the Friedman's test in the source window and the boxplot in the plots window. There appear to be large differences between the medians of the three groups of fake data (medians range from 87 - 114).

The dark bar in the middle of the box is the median, the edges of the box indicate the interquartile range (IQR; quartile 1 to quartile 3), the whiskers indicate the minimum and maximum (Q1 - 1.5*IQR and Q3 + 1.5*IQR, respectively), and any circles outside the whiskers indicate outliers in that group / condition.

Factorial ANOVA

Please note that there is no non-parametric alternative to a factorial ANOVA. If your factorial ANOVA does not meet the assumptions, you could try transforming your dependent variable data and running all assumptions again.

Suggest an edit to this guide

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.