### Tutorial Files

Before we begin, you may want to download the sample data (.csv) used in this tutorial. Be sure to right-click and save the file to your R working directory. This dataset contains a hypothetical sample of 27 participants who are divided into three stress reduction treatment groups (mental, physical, and medical) and three age groups (young, mid, and old). The stress reduction values are represented on a scale that ranges from 0 to 10. This dataset can be conceptualized as a comparison between three stress treatment programs, one using mental methods, one using physical training, and one using medication, across three age groups. The stress reduction values represent how effective the treatment programs were at reducing participant's stress levels, with higher numbers indicating higher effectiveness. Note that the numbers in this dataset are not very realistic and are simply used to make this example possible.### Beginning Steps

To begin, we need to read our dataset into R and store its contents in a variable.

- > #read the dataset into an R variable using the read.csv(file) function
- > dataTwoWayComparisons <- read.csv("dataset_ANOVA_TwoWayComparisons.csv")
- > #display the data
- > dataTwoWayComparisons

The first ten rows of our dataset.

### Omnibus Test

Let's run a general omnibus test to assess the main effects and interactions present in the dataset.

- > #use anova(object) to test the omnibus hypothesis
- > #Are main effects or interaction effects present in the independent variables?
- > anova(lm(StressReduction ~ Treatment * Age, dataTwoWayComparisons))

The omnibus ANOVA test

### Pairwise Comparisons

Since the omnibus test was significant for both variables and no interaction effect was present, we can proceed to testing the main effect pairwise comparisons. To accomplish this, we will apply our pairwise.t.test() function to each of our independent variables. For more details on the pairwise.t.test() function, see the One-Way ANOVA with Pairwise Comparisons tutorial.

- > #use pairwise.t.test(x, g, p.adj) to test the pairwise comparisons between the treatment group means
- > #What significant differences are present amongst the treatment means?
- > pairwise.t.test(dataTwoWayComparisons$StressReduction, dataTwoWayComparisons$Treatment, p.adj = "none")
- > #use pairwise.t.test(x, g, p.adj) to test the pairwise comparisons between the age group means
- > #What significant differences are present amongst the age group means?
- > pairwise.t.test(dataTwoWayComparisons$StressReduction, dataTwoWayComparisons$Age, p.adj = "none")

Pairwise comparisons of treatment group means

Pairwise comparisons of age group means

Note that the desired p-adjustment method will vary by researcher, study, etc. Here, we will assume an alpha level of .05 for all tests, effectively making no adjustment for the family-wise Type I error rate.

These results indicate that there are are no statistically significant pairwise differences between the treatment groups and that all of the comparisons between age groups are statistically significant. The age group means are 8 for young, 5 for mid, and 2 for old. Consequently, we are inclined to conclude that, regardless of treatment, young patients are going to be most responsive, followed by middle aged patients, followed by older ones. However, there is insufficient support to differentiate between the effectiveness of the treatment methods themselves.

Thanks for this tutorial! Do you have any suggestions for the best way to handle pairwise comparisons when you have a significant interaction? If you had a significant interaction in the example above, is there some way to get R to produce simple main effects of Treatment at each level of Age? Would you need to set a family-wise error rate because you'd be conducting 3 tests in that case? Thanks again!

ReplyDeleteThanks, Bill. Yes, I have a tutorial that covers exactly the situation that you described. Use the Two-Way ANOVA with Interactions link from the Topics menu on the right-hand side of this blog.

ReplyDeleteThanks for the tutorial. Could you tell how you would do a complete pairwise comparison of all conditions. I mean not only within one factor.

ReplyDelete