By John M Quick

The R Tutorial Series provides a collection of user-friendly tutorials to people who want to learn how to use R for statistical analysis.


My Statistical Analysis with R book is available from Packt Publishing and Amazon.


R Tutorial Series: Two-Way ANOVA with Interactions and Simple Main Effects

When an interaction is present in a two-way ANOVA, we typically choose to ignore the main effects and elect to investigate the simple main effects when making pairwise comparisons. This tutorial will demonstrate how to conduct pairwise comparisons when an interaction is present in a two-way ANOVA.

Tutorial Files

Before we begin, you may want to download the sample data (.csv) used in this tutorial. Be sure to right-click and save the file to your R working directory. This dataset contains a hypothetical sample of 60 participants who are divided into three stress reduction treatment groups (mental, physical, and medical) and two gender groups (male and female). The stress reduction values are represented on a scale that ranges from 1 to 5. This dataset can be conceptualized as a comparison between three stress treatment programs, one using mental methods, one using physical training, and one using medication across genders. The values represent how effective the treatment programs were at reducing participant's stress levels, with higher numbers indicating higher effectiveness.

Beginning Steps

To begin, we need to read our dataset into R and store its contents in a variable.
  1. > #read the dataset into an R variable using the read.csv(file) function
  2. > dataTwoWayInteraction <- read.csv("dataset_ANOVA_TwoWayInteraction.csv")
  3. > #display the data
  4. > dataTwoWayInteraction

The first ten rows of our dataset.

Omnibus Test

Let's run a general omnibus test to assess the main effects and interactions present in the dataset.
  1. > #use anova(object) to test the omnibus hypothesis
  2. > #Are main effects or interaction effects present in the independent variables?
  3. > anova(lm(StressReduction ~ Treatment * Gender, dataTwoWayInteraction))

The omnibus ANOVA test

Divide the Data

The significant omnibus interaction suggests that we should ignore the main effects and instead investigate the simple main effects for our independent variables. To do so, we need to divide our dataset along each level of our treatment variable. We can create subsets of our dataset using the subset(data, condition) function, where data is the original dataset and condition contains the parameters defining the subset.
  1. > #use subset(data, condition) to divide the original dataset
  2. > #medical subset
  3. > dataMedical <- subset(dataTwoWayInteraction, Treatment == "medical")
  4. > #mental subset
  5. > dataMental <- subset(dataTwoWayInteraction, Treatment == "mental")
  6. > #physical subset
  7. > dataPhysical <- subset(dataTwoWayInteraction, Treatment == "physical")

Group ANOVAs

With datasets representing each of our treatment groups, we can now run an ANOVA for each that investigates the impact of gender. You may notice that this is effectively running three one-way ANOVAs with gender being the independent variable. Therefore, we should control for Type I error by dividing our typical .05 alpha level by three (.017).
  1. > #run ANOVA on the treatment subsets to investigate the impacts of gender within each
  2. > #medical
  3. > anova(lm(StressReduction ~ Gender, dataMedical))
  4. > #mental
  5. > anova(lm(StressReduction ~ Gender, dataMental))
  6. > #physical
  7. > anova(lm(StressReduction ~ Gender, dataPhysical))

The gender within treatment group ANOVA tests

At an alpha level of .017, the gender effect within the mental (p = .014) and physical (p < .001) groups was statistically significant. In the mental condition, the means are 3 for males and 4 for females. In the physical condition, the means are 4 for males and 2 for females. These results suggest that the mental treatment is more effective in reducing stress for females than males, while the physical treatment is more effective for males than females. Further, there is insufficient statistical support for a gender difference in the medical treatment.

Pairwise Comparisons

Note that since our gender variable contains only two levels, there is no need to conduct follow-up comparisons. However, should you reach this point with an independent variable of more than three levels, you could conduct pairwise comparisons in the same manner as demonstrated in the Two-Way ANOVA with Comparisons tutorial. In this case, remember to carry through your reduced Type I error rate from the preceding ANOVA tests.

Complete Two-Way ANOVA with Interactions Example

To see a complete example of how two-way ANOVA simple main effects can be explored in R, please download the two-way ANOVA interaction example (.txt) file.

10 comments:

  1. I'd strongly recommend adding two way table of means

    tapply(StressReduction,list(Gender,Treatment),mean)

    ReplyDelete
  2. Thanks for the tip. That is very useful.

    ReplyDelete
  3. It is not clear that you are using the model error for your SME analysis or your pairwise comparisons. With between subject factors in a two factor ANOVA, it is reasonable to use the total model error, and not just the error from the subsets. If the two factors were within subjects, I'd change my tune and do as your did, because assumptions of sphericity are needed.

    ReplyDelete
  4. Sorry for late response.
    Quinn & Keough (2002) recommend a simple main effects tests that instead use the MS resid from the full, 2way analysis. The logic is this: the best possible estimate of residual variance is given in the full model, using all the data. By splitting the data into 3, and using the MSresid from each of those smaller data sets, we lose power. Are you aware of any procedure in R that let us do such test. I.e. something like this:

    full.lm = lm(StressReduction ~ Treatment * Gender, dataTwoWayInteraction)
    medical.lm = MainEffect(full.lm, at=Treatment=="medical")
    mental.lm = MainEffect(full.lm, at=Treatment=="mental")
    physical.lm = MainEffect(full.lm, at=Treatment=="physical")

    So that each of the 3 models testing for the effects of sex, uses the MS res from the "full.lm" model.

    ReplyDelete
  5. This sounds like a good question for the R-Help listserv, assuming it hasn't been addressed already. You're more likely to find someone who is simultaneously an expert in the literature, methods, and code there.

    ReplyDelete
    Replies
    1. This is a really great tutorial. Thanks for putting it up. Is there anyway to compare all treatments and all genders to each other?

      Delete
    2. The ANOVA pairwise comparisons tutorial may be helpful to you, since it shows a number of methods for making comparisons. Links to all of the ANOVA tutorials can be found under the ANOVA heading on the right-hand side of this blog. Remember to keep track of your Type I error/significance threshold as you make many comparisons.

      Delete
  6. It would be nice to conduct the one-way anova for each gender separately, as well.

    ReplyDelete
  7. Hi John,

    Great tutorial. Is there any way to conduct t-tests of your contrasts (e.g., f vs m for medical; f vs m for mental; f vs m for physical) using the pairwise.t.test function so that you can adjust the p-values using Holm, FDR, etc corrections?

    ReplyDelete
  8. For anyone else who's finding this years later through Google searches, there's a new paper covering different approaches to post hoc analysis of factorial / two-way / between groups ANOVA.

    http://cran.r-project.org/web/packages/phia/vignettes/phia.pdf

    ReplyDelete