TwoWay ANOVA Post hoc analysis
If a significant main effect or interaction is found, then you can only conclude that there is a significant difference amongst the levels of your IV(s) somewhere. You still have to isolate exactly where the significant differences lie. If an IV has only two levels then the significant Fvalue is sufficient to tell you that the two levels are significantly different from each other. If, however, you have three or more levels for an IV you need to follow up the significant Fvalue with Tukey's HSD post hoc test. The method for doing this is the same as that encountered in the previous two chapters and is illustrated in the examples following.
If you find a significant interaction, you also need to follow up that finding with post hoc tests. Even though a graph nicely illustrates the relationship between the two IVs, a graph is open to "scale abuse". That is, by choosing an appropriate scale any difference can be made to look like a large difference on a graph. We need to have a numerical way of objectively deciding if a certain difference is actually significant or not. This is again demonstrated in the following examples. The main thing to remember is that when examining a significant interaction with post hoc tests, it is actually the individual cell means that are being compared. So that in a 2 X 2 ANOVA there are four cell means to be compared. In a 2 X 3 ANOVA, there are six cell means to be compared, etc. The number of means being compared is important for determining the qvalue in the HSD formula.
Also note that textbooks differ on the best approach to take in regard to post hoc tests and which post hoc test is most suitable for a between groups design versus a repeated measures design and what to do when assumptions are violated. Here we will use Tukey's HSD using a pooled error term (some texts disagree with this). This approach lacks some theoretical exactness but in terms of consistency and ease and breadth of application it will be satisfactory.
