Hi everyone!
Perhaps a stupid question, but here I go. I have a 2x2 between group factorial design and got these results from the analysis (I chose ANOVA since Levene´s test, p = .926):
IV 1: p < .001
IV 2: p = .101
Interaction effect: p < .001
So, I have a main effect of IV 1, no main effect of IV 2, and lastly an interaction effect.
Am I good with the analyses, since there is only one main effect among the IV:s, that is , I don´t have to do a post hoc test? Is my interpretation correct? Do my choice of analysis methods look decent?
2x2 factorial design. Need post hoc test in this case?
Re: 2x2 factorial design. Need post hoc test in this case?
Depends on your research question.
The significant main effect of IV1 will be averaged across IV2. But you may want to check whether the effect of IV1 is also significant for each level of IV1. Say IV2 is age, with 2 groups, Young and Old. Is IV1 significant for Young only as well? And for Old only? Or only when both are averaged?
Also, you may want to understand what drives the interaction. So you could test whether IV2 shows a significant difference for the separate levels of IV1. Say IV1 is the type of training people do, training A or training method B. The interaction shows that the training effect is significantly different for Young and Old. You may want to test separately whether training A vs B is significant for Old, and whether A vs B is significant for Young only.
It's not a must, but in my type of research I need to know these things.
The significant main effect of IV1 will be averaged across IV2. But you may want to check whether the effect of IV1 is also significant for each level of IV1. Say IV2 is age, with 2 groups, Young and Old. Is IV1 significant for Young only as well? And for Old only? Or only when both are averaged?
Also, you may want to understand what drives the interaction. So you could test whether IV2 shows a significant difference for the separate levels of IV1. Say IV1 is the type of training people do, training A or training method B. The interaction shows that the training effect is significantly different for Young and Old. You may want to test separately whether training A vs B is significant for Old, and whether A vs B is significant for Young only.
It's not a must, but in my type of research I need to know these things.
Re: 2x2 factorial design. Need post hoc test in this case?
Thank you for your answer!
I´m very new to both ANOVA and Jamovi, so I´m not sure I understand fully (yet) , but please bare with me.
My IV1 is gender (woman, man) and IV2 is physical activity (biking, walking). DV is score on a memory test. I interpret my result as follows:
1. Having a female sex yields a significantly better test score than having a male sex, F(1, 20) = 103.05, p < .001, ηp2 = .84)
2. Biking does not yield a higher test score than walking (or vice versa???), F(1, 20) = 2.95, p =.10, ηp2 = .13)
3. There´s an interaction effect where being female and walking yields a higher test score (than being female and biking and being man and biking or walking respectively???), F(1, 20) = 184.61, p < .001, ηp2 = .90
Am I on the right track? What do you suggest I do as complementary analysis here?
I´m very new to both ANOVA and Jamovi, so I´m not sure I understand fully (yet) , but please bare with me.
My IV1 is gender (woman, man) and IV2 is physical activity (biking, walking). DV is score on a memory test. I interpret my result as follows:
1. Having a female sex yields a significantly better test score than having a male sex, F(1, 20) = 103.05, p < .001, ηp2 = .84)
2. Biking does not yield a higher test score than walking (or vice versa???), F(1, 20) = 2.95, p =.10, ηp2 = .13)
3. There´s an interaction effect where being female and walking yields a higher test score (than being female and biking and being man and biking or walking respectively???), F(1, 20) = 184.61, p < .001, ηp2 = .90
Am I on the right track? What do you suggest I do as complementary analysis here?
Re: 2x2 factorial design. Need post hoc test in this case?
You wrote "There´s an interaction effect where being female and walking yields a higher test score (than being female and biking and being man and biking or walking respectively." That may be true, but it isn't a description of an interaction. The interaction is that the change in the mean recall score, as a function of sex, is different for biking than for walking. The exact pattern of the interaction will be revealed by a plot of the four means. (For example, maybe the change in the mean recall score, as a function of sex, is GREATER for biking than for walking.)
And note that the tests for main effects and for the interaction don't address the question of whether any of the four means is significantly different from any of the other means. For that, you would need post-hoc tests.
And note that the tests for main effects and for the interaction don't address the question of whether any of the four means is significantly different from any of the other means. For that, you would need post-hoc tests.
Re: 2x2 factorial design. Need post hoc test in this case?
Thank you for your answer!
I´ve proceeded with post hoc tests and compared the different means.
I´ve proceeded with post hoc tests and compared the different means.
Re: 2x2 factorial design. Need post hoc test in this case?
Since you have a strong interaction, then the main effects (specially for sex) could be marginal (secondary) to the interaction. This is known as the principle of marginality. In your example you may not be able to claim that women in general perform better than men, only those women who walk!
Re: 2x2 factorial design. Need post hoc test in this case?
RE: "Since you have a strong interaction, then the main effects (specially for sex) could be marginal (secondary) to the interaction. This is known as the principle of marginality. In your example you may not be able to claim that women in general perform better than men, only those women who walk!"
People disagree on that point. In my opinion, the purpose of a factorial ANOVA is to partition variance due to the main effects and interacts, respectively. One source of variance does not in any way contaminate another (as long as all assumptions are met). Thus, the reported, significant main effect of sex indicates that the mean memory performance was significantly better for women than men, regardless of the presence or absence of an interaction--even if turns out that the sex difference is slightly reversed for bikers compared to walkers. In sum, I think each main effect is about the effect of one factor, when the data are averaged across the levels of the other factor(s).
People disagree on that point. In my opinion, the purpose of a factorial ANOVA is to partition variance due to the main effects and interacts, respectively. One source of variance does not in any way contaminate another (as long as all assumptions are met). Thus, the reported, significant main effect of sex indicates that the mean memory performance was significantly better for women than men, regardless of the presence or absence of an interaction--even if turns out that the sex difference is slightly reversed for bikers compared to walkers. In sum, I think each main effect is about the effect of one factor, when the data are averaged across the levels of the other factor(s).
Re: 2x2 factorial design. Need post hoc test in this case?
Thanks reason180, I appreciate your argument, but still think there is often a marginality issue.