Post Hoc problems - repeated ANOVA - 2 within factors

Discuss statistics related things
Post Reply
Osiris
Posts: 1
Joined: Fri Mar 09, 2018 10:20 am

Post Hoc problems - repeated ANOVA - 2 within factors

Post by Osiris »

Hello,
Thank uou very for JAMOVI. It looks like this is going to be a very very nice soft when fully developped.
If I am not mistakened, I think there are problems with the post-hoc comparisons that follows a repeated ANOVA. I tried a 2 within factors (2 levels each). To be more precise, it seems to me that the df values for the post-hoc comparisons are wrong. I have enclosed the anlaysis that I performed which includes at the end a Paired Samples T-Test for one of the post-hoc comparison (see circled). See the difference. Thank you very much.




Pascale Pomietto
Attachments
OutputANOVA.png
OutputANOVA.png (16.84 KiB) Viewed 3960 times
User avatar
Ravi
Posts: 194
Joined: Sat Jan 28, 2017 11:18 am

Re: Post Hoc problems - repeated ANOVA - 2 within factors

Post by Ravi »

Hi,
Thank you for the kind words. So the post-hoc tests in all the ANOVA's are based on the estimated marginal means. This is different from just doing t-tests and then correcting the p-value afterwards. In the estimated marginal means approach you do post-hoc tests on the actual model estimates, while by doing a series of t-tests you're basically fitting a number of models that have nothing to do with the linear model used for the RM ANOVA. Because we use a different approach, the df's are also different. We use the emmeans R package to calculate the post-hoc tests. You can find more information over here: https://cran.r-project.org/web/packages ... isons.html

Cheers,
Ravi
User avatar
reason180
Posts: 274
Joined: Mon Jul 24, 2017 4:56 pm

Re: Post Hoc problems - repeated ANOVA - 2 within factors

Post by reason180 »

I have a slightly different hypothesis about at least part of what may be going on here. I think that since jamovi is relying on R packages, and therefore relying on the sometimes uneven and limited documentation provided by those individuals who have produced the r packages, there are some thing happening in the jamovi (and R) analyses that we have to figure out by trial and error rather than being able to read about it in the r package documentation. My trial-and-error investigation of the emmeans function lead me to believe that one aspect of the apparent wrongness of the degrees of freedom is tis: Whereas host hoc tests in the between-subject ANOVA routiine are Student's t tests in which the degrees of freedom don't depend on the error variances, those in the repeated-measures routine do depend on the error variances, with the possibility that the degrees of freedom can sometimes even take on a fractional value. Am I correct on this point?
Post Reply