jamovi

Hi guys. First of all, congrats on such amazing software. If I were Bill Gates I would have donated you the money he spent on that boat he bought this week.

So, the problem I found: after finding a significant interaction in a 2 x 7 RM ANOVA, I ran Bonferroni-corrected paired-sample t-tests (post hoc window) and found results that don't make sense: t(149) = -3.672, p =.03 but the way I understand it, with Bonferroni correction, a t stat of 149 degrees of freedom and a value of 3.67 should yield a p value of .0003 and not .03. Is there something non-standard that this Bonferroni correction is doing?

Did you try the noncorrection feature? is the P = .03 from the noncorrected?

Yes, but I want the Bonferroni correction - it's just that the Bonferroni correction in Jamovi gives me a number that doesn't correspond to what the calculation of the Bonferroni correction (with other software and by hand) gives.

I thought it might have to do with the fact that Jamovi based its analyses on estimated marginal means, but I don't know...

hi,

"I ran Bonferroni-corrected paired-sample t-tests (post hoc window)"

so the post-hoc tests are based on the estimated marginal means, and aren't paired-samples t-tests per se. the post-hoc tests in jamovi are based on the model you are fitting, in this case, a repeated measures ANOVA. an independent set of paired-samples t-tests is a different model.

does that go some way to explaining the discrepancy?

the code for post-hocs is here:

https://github.com/jamovi/jmv/blob/mast ... #L586-L591

cheers

jonathon

Thank you, that's very clarifying. However, I don't understand what these post hoc comparisons are, conceptually. If they are based on EEMs and are Bonferroni corrected, what are they?

here's an intro to emmeans:

https://cran.r-project.org/web/packages ... asics.html

the way i normally demonstrate emmeans, is to use the 'Tooth Growth' data set, run a two-way ANOVA (len ~ dose * supp), and plot the interaction. *then* remove the the interaction term from the model ... you'll see that this forces the lines in the interaction plot to be parallel. so this is what emmeans are ... the means estimated for each group, by the model you are fitting.

the post-hoc tests are, in turn, comparisons between these estimated means. if you think about it, it has to be this way - otherwise your post-hoc tests are unrelated to your ANOVA.

here's the wikipedia article on bonferroni correction:

https://en.wikipedia.org/wiki/Bonferroni_correction

let me know if you have any more questions.

cheers

jonathon

Thanks so much, Jonathon. That was very helpful. I wonder if using emmeans is justified when you have an experimental design like in neuroscience experiments, where the data is very well balanced...

yes, there's no problem using them with balanced data.

being able to use emmeans with unbalanced data is certainly advantage, but the bigger point here is that emmeans are based on the model you are fitting. they are based on the ANOVA.

if you want to fit a lot of separate t-tests, then it's appropriate to perform cohen's d on the descriptives, but if you're fitting a more complex model than that, then it makes sense that your cohen's d should be performed on the model.

jonathon