Cohen's d in post-hoc test

Discuss the jamovi platform, possible improvements, etc.

by Vicente_inefo » Fri Oct 25, 2019 8:15 pm

Hi,

I have performed a mixed 2x2 ANOVA and I noticed that the post-hoc test does not report the effect size. Is there any possibility to implement it?

On the other hand, if JAMOVI computes Cohen's d by the formula d = t/sqrt(n), it would be correct to compute Cohen's d by using the t value reported by JAMOVI and the n value of the respective group?

Please, see the attached file for more details

I accept any other suggestion, thanks in advance
Attachments
Captura de pantalla 2019-10-25 a las 22.12.43.png
Post-hoc test table
Captura de pantalla 2019-10-25 a las 22.12.43.png (115.35 KiB) Viewed 575 times
Vicente_inefo
 
Posts: 3
Joined: Wed May 29, 2019 11:39 pm

by jonathon » Sat Oct 26, 2019 10:23 pm

hi,

the issue is that the post-hoc tests in jamovi are based on the estimated marginal means, rather than simply being pairwise comparisons of the raw data. so calculating cohen's d for these is a bit more complicated ... i *think* it's possible to calculate cohen's d for emmeans, but even then i'm not completely sure.

i've asked a few people about it, and i'll see what they come back with.

cheers
User avatar
jonathon
 
Posts: 1052
Joined: Fri Jan 27, 2017 10:04 am

by Vicente_inefo » Mon Oct 28, 2019 11:53 am

Thank you very much, I will remain attentive to your response
Vicente_inefo
 
Posts: 3
Joined: Wed May 29, 2019 11:39 pm

by jose.lopez » Mon Feb 10, 2020 3:35 pm

Hi,

I was also confused to find that the Cohen's d calculated with the post-hoc comparisons does not match the classical (y1-y2)/S formula. It was useful to learn from Jonathon that the values are based on estimated marginal means. However, I'd still be interested in Cohen's d based on the observed means for teaching purposes, so I'm wondering if anyone can help me find an option to obtain these with jamovi?

Cheers,
José
jose.lopez
 
Posts: 1
Joined: Mon Feb 10, 2020 11:52 am

by jonathon » Mon Feb 10, 2020 10:39 pm

hey,

yeah, we'd like to provide a 'pairwise' comparisons module, that works on descriptives, rather than emms. don't get me wrong, i think working off descriptives is wrong most of the time :P but it seems to be something a lot of people want.

cheers

jonathon
User avatar
jonathon
 
Posts: 1052
Joined: Fri Jan 27, 2017 10:04 am

by reason180 » Sun Feb 16, 2020 2:54 am

It may be that people are skeptical of emmeans because they're not sure exactly how the estimated marginals relate to the regular old marginals.

I think in the case of an ANOVA, the estimated marginal means are identical to the marginal means UNLESS you're averaging over the levels of a know factor such that the averaged-over levels have unequal sample sizes. In the latter case, the estimated marginal mean are the means of the averaged-over means, rather than the means of raw data. Thus for example, if there are scores for left- and right-handed men and women, and if there are 10 left-handed and 90 right-handed participants, and if the ANOVA model includes handedness and gender as factors, and if one is interested in looking just at the main effect of gender then:

The estimated marginal means for the main effect of gender are not the mean scores for the women versus the mean scores for the men. Rather they are the mean of two means--the mean score for left-handed women and the mean score for right-handed women--versus the mean of two other means--the mean score for left-handed men and the mean score for right-handed men. Thus the estimated marginal means are the means that one would find had the study included an equal number of left- and right-handed participants.

But I don't know how to calculate the variances that correspond to those estimated marginal means (when the sizes of the averaged samples are unequal). This is because it's more complicated: It involves not just pooling variances (consistent with the equal-variance assumption) but also modeling what the variances would be had the sample sizes for the averaged categories been equal.

(Of course, all of this is even more complicated when the dealing with ANCOVA.)
reason180
 
Posts: 36
Joined: Mon Jul 24, 2017 4:56 pm

by mcfanda@gmail.com » Mon Feb 17, 2020 9:55 am

The easiest way is to use the t-test. d=t*sqrt(1/n1+1/n2)
User avatar
mcfanda@gmail.com
 
Posts: 180
Joined: Thu Mar 23, 2017 9:24 pm


Return to General