Cohen's d for T test paired samples

General help and assistance with jamovi. Bug reports can be made at our issues page: https://github.com/jamovi/jamovi/issues . (If you're unsure feel free to discuss it here)
Post Reply
Jacques
Posts: 3
Joined: Wed Aug 24, 2022 2:19 pm

Cohen's d for T test paired samples

Post by Jacques »

Hello. The Cohen's d effect size given by Jamovi (version 2.2.5.0) for T test for two paired samples is not the same as the one I calculate by hand (means dif/pooled_sd) or the one given by Python pingouin.ttest module.
For T test for independent samples, the Conhen’s d given by Jamovi is the same as the one I calculated and as pingouin.ttest module.
Can you explain why is this difference for paired samples ?
User avatar
MAgojam
Posts: 421
Joined: Thu Jun 08, 2017 2:33 pm
Location: Parma (Italy)

Re: Cohen's d for T test paired samples

Post by MAgojam »

Hey @Jacques,

how then to see, here from the sources R:
https://github.com/jamovi/jmv/blob/mast ... .R#L66-L68

The Cohen's d effect size is obtained from the ratio between the difference of the 2 means and the standard deviation of the differences.

Cheers,
Maurizio
Jacques
Posts: 3
Joined: Wed Aug 24, 2022 2:19 pm

Re: Cohen's d for T test paired samples

Post by Jacques »

Hi Magojam
Thank you for your answer
In https://github.com/jamovi/jmv/blob/mast ... .R#L66-L68
I see that
d <- (m1-m2)/pooledSD #Cohen's d
with
pooledSD <- tryNaN(stats::sd(column1-column2))
but this is not the pooled SD, it the SD of the difference.

Python pingouin.ttest uses for the denominator the pooledSD which is
SDpooled = √((SD12 + SD22) ⁄ 2)

Is there two possible ways to calculate d or someone is wrong ?
Bobafett
Posts: 76
Joined: Thu Jul 18, 2019 11:33 am

Re: Cohen's d for T test paired samples

Post by Bobafett »

There are multiple ways to generate Cohen's d measures [as I recall DeShon & Morris (2008?) suggested factoring in the correlation between the two variables] and as such not everyone will agree with what the final equation used will be. I guess an important caveat is that these are only estimates of effect size and the final sum produced by whatever route should not differ drastically....
Jacques
Posts: 3
Joined: Wed Aug 24, 2022 2:19 pm

Re: Cohen's d for T test paired samples

Post by Jacques »

With my samples (48 observations) the two methods differ a lot in their results :
d = 0.41 with Jamovi using d=(m1-m2)/SDdif
d = 0.27 (Python pingouin) using d=(m1-m2)/pooledSD
This is because the standard deviation of the differences is not the same as the pooled standard deviation of the two dependant samples.
User avatar
MAgojam
Posts: 421
Joined: Thu Jun 08, 2017 2:33 pm
Location: Parma (Italy)

Re: Cohen's d for T test paired samples

Post by MAgojam »

Hey @Jacques,

try another way to get effect size estimates for a "Paired Sample T-Test".
Divide the t-statistic of your paired t-test by the square root of your N (48), you should get d = 0.41

Cheers,
Maurizio
Bobafett
Posts: 76
Joined: Thu Jul 18, 2019 11:33 am

Re: Cohen's d for T test paired samples

Post by Bobafett »

The equation for the paired t-test assesses whether the mean difference across all the pairs of scores is significant, so would it not be possible to argue that the SD of these differences is more useful than the pooled SD of the respective conditions? I would think the (pooled) SD for both conditions is better suited for the independent t-test when this analysis has to be based on two separate groups ...

(sent you a pm by accident!)
Post Reply