## Different Cohen's d depending on App or R

General help and assistance with jamovi. Bug reports can be made at our issues page: https://github.com/jamovi/jamovi/issues . (If you're unsure feel free to discuss it here)

## Different Cohen's d depending on App or R

I'm not sure if this has been posted before but I noticed that getting the effect size for pairwise comparisons in ANOVA directly through the jamovi program yields different results from running the exact same analysis through R.

For instance, one of the pairwise comparisons has VAR1 (n = 75) with M ± SD = 6.8 ± 3.5 and VAR2 (n = 74) with M ± SD = 4.5 ± 3.0, which comes out to be approximately a cohen's d of ~0.7. This seems to be calculated correctly in the jamovi application but when I run the same analysis through R, the cohen's d comes out to be around around d = 0.25. All other metrics (e.g., p-value, test statistic, etc.) are the same, however.

From doing a bit of calculation, it looks like the ANOVA cohen's d calculation when done through R is dividing the Cohen's d calculation by a constant of 2.81898 (I have a 2x2 factorial two-way ANOVA). Is this the program in R erroneously applying a Bonferroni-style adjustment to the Cohen's d?

*I'm using jamovi v.1.8.1.0 and I've updated the jmv/jmvcore packages on R but am still getting the same issue. I ran the same analysis through the jamovi program's R editor module and it's coming out correctly, so it seems isolated to just R.
hjin

Posts: 4
Joined: Wed Jun 23, 2021 2:06 am

## Re: Different Cohen's d depending on App or R

for post-hoc tests, in jamovi, the cohen's d are based on the estimated marginal means, which might be different to what you're doing in R?

jonathon

jonathon

Posts: 1803
Joined: Fri Jan 27, 2017 10:04 am

## Re: Different Cohen's d depending on App or R

Hi Jonathon,

Thanks for the response! Sorry, to clarify, I'm running the exact same jmv ANOVA on R using the jmv syntax (where I copy the syntax that the jamovi program gives me and paste it in R which makes it a bit easier for just record purposes).

So when I run the analysis in the jamovi program, I get the following:

jamovi1.JPG (13.74 KiB) Viewed 4799 times

But when I run the same analysis using the jmv syntax in R, I get the following:

jamovi2.JPG (19.21 KiB) Viewed 4799 times

Note that the p-values are the same but the effect sizes are showing up different for some reason.

I'm using the following syntax:

jmv::ANOVA(
formula = dv ~ iv1 + iv2 + iv1:iv2,
data = data,
effectSize = "partEta",
postHoc = ~ iv1:iv2,
postHocES = "d",
emMeans = ~ iv1:iv2
)
hjin

Posts: 4
Joined: Wed Jun 23, 2021 2:06 am

## Re: Different Cohen's d depending on App or R

oh yup, update your version of jmv and it will fix it. there's all these additional requirements that CRAN have imposed on R packages, and it's made it difficult for us to update the version on CRAN.

remotes::install_github('jamovi/jmv')

cheers

jonathon

jonathon

Posts: 1803
Joined: Fri Jan 27, 2017 10:04 am

## Re: Different Cohen's d depending on App or R

Ah I see now that the one from CRAN is v.1.2 instead of the newest one!
I updated the version to the newest version but now I'm getting the following error:

Error: 'OptionOutput' is not an exported object from 'namespace:jmvcore'

I'm not sure what the problem can be,
I tried multiple things, like changing all my variables to numeric, or changing the class of my grouping variables to factors, and got rid of every additional command in the syntax but jmv::ANOVA won't even run with just the formula and data commands.

Other analyses (e.g., correlation, descriptives, etc.) seem to be running fine though so I'm not sure if something else is wrong on my end.
hjin

Posts: 4
Joined: Wed Jun 23, 2021 2:06 am

## Re: Different Cohen's d depending on App or R

hey, you need to update your jmvcore too (sorry, i forgot to mention):

remotes::install_github('jamovi/jmvcore')

jonathon

Posts: 1803
Joined: Fri Jan 27, 2017 10:04 am

## Re: Different Cohen's d depending on App or R

Everything's coming out great now, thank you Jonathon!
hjin

Posts: 4
Joined: Wed Jun 23, 2021 2:06 am

## Re: Different Cohen's d depending on App or R

Hi, @hjin.
Your report reminded me of a similar thing, which happened some time ago to a colleague who asked me for help.
I had checked using Rj with "jamovi R" and "System R" on my PC without highlighting any differences, while this was happening on the colleague's PC.
The differences highlighted in the two R environments were due to two becoming versions of jmv.
The colleague updated jmv from CRAN with a later version than the one present in the last jamovi used.
I suggest you check the version you have in system R and if different from jamovi R, update from github.
You should have something like in the attachment.
screenshot.png (34.17 KiB) Viewed 4766 times

If the problem doesn't go away, it comes back.

Cheers,
Maurizio

---------------------
Sorry, Jonathon.
The page did not update and I only realized now that you had already suggested the solution.

Cheers,
Maurizio

MAgojam

Posts: 172
Joined: Thu Jun 08, 2017 2:33 pm
Location: Parma (Italy)