Different Cohen's d depending on App or R

General help and assistance with jamovi. Bug reports can be made at our issues page: https://github.com/jamovi/jamovi/issues . (If you're unsure feel free to discuss it here)

by hjin » Wed Jun 23, 2021 2:24 am

I'm not sure if this has been posted before but I noticed that getting the effect size for pairwise comparisons in ANOVA directly through the jamovi program yields different results from running the exact same analysis through R.

For instance, one of the pairwise comparisons has VAR1 (n = 75) with M ± SD = 6.8 ± 3.5 and VAR2 (n = 74) with M ± SD = 4.5 ± 3.0, which comes out to be approximately a cohen's d of ~0.7. This seems to be calculated correctly in the jamovi application but when I run the same analysis through R, the cohen's d comes out to be around around d = 0.25. All other metrics (e.g., p-value, test statistic, etc.) are the same, however.

From doing a bit of calculation, it looks like the ANOVA cohen's d calculation when done through R is dividing the Cohen's d calculation by a constant of 2.81898 (I have a 2x2 factorial two-way ANOVA). Is this the program in R erroneously applying a Bonferroni-style adjustment to the Cohen's d?

*I'm using jamovi v.1.8.1.0 and I've updated the jmv/jmvcore packages on R but am still getting the same issue. I ran the same analysis through the jamovi program's R editor module and it's coming out correctly, so it seems isolated to just R.
hjin
 
Posts: 4
Joined: Wed Jun 23, 2021 2:06 am

by jonathon » Wed Jun 23, 2021 2:39 am

for post-hoc tests, in jamovi, the cohen's d are based on the estimated marginal means, which might be different to what you're doing in R?

jonathon
User avatar
jonathon
 
Posts: 1803
Joined: Fri Jan 27, 2017 10:04 am

by hjin » Wed Jun 23, 2021 4:46 am

Hi Jonathon,

Thanks for the response! Sorry, to clarify, I'm running the exact same jmv ANOVA on R using the jmv syntax (where I copy the syntax that the jamovi program gives me and paste it in R which makes it a bit easier for just record purposes).

So when I run the analysis in the jamovi program, I get the following:

jamovi1.JPG
jamovi1.JPG (13.74 KiB) Viewed 4797 times


But when I run the same analysis using the jmv syntax in R, I get the following:

jamovi2.JPG
jamovi2.JPG (19.21 KiB) Viewed 4797 times


Note that the p-values are the same but the effect sizes are showing up different for some reason.


I'm using the following syntax:

jmv::ANOVA(
formula = dv ~ iv1 + iv2 + iv1:iv2,
data = data,
effectSize = "partEta",
postHoc = ~ iv1:iv2,
postHocES = "d",
emMeans = ~ iv1:iv2
)
hjin
 
Posts: 4
Joined: Wed Jun 23, 2021 2:06 am

by jonathon » Thu Jun 24, 2021 1:26 am

oh yup, update your version of jmv and it will fix it. there's all these additional requirements that CRAN have imposed on R packages, and it's made it difficult for us to update the version on CRAN.

you can install the latest version with:

remotes::install_github('jamovi/jmv')

cheers

jonathon
User avatar
jonathon
 
Posts: 1803
Joined: Fri Jan 27, 2017 10:04 am

by hjin » Thu Jun 24, 2021 3:28 am

Ah I see now that the one from CRAN is v.1.2 instead of the newest one!
I updated the version to the newest version but now I'm getting the following error:

Error: 'OptionOutput' is not an exported object from 'namespace:jmvcore'

I'm not sure what the problem can be,
I tried multiple things, like changing all my variables to numeric, or changing the class of my grouping variables to factors, and got rid of every additional command in the syntax but jmv::ANOVA won't even run with just the formula and data commands.

Other analyses (e.g., correlation, descriptives, etc.) seem to be running fine though so I'm not sure if something else is wrong on my end.
hjin
 
Posts: 4
Joined: Wed Jun 23, 2021 2:06 am

by jonathon » Thu Jun 24, 2021 5:30 am

hey, you need to update your jmvcore too (sorry, i forgot to mention):

remotes::install_github('jamovi/jmvcore')
User avatar
jonathon
 
Posts: 1803
Joined: Fri Jan 27, 2017 10:04 am

by hjin » Thu Jun 24, 2021 5:44 am

Everything's coming out great now, thank you Jonathon!
hjin
 
Posts: 4
Joined: Wed Jun 23, 2021 2:06 am

by MAgojam » Thu Jun 24, 2021 7:34 am

Hi, @hjin.
Your report reminded me of a similar thing, which happened some time ago to a colleague who asked me for help.
I had checked using Rj with "jamovi R" and "System R" on my PC without highlighting any differences, while this was happening on the colleague's PC.
The differences highlighted in the two R environments were due to two becoming versions of jmv.
The colleague updated jmv from CRAN with a later version than the one present in the last jamovi used.
I suggest you check the version you have in system R and if different from jamovi R, update from github.
You should have something like in the attachment.
screenshot.png
screenshot.png (34.17 KiB) Viewed 4764 times

If the problem doesn't go away, it comes back.

Cheers,
Maurizio

---------------------
Sorry, Jonathon.
The page did not update and I only realized now that you had already suggested the solution.

Cheers,
Maurizio
User avatar
MAgojam
 
Posts: 172
Joined: Thu Jun 08, 2017 2:33 pm
Location: Parma (Italy)


Return to Help