Choice of effect size for Mann-Whitney/Wilcoxon tests

Discuss statistics related things

by Bobafett » Thu Jul 18, 2019 12:58 pm

Hi all,
First of all let me say what a great job you're doing with jamovi - it's a fantastic tool to teach statistics with. I trialled it with our postgraduate students this year, and we're rolling it out to our undergraduate students this coming academic year.

I have a query about the choice of effect size for running Mann-Whitney or Wilcoxon Signed Ranks analyses - both default to reporting Cohen's d (presumably as they are sub-options in the t-test analyses). However, I was always taught to use r - as in 'Z / sqrt #data points' in the instance of these non-parametric equivalents and this is what we've previously taught students to use when running SPSS.

Is it possible to have this calculated instead of or as well as Cohen's d when selecting the non-parametric option?

Cheers - and keep up the good work!
Bobafett
 
Posts: 4
Joined: Thu Jul 18, 2019 11:33 am

by jonathon » Fri Jul 19, 2019 2:03 am

hi,

i feel like using a square root with non-parametric data is problematic. so with parametric data, there's some mathematical foundation for using squares, and summing them, but i think that's all possible because we know that two is twice one, etc.

but with non-parametric data, we don't know that two isn't ten times one, or a hundred times one, so i don't feel like taking square roots would be a correct approach ... (but you're possibly getting the impression that i only grasp this stuff intuitively). i probably need ravi to chime in.

could you point us to an article which explains it?

with thanks

jonathon
User avatar
jonathon
 
Posts: 1270
Joined: Fri Jan 27, 2017 10:04 am

by Bobafett » Fri Aug 30, 2019 11:18 am

Hi Jonathon,
Apologies for the delay in posting a reply - summer came and went!

My issue with using Cohen's d is that these are derived from differences in means and SD which are not really applicable to non-parametric statistics - and goes against why we would advocate using NPs to our students . For the past few years I've used the above r calculation when reporting effect sizes and used Andy Field's text book 'Discovering Statistics using SPSS' (3rd Ed). I've since read up on this issue a little more and found a discussion thread on Researchgate which indicates this calculation was proposed by Rosenthal*. The thread also proposes other formula which subsequent contributors have queried, so I guess there does not appear to be a clear consensus on this issue...

Cheers
Paul


Rosenthal (1994). Parametric measures of effect size. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis. (pp. 231-244). New York: Russell Sage Foundation.
Bobafett
 
Posts: 4
Joined: Thu Jul 18, 2019 11:33 am

by coledavis » Tue Sep 10, 2019 9:29 pm

For what it's worth, I believe that Hedges' g is preferred to Cohen's d where there are less than 20 cases.
coledavis
 
Posts: 27
Joined: Sun Jul 02, 2017 1:35 pm

by Mik » Tue Oct 29, 2019 3:38 am

Hello,

The best would be to use a non-parametric effect size like Vargha & Delaney  measure of dominance. It is implemented in the RProbSup R package. With that package, you will also get the confidence interval around the effect size. I advise you read that article: https://www.researchgate.net/profile/Jo ... esigns.pdf

Peng, C.-Y. J., & Chen, L.-T. (2014). Beyond Cohen's d : Alternative Effect Size Measures for Between-Subject Designs. The Journal of Experimental Education, 82(1), 22-50. doi:10.1080/00220973.2012.745471

"[Â] estimate[s] the degree of one distribution dominating over the other distribution" (p.40)

"Of the nine estimators summarized in Table 2b, we recommend the four estimators of dominance in Category (B) to supplement Cohen’s d to conceptualize ES beyond mean differences. Of these four estimators, Vargha and Delaney’s  stands out for its meaningful interpretability in terms of stochastic equality/superiority or stochastic homogeneity/heterogeneity in a variety of research contexts and for a variety of data types. Compared to Cohen’s d, Vargha and Delaney’s  represents a radical reconceptualization of ES with sound statistical properties and well developed theoretical framework." (p.45)

That would be great if Jamovi could provide the  effect size.
Mik
 
Posts: 1
Joined: Tue Oct 29, 2019 3:29 am


Return to Statistics

cron