Request for adding to the T-Test module the effect sizes R² / Eta squared and probability of superiority

[Martin van Leerdam]

I would like to request a couple of what I believe might be important additions to the present output of the Jamovi Independent Samples T-Test module.

At present, the resulting table presents the difference between two group means and, if requested, the standardized effect size Cohen’s d. What I think is seriously lacking here is another scale-free effect size, namely R² (or Eta squared, which is identical in this case). Considering that a measure of explained variance (R² / Eta squared) is routinely provided in the regression / ANOVA / GLM modules, why not also providing it in the Independent Samples T-Test module and saving Jamovi users the trouble of running multiple group comparison analyses to get all the relevant effect sizes?

A second addition to the effect sizes I would like to suggest is stochastic superiority (a.k.a. probability of superiority effect size measure / the Common Language Effect Size), which presents the probability that a randomly selected respondent from group A has a higher score on the outcome variable than a randomly selected respondent from group B. This alternative measure of group differences is easy to calculate / to program, can be based both on Cohen’s d and Mann-Whitney U and is in my opinion a very useful additional effect size measure (just try for once, using the raw measurement units or, even worse, using Cohen’s d, to generate a clear intuition of the size of an intervention effect with researchers or other professionals who are not statistical experts). For example, if a mean difference of 4.41 points on some non-intuitive scale translates into a Cohen’s d of 1.35, then the corresponding probability of superiority is 0.83. This means that if you would take many randomly selected pairs of respondents from group A and group B, then in 83% of those pairs the respondent from group A has a higher score than the respondent of group B. This probability provides another kind of insight into the degree the two groups differ on the outcome variable.

I think that these two changes would provide the Jamovi user with (much needed) additional insight into the (practical / clinical) importance of the observed difference between two groups.

Kelley, K., & Preacher, K. J. (2012). On effect size. Psychological Methods, 17(2), 137–152.
McGraw, K. O., & Wong, S. P. (1992). A common language effect size statistic. Psychological Bulletin, 111(2), 361–365. https://doi.org/10.1037/0033-2909.111.2.361
Ruscio, J. (2008). A probability-based measure of effect size: Robustness to base rates and other factors. Psychological Methods, 13, 19–30.
Ruscio, J., & Gera, B. L. (2013). Generalizations and extensions of the probability of superiority effect size estimator. Multivariate Behavioral Research, 48(2), 208–219.
Ruscio, J., & Mullen, T. (2012). Confidence intervals for the probability of superiority effect size measure and the area under a receiver operating characteristic curve. Multivariate Behavioral Research, 47, 201–223.
Vargha, A., & Delaney, H. D. (2000). A critique and improvement of the CL common language effect size statistics of McGraw and Wong. Journal of Educational and Behavioral Statistics, 25,101–132.