Help with understanding non-parametric tests

[2100169]

Hello, I recently used jamovi to calculate some group differences for some data, and I would like some help in interpreting the results.
I referred to these tutorials: https://www.youtube.com/watch?v=rPAEOV_ ... p=gAQBiAQB https://www.youtube.com/watch?v=7OD7-fK ... p=gAQBiAQB

I would like help in
1) understanding what the W and U test statistics mean, and
2) how can I interpret the rank biserial correlation effect size (I understand that this is to a gauge of how 'generalisable' the results are to the population, and that there are ranges indicated by Cohen for how large an effect size is), especially what does a negative rrb indicate?

Thank you in advance

[reason180]

Contextualized within the example shown, below, the meaning of the U statistic is: "we take each tortoise in turn, and count the number of hares it beats" [Wikipedia].

The "effect size" is therefore some standardized measure of the extent to which the tortoises beat the hares [note that the tortoises actually didn't beat the hare in this example]. U isn't such a measure because U will tend to depend on the sample size N. The rank-biserial correlation is one standardized effect size. It's a correlation indicating the extent to which the tortoises beat the hares, in this example.

Untitleds54.png (173.56 KiB) Viewed 1889 times

[2100169]

@reason180

thank you for the concise explanation! I understand the concept of the U statistic a little better now. I think I also read that the W and U test statistics may not be an accurate representation of the effect if the sample size is large?

I was a little confused about how the rank biserial correlation is calculated (i.e. which group has higher ranks), so I did some tests with sample data to understand how jamovi calculates the directionality. For paired tests (Wilcoxon): group 1 is the one on the left and group 2 is the one on the right - if group 1 generally has higher ranks than group 2, the effect size would be positive. For independent tests (Mann Whitney U) it depends on the ordering in the data tab - the group on top is group 1 (the Pre group in this case) and if group 1 is higher than group 2, effect size would be negative. Please correct me if I'm wrong and if there was an easier way to figure this out!

Thank you for your help nonetheless!

[reason180]

Whenever I use this kind of test, I need to leek at the pattern of means and medians to be sure I know the direction of any obtained effect. I've never heard of the test lacking validity when the sample size is large. (Generally, if sample size is an issue with any given test, it's small samples that are the problem, not large samples.)

[Bobafett]

"...count the number of hares it beats" - ha - I like that encapsulation!

As I recall, the rank biserial correlation does indeed reflect the ratio of participants in group#1 beating participants in group #2. It does this by ranking the scores of all your participants in both your groups - and then determines how many from one group 'beat' the other). In your instance, I daresay that all 6 participants in one group performed better than all 6 in the other group (and vice versa when you did this the other way round). In the image below, the first example (light grey) recreates this - you can see that the first person from group1 beat all 6 people from group2, as in fact was the case for everyone in group 1. Hence, out of 36 possible combinations, all 36 were in favour of group 1 (6*6= 36 wins). The calculation for the rank biserial correlation is then wins - losses / total i.e. 36 - 0 / 36 = 100% (or 1.00)

In the second example (dark grey) I've deliberately changed things. In this example, you should be able to see that the first 5 people from group #1 each beat the 6 from group#2 (thus 5*6= 30 wins). However, one person from group#2 beat the last person from group#1 (1 loss). The last person from group#1 went on to beat 5 people from group#2 (5 wins). So add these together and you get 35 wins but 1 loss.

Do the math and you get 35 - 1 / 36 = 0.94 Try it and see!

Interpret this the same as you would any correlation according to Cohen's rule of thumb i.e. .1 = small, .3 = moderate and .5 = large

[2100169]

@Bobafett

Thank you for the explanation! Made it really easy to understand the concept of rank biserial correlation. My trouble was figuring out which group beat out which, but tinkering with a sample dataset helped me figure out.