effect size relative to raw score but not difference score
Posted: Sun Mar 21, 2021 5:27 am
Hi, I would like to know is there any way to calculate effect size relative to raw score instead of difference score, in a paired-sample t-test.
I know the typical way to calculate Cohen's d for paired-sample t-test is to divide mean difference for difference score by standard deviation of difference score. So actually the effect is relative to difference score instead of raw score.
I found it seems to be able to calculate Cohen's d relative to raw score instead of difference score and would like to know the detail about how to do it. Does anyone know it?
from an online jamovi user-guide: https://www.learnstatswithjamovi.com/
on page 272:
"To the extent that you care about the practical consequences of your research, you often want to measure the effect size relative to the original variables, not the difference scores (e.g., the 1% improvement in Dr Chico’s class over time is pretty small when measured against the amount of between-student variation in grades), in which case you use the same versions of Cohen’s d that you would use for a Student or Welch test. It’s not so straightforward to do this in jamovi; essentially you have to change the structure of the data in the spreadsheet view so I won’t go into that here, but the Cohen’s d for this perspective is quite different: it is 0.22 which is quite small when assessed on the scale of the original variables."
Firstly, I don't quite understand "against the amount of between-student variation in grades"?
Secondly, I can't find the jamovi file mentioned in note 12: "If you are interested, you can look at how this was done in the chico2.omv file" (I would like to see how to "change the structure of the data")
Does anyone know about this?
Thanks a lot!
I know the typical way to calculate Cohen's d for paired-sample t-test is to divide mean difference for difference score by standard deviation of difference score. So actually the effect is relative to difference score instead of raw score.
I found it seems to be able to calculate Cohen's d relative to raw score instead of difference score and would like to know the detail about how to do it. Does anyone know it?
from an online jamovi user-guide: https://www.learnstatswithjamovi.com/
on page 272:
"To the extent that you care about the practical consequences of your research, you often want to measure the effect size relative to the original variables, not the difference scores (e.g., the 1% improvement in Dr Chico’s class over time is pretty small when measured against the amount of between-student variation in grades), in which case you use the same versions of Cohen’s d that you would use for a Student or Welch test. It’s not so straightforward to do this in jamovi; essentially you have to change the structure of the data in the spreadsheet view so I won’t go into that here, but the Cohen’s d for this perspective is quite different: it is 0.22 which is quite small when assessed on the scale of the original variables."
Firstly, I don't quite understand "against the amount of between-student variation in grades"?
Secondly, I can't find the jamovi file mentioned in note 12: "If you are interested, you can look at how this was done in the chico2.omv file" (I would like to see how to "change the structure of the data")
Does anyone know about this?
Thanks a lot!