jamovi

Hi Maurizio
Sorry for not being clear. I meant newer versions of the MAJOR module. This issue has been raised repeatedly in the MAJOR GitHub, and some are marked as resolved.

Major is fantastic by all accounts, and I am so glad Kyle mead it. It is sad to hear he is not much around tending to it.
Thank you for the tremendous support you and Jonathon are providing us!

It seems a probable solution would be to have an older version of the module and jamovi on another machine. Or use another meta-analysis tool.

This post has evolved into a sort of my personal log on how to solve the question at hand - compare two effect sizes. It may come useful to someone else...

In my last efforts, I planned on using meta-analytical approaches for the comparison. I would have compared the two effect sizes using Cochran's Q (and its p-value) and I would have considered this test of heterogeneity as a test of the difference between effects. Although it is seemingly plausible, Q seems to be useful only for comparison of >10 effects (more on this in https://bookdown.org/MathiasHarrer/Doin ... ysis_in_R/).

By one of the first replies from Maurizio, I should calculate a confidence interval for the "difference of mean differences" and check if this includes 0. I came across one of the famous BMJ Statistical notes (https://www.bmj.com/content/326/7382/219.full) with a possible solution.

" If the estimates are E1 and E2 with standard errors SE(E1) and SE(E2), then the difference d=E1-E2 has standard error SE(d)=√[SE(E1)^2 + SE(E2)^2.Then the ratio z=d/SE(d) gives a test of the null hypothesis that in the population the difference d is zero, by comparing the value of z to the standard normal distribution. The 95% confidence interval for the difference is d-1.96×SE(d) to d+1.96×SE(d). "

Using whatever effect-size measure you want, calculate a pair of effect sizes for each participant. Then subtract one effect size from the other to compute, for each participant, what one might call Delta_ES. Then run a single sample t test to assess whether the mean value of Delta_ES is significantly different from 0.00.

Hey Reason180,
Thanks for your reply and suggestion to use a one-sample t-test comparing against 0.

Usually, there are more ways to answer one question. Would you think the solution I outlined is less optimal than yours or a different approach that should ultimately yield overlapping answers?

Hi Vit.

I did not follow all the details of the other suggestions. However, I believe my simple solution is a correct one, and has the advantage of not exceeding the scope of the student's curriculum.