jamovi

Hi, I'm getting slightly different results, especially with smaller (e.g., N ≤ 30) sample sizes, when I run moderation analyses with medmod (versus results from the equivalent model specified using GAMLj > linear model). Specifically, the Z values (and associated p values) for the parameter estimates from medmod are off a bit when comparing them with the equivalent t (and p) values for parameter estimates.

My understanding is that this arises because the med/mod functions in the medmod module are scaffolded onto lavaan, which runs a Wald test and computes Z-statistics that are tested against a chi-square (vs student's t) distribution. Many times, Wald (and vanilla t) tests will lead you to the same decision/inference (i.e., to reject the null or not). But, it seems that with smaller sample sizes the p-values can be different and that can be problematic.

If possible, could a future release of medmod give the option of computing t (and associated p) values instead of the Wald test (with z and p values) inherited from lavaan?

So medmod indeed uses lavaan for its estimation while GAMLj uses the regular lm function. With medmod I wanted to create a package/module that makes it easy for people to use the power of lavaan for these type of analyses (and possibly also move on to lavaan for more complicated models). For this use case, I think it's important that the results are the same as the results in lavaan.

GAMLj uses lm() for the GLM, so this explains the differences