Different inferential stats (t vs z) values in medmod module

Everything related to the development of modules in jamovi

by richlopez41 » Mon Feb 19, 2018 11:01 pm

Hi, I'm getting slightly different results, especially with smaller (e.g., N ≤ 30) sample sizes, when I run moderation analyses with medmod (versus results from the equivalent model specified using GAMLj > linear model). Specifically, the Z values (and associated p values) for the parameter estimates from medmod are off a bit when comparing them with the equivalent t (and p) values for parameter estimates.

My understanding is that this arises because the med/mod functions in the medmod module are scaffolded onto lavaan, which runs a Wald test and computes Z-statistics that are tested against a chi-square (vs student's t) distribution. Many times, Wald (and vanilla t) tests will lead you to the same decision/inference (i.e., to reject the null or not). But, it seems that with smaller sample sizes the p-values can be different and that can be problematic.

If possible, could a future release of medmod give the option of computing t (and associated p) values instead of the Wald test (with z and p values) inherited from lavaan?
Posts: 3
Joined: Mon Feb 19, 2018 6:48 pm

by Ravi » Tue Feb 20, 2018 11:24 pm

So medmod indeed uses lavaan for its estimation while GAMLj uses the regular lm function. With medmod I wanted to create a package/module that makes it easy for people to use the power of lavaan for these type of analyses (and possibly also move on to lavaan for more complicated models). For this use case, I think it's important that the results are the same as the results in lavaan.
User avatar
Posts: 81
Joined: Sat Jan 28, 2017 11:18 am

by mcfanda@gmail.com » Wed Feb 21, 2018 3:56 pm

GAMLj uses lm() for the GLM, so this explains the differences
Posts: 119
Joined: Thu Mar 23, 2017 9:24 pm

Return to Module development