gamlj3: Trying to achieve cross-application replication of generalized linear mixed effects model

[reason180]

@mcfanda

I know it's sometimes difficult to address questions about why statistical results differ from one application (jamovi) to another, very similar application. However, I do think it's scientifically useful to strive for cross-application replicability.

I have a generalized linear mixed effects model that includes two fixed-effects factors on one random-effect variable. I've tried everything I can think of to get the same omnibus test results in jamovi as I do in another particular stats app, but I've been unable to (and even various model estimates differ quite a bit across the two apps).

Is there something obvious I'm doing wrong? Or are there some hidden model parameters that are making things turn differently across the two apps? (I've emailed mfanda the analysis/data files.)

[mcfanda@gmail.com]

jasp uses deviation constrasts. So, in gamlj set constrats to "deviation" coding. However, your model has singular fit, so results are not exactly the same because gamlj tries different optimizers (i do not know how jasp handles these cases). Nonetheless, results are pretty similar, rounding aside

[mcfanda@gmail.com]

you should also be sure that the random component is the same in jasè and jamovi. If I am not mistaken, in gamlj you did not include the random intercept.

[mcfanda@gmail.com]

[this is equivalent to gamlj results in R

Code: Select all

file<-"local/data/mer_jasp.csv"
data<-read.csv(file)
data$Framing<-factor(data$Framing)
contrasts(data$Framing)<--contr.sum(2)/2
data$BothDisliked<<-factor(data$BothDisliked)
contrasts(data$BothDisliked)<--contr.sum(2)/2
data$Participant<-factor(data$Participant)
library(lmerTest)
data$PrefToAbstain<-factor(data$PrefToAbstain)
mod<-lme4::glmer(PrefToAbstain~Framing*BothDisliked+(Framing*BothDisliked|Participant),
                 data=data,
                 family=binomial(),
                 control = lme4::glmerControl(optimizer = "bobyqa"))
summary(mod)
car::Anova(mod,type=3,test="Chisq")

[reason180]

Thanks. The JASP file did include the interecept, it's just that a test of the intercept location wasn't included in the output (the model runs the same regardless of whether the slope is tested) . . .
-

Untitled.png (311.6 KiB) Viewed 2921 times

-
Consistent with what you wrote about the optimizer, I did come across this post, which suggests that the optimizer choice is different in JASP compared to gamlj3 in jamovi:
https://svmiller.com/blog/2018/06/mixed ... er-checks/

[mcfanda@gmail.com]

great