Hi there,
I am trying to run a power analysis of a GLMM (Gamma - log with mixed effects) using the MonteCarlo function.
It works perfectly up to one caveat: once in a while I get a warning that the simulated data resulted in a model that does not converge/is nearly unidentifiable/yields a huge eigenvalue/etc, in which case I would want MonteCarlo to ignore this data set and move on to the next repetition.
However, gamlj does output a result anyway in these cases, and apparently most of the time the specific effect I'm interested in appears to be significant. This results in a number of suspicious positives which pollute the overall analysis.
Does anyone have an idea on how to get rid of these cases, or perhaps (unlikely, I believe) how to argue that I can more or less trust the output anyway?
Thanks for any ideas!
Arnaud
gamlj will output a result when the model does not converge
- Arnaud Mortier
- Posts: 32
- Joined: Tue Apr 27, 2021 11:39 am
- mcfanda@gmail.com
- Posts: 462
- Joined: Thu Mar 23, 2017 9:24 pm
Re: gamlj will output a result when the model does not conve
Hi,
as regards dealing with warnings, you can access them as follows:
You can decide what to do with the output based on the content of `problems`. As regards trusting the results, it really depends on the issue and the application. "boundary (singular) fit" does not generally affect the estimates, it signals that you have small variances in the random effects. "huge eigenvalues" are mostly solved with changing variables cases, whereas "does not converge" is probably the case you do not want to trust. Nonetheless, it's difficult to make a generalization on these issues without inspecting the causes of the issue.
as regards dealing with warnings, you can access them as follows:
Code: Select all
data("schoolexam")
mod<-gamlj::gamljGlmMixed(
formula = pass ~ 1 + math*activity+( 1 +math| school ),
data = schoolexam,
modelSelection = "logistic",
plotHAxis = math,
correlatedEffects = "nocorr",
cimethod = "wald")
problems<-FALSE
if (length(mod$info$notes)>0)
problems<-mod$info$notes[[1]]$note
- Arnaud Mortier
- Posts: 32
- Joined: Tue Apr 27, 2021 11:39 am
Re: gamlj will output a result when the model does not conve
Thanks for your answer!
I was able to go through the process manually until I found a case where warnings were issued (11 of them at once in fact!).
The $info$notes[[1]]$note still contained only one line, namely 'R squared cannot be computed'. However I can access the warnings contents via warnings().
I then figured that I had not thought of preventing extreme outliers, which will necessarily happen over thousands of repetitions if you do nothing. I've done that now, there are still some warnings but they are much rarer.
Thanks again to you and Jonathon for all the help!
I was able to go through the process manually until I found a case where warnings were issued (11 of them at once in fact!).
The $info$notes[[1]]$note still contained only one line, namely 'R squared cannot be computed'. However I can access the warnings contents via warnings().
I then figured that I had not thought of preventing extreme outliers, which will necessarily happen over thousands of repetitions if you do nothing. I've done that now, there are still some warnings but they are much rarer.
Thanks again to you and Jonathon for all the help!
- Arnaud Mortier
- Posts: 32
- Joined: Tue Apr 27, 2021 11:39 am
Re: gamlj will output a result when the model does not conve
For the record, here is an example of a data set where the model will not converge and that has no outliers (well, maybe you can argue that Part 12 and Part 79 are outliers but the model still doesn't converge when they are filtered out).
The command I use is
The command I use is
Code: Select all
result<-gamljGlmMixed(formula = BD ~ 1 + tps + group + tps:group + ( 1 | part ),
data = df,
showParamsCI=TRUE,
showExpbCI = TRUE,
plotHAxis = tps,
plotSepLines = group,
plotError = "se",
modelSelection="custom",
custom_family="Gamma",
custom_link="log")
- Attachments
-
- culprit.csv
- (9.33 KiB) Downloaded 579 times
- mcfanda@gmail.com
- Posts: 462
- Joined: Thu Mar 23, 2017 9:24 pm
Re: gamlj will output a result when the model does not conve
Hi,
thanks for the reply. Convergence is a big issue in mixed models. There's some literature about that. In terms of gamlj, it uses `glmer()` function of the `lme4` R package, so any information you find about that model function, regarding convergence and beyond, applies to gamlj as well.
thanks for the reply. Convergence is a big issue in mixed models. There's some literature about that. In terms of gamlj, it uses `glmer()` function of the `lme4` R package, so any information you find about that model function, regarding convergence and beyond, applies to gamlj as well.
- Arnaud Mortier
- Posts: 32
- Joined: Tue Apr 27, 2021 11:39 am
Re: gamlj will output a result when the model does not conve
Thanks, that's really useful because indeed I could find a lot of information about convergence issues related to that particular function.
Cheers
Arnaud
Cheers
Arnaud