gamlj will output a result when the model does not converge

Discuss statistics related things
Post Reply
User avatar
Arnaud Mortier
Posts: 32
Joined: Tue Apr 27, 2021 11:39 am

gamlj will output a result when the model does not converge

Post by Arnaud Mortier »

Hi there,

I am trying to run a power analysis of a GLMM (Gamma - log with mixed effects) using the MonteCarlo function.

It works perfectly up to one caveat: once in a while I get a warning that the simulated data resulted in a model that does not converge/is nearly unidentifiable/yields a huge eigenvalue/etc, in which case I would want MonteCarlo to ignore this data set and move on to the next repetition.

However, gamlj does output a result anyway in these cases, and apparently most of the time the specific effect I'm interested in appears to be significant. This results in a number of suspicious positives which pollute the overall analysis.

Does anyone have an idea on how to get rid of these cases, or perhaps (unlikely, I believe) how to argue that I can more or less trust the output anyway?

Thanks for any ideas!
Arnaud
User avatar
mcfanda@gmail.com
Posts: 457
Joined: Thu Mar 23, 2017 9:24 pm

Re: gamlj will output a result when the model does not conve

Post by mcfanda@gmail.com »

Hi,
as regards dealing with warnings, you can access them as follows:

Code: Select all

data("schoolexam")
mod<-gamlj::gamljGlmMixed(
  formula = pass ~ 1 + math*activity+( 1 +math| school ),
  data = schoolexam,
  modelSelection = "logistic",
  plotHAxis = math,
  correlatedEffects = "nocorr",
  cimethod = "wald")

problems<-FALSE
if (length(mod$info$notes)>0)
     problems<-mod$info$notes[[1]]$note

You can decide what to do with the output based on the content of `problems`. As regards trusting the results, it really depends on the issue and the application. "boundary (singular) fit" does not generally affect the estimates, it signals that you have small variances in the random effects. "huge eigenvalues" are mostly solved with changing variables cases, whereas "does not converge" is probably the case you do not want to trust. Nonetheless, it's difficult to make a generalization on these issues without inspecting the causes of the issue.
User avatar
Arnaud Mortier
Posts: 32
Joined: Tue Apr 27, 2021 11:39 am

Re: gamlj will output a result when the model does not conve

Post by Arnaud Mortier »

Thanks for your answer!
I was able to go through the process manually until I found a case where warnings were issued (11 of them at once in fact!).
The $info$notes[[1]]$note still contained only one line, namely 'R squared cannot be computed'. However I can access the warnings contents via warnings().
I then figured that I had not thought of preventing extreme outliers, which will necessarily happen over thousands of repetitions if you do nothing. I've done that now, there are still some warnings but they are much rarer.
Thanks again to you and Jonathon for all the help!
User avatar
Arnaud Mortier
Posts: 32
Joined: Tue Apr 27, 2021 11:39 am

Re: gamlj will output a result when the model does not conve

Post by Arnaud Mortier »

For the record, here is an example of a data set where the model will not converge and that has no outliers (well, maybe you can argue that Part 12 and Part 79 are outliers but the model still doesn't converge when they are filtered out).

The command I use is

Code: Select all

result<-gamljGlmMixed(formula = BD ~ 1 + tps + group + tps:group + ( 1 | part ), 
   data = df, 
   showParamsCI=TRUE, 
   showExpbCI = TRUE, 
   plotHAxis = tps, 
   plotSepLines = group,
   plotError = "se", 
   modelSelection="custom",
   custom_family="Gamma", 
   custom_link="log")
Attachments
culprit.csv
(9.33 KiB) Downloaded 566 times
User avatar
mcfanda@gmail.com
Posts: 457
Joined: Thu Mar 23, 2017 9:24 pm

Re: gamlj will output a result when the model does not conve

Post by mcfanda@gmail.com »

Hi,
thanks for the reply. Convergence is a big issue in mixed models. There's some literature about that. In terms of gamlj, it uses `glmer()` function of the `lme4` R package, so any information you find about that model function, regarding convergence and beyond, applies to gamlj as well.
User avatar
Arnaud Mortier
Posts: 32
Joined: Tue Apr 27, 2021 11:39 am

Re: gamlj will output a result when the model does not conve

Post by Arnaud Mortier »

Thanks, that's really useful because indeed I could find a lot of information about convergence issues related to that particular function.
Cheers
Arnaud
Post Reply