Significance tests for logistic regression

Discuss statistics related things

by DavidMShuker » Tue Nov 10, 2020 12:03 pm

Apologies if this has come up before, but I could not see it.

If I run a binary logistic regression using the "Binomial Logistic Regression" from the "Regression" tab I get different results to an analysis run using the "Generalised Linear Models" from the "Linear Models" tab (choosing latter case the logistic regression option).

My model has a 0,1 dependent variable, one factor, and two covariates. SPSS gives results more in line with the former analysis (although numbers still rather different from both).

It doesn't look like a type I/type III sums of squares-type issue, but...
DavidMShuker
 
Posts: 5
Joined: Tue Nov 10, 2020 11:48 am

by mcfanda@gmail.com » Wed Nov 11, 2020 3:22 pm

HI
could you please share an example of analyses that show the discrepancy?
User avatar
mcfanda@gmail.com
 
Posts: 217
Joined: Thu Mar 23, 2017 9:24 pm

by DavidMShuker » Fri Nov 13, 2020 3:05 pm

Sure - attached pictures of the two outputs from the same data (AIC/Deviances match, as do the results for the interaction terms, but not sure what is going on with the others. A different test/SS??
Attachments
Binomial logistic regression.png
Binomial logistic regression.png (16.39 KiB) Viewed 206 times
Generalised linear model.png
Generalised linear model.png (44.82 KiB) Viewed 206 times
DavidMShuker
 
Posts: 5
Joined: Tue Nov 10, 2020 11:48 am

by DavidMShuker » Mon Nov 23, 2020 3:10 pm

I'm kind of guessing no-one knows what the issue is here?
DavidMShuker
 
Posts: 5
Joined: Tue Nov 10, 2020 11:48 am

by mcfanda@gmail.com » Wed Nov 25, 2020 2:04 pm

It's very simple. Because you have interactions in the model, the main effect of each variable is computed for the other variables equal to zero (this is not a software issue, it's a statistical fact). Because you have factors, what zero means depends on the way the factors are coded. Binomial Logistic Regression codes the factors using the dummy method (0,1), so the main effects are simple effects computed for the reference group of the other (see Reference levels tab in the input tab).
"Generalised Linear Models" coded by default the factors with "simple" method, which essentially centers the coding to zero. Thus, the main effects you see are average effects (like in standard ANOVA).

That applies only to main effects, interactions (the highest order interactions in the models) are not affects by the factor coding
User avatar
mcfanda@gmail.com
 
Posts: 217
Joined: Thu Mar 23, 2017 9:24 pm

by DavidMShuker » Wed Nov 25, 2020 3:06 pm

Thanks for getting back in touch.

I think I see what you are getting at, although I have two queries. First, the main problem I have is that a binomial logistic regression is a generalised linear model with a logit link function and binomial errors, and if I run a GLZM on 0,1 data surely it should give the same result as what is termed here a binomial logistic regression with the same 0,1 data? (For example, you can call effectively the same model is SPSS via two routes, either from a logistic regression menu or from the generalised linear models menu.)

Second, since the effects are being tested with (omnibus) LR tests, as I am not sure how the explanation with respects to reference category works. I would get that if we were looking at coefficients with respect to a reference, with/without an intercept fitted, then differences in how the reference category is set-up (albeit somewhat hidden under the bonnet) could lead to different results, but I would not expect this in terms of LR tests of effects.

I hope that is not too garbled.
DavidMShuker
 
Posts: 5
Joined: Tue Nov 10, 2020 11:48 am

by DavidMShuker » Wed Nov 25, 2020 3:25 pm

Okay, have played with the factor coding in the GLZM module and if set the factors to be "Dummy" I retrieve the same results as from the binomial logistic regression (both factors contained either "a" or "b" for each observation. So that has solved that - thanks! Interesting to have never come across that problem before in other packages though. As logistic regressions are just glms, I wonder if there scope for confusion here...? Clarified it for me though, although still surprised it influences the LR tests of effects, and not just the tests of model coefficients, but I guess more reading needed.
DavidMShuker
 
Posts: 5
Joined: Tue Nov 10, 2020 11:48 am

by jonathon » Thu Nov 26, 2020 11:48 pm

fun fact about spss, if you run the univariate analysis it gives you two tables, an ANOVA table, and a parameter estimates table. you'd think the two tables are from the same analysis, but they're not! they're two separate analyses, using different coding for each!

jonathon
User avatar
jonathon
 
Posts: 1483
Joined: Fri Jan 27, 2017 10:04 am


Return to Statistics