Suggestion for ANOVA/ANCOVA modules in jmv

Everything related to the development of modules in jamovi
Post Reply
User avatar
mcfanda@gmail.com
Posts: 457
Joined: Thu Mar 23, 2017 9:24 pm

Suggestion for ANOVA/ANCOVA modules in jmv

Post by mcfanda@gmail.com »

HI
I had the opportunity to check out the contrasts in the ANOVA/ANCOVA modules and I found some strange results for the contrasts named deviation, simple and difference.

deviation labels are defined with the wrong reference group.
In case of three groups, for instance, the labels one gets now are:
2-1,2,3 (2 against the overal mean)
3-1,2,3 (3 against the overal mean)
but the contrasts parameters one actually gets correspond to the following labels:
(1,2,3)-2
(1,2,3)-3
I've looked into the module code and for some reason the "deviation" contrasts
is customly computed. One can use contr.sum() instead to get the right parameters.

Also simple is weird. Contrast simple (as defined by SPSS) is basically dummy coding 0-1, so
each contrast estimate should be the difference between the group scoring 1 in that contrast and the reference group. That's what you get in SPSS and in R using contr.treatment()
However, in the jamovi modules you get the actual difference divided by K, where
K is the number of groups. The inferential test are correct, but the parameter estimate is confusing.

The contrast difference is also defined differently of what the labels say and what you would expect from SPSS and standard R. Assuming again three groups, the first contrast one gets now is the difference between 2 and 1 groups divided by 2. The second is group 3 minus mean(1,2) dived by 3. Again, this behaviour is unexpected by users moving to jamovi from SPSS or SAS.

Because I was using the ANOVA module code for a module I'm developing, I updated the code found in ANOVA and ANCOVA modules such that it now gives contrasts yielding the same results of SPSS and standard R

Code: Select all

if (type == 'simple') {
  
  contrast <- contr.treatment(levels)
  dimnames(contrast) <- NULL
  
} else if (type == 'deviation') {
  
  contrast <- stats::contr.sum(levels)
  dimnames(contrast) <- NULL
  
} else if (type == 'difference') {
  
  contrast <- stats::contr.helmert(levels)
  for(i in 1:ncol(contrast))
    contrast[,i]<-contrast[,i]/(i*2)
  dimnames(contrast) <- NULL
  
}

I might be wrong, of course, and did not understand the intentions of the module design.
marcello
User avatar
jonathon
Posts: 2613
Joined: Fri Jan 27, 2017 10:04 am

Re: Suggestion for ANOVA/ANCOVA modules in jmv

Post by jonathon »

great! thanks for looking into this, we'll take a look.

jonathon
User avatar
Ravi
Posts: 194
Joined: Sat Jan 28, 2017 11:18 am

Re: Suggestion for ANOVA/ANCOVA modules in jmv

Post by Ravi »

So if I understand it correctly, the estimates are different from SPSS but the p-value the same? That's indeed because we coded the contrasts in a slightly different way; both ways are correct I think. However, we do want to emulate the results from SPSS as much as possible so I will look into changing this.

Thanks!
User avatar
mcfanda@gmail.com
Posts: 457
Joined: Thu Mar 23, 2017 9:24 pm

Re: Suggestion for ANOVA/ANCOVA modules in jmv

Post by mcfanda@gmail.com »

Well, the p-values are different for simple but not for difference, but I don't think that's the problem (ultimately, spss can be wrong too :-) ). The point is that the contrasts estimates do not correspond to the labels, so if one wants to interpret the estimate (something that people one day way start to do), they will get confused. My suggestion is to code the contrasts in a way that exactly corresponds to what the comparisons are.
User avatar
Ravi
Posts: 194
Joined: Sat Jan 28, 2017 11:18 am

Re: Suggestion for ANOVA/ANCOVA modules in jmv

Post by Ravi »

I looked at the code and there are indeed some inconsistent (and in case of the deviation coding, incorrect) things. I used this website to check the coding of the contrasts.

For deviation the sign was wrong, so I changed that. Simple contrast is not exactly the same as dummy coding:
The difference in the regression output between dummy coding and simple coding scheme is in the intercepts. In the dummy coding scheme, the intercept corresponds to the cell mean of the reference group, while in the simple coding scheme, the intercept corresponds to the mean of cell means.
.

The current coding is not wrong, but to be consistent with the earlier mentioned website the simple coding is now divided by k. The deviation contrast was not consistent with the helmert contrast, which it should be because it's just a reversed helmert contrast. I fixed this too.

Thanks again for reporting this! It's really helpful for us when people (double)check the results jamovi gives.
User avatar
mcfanda@gmail.com
Posts: 457
Joined: Thu Mar 23, 2017 9:24 pm

Re: Suggestion for ANOVA/ANCOVA modules in jmv

Post by mcfanda@gmail.com »

Hi,
1) what is your opinion about adding a "dummy" contrast (maybe called "treatment") as a jamovi standard, so that people can use it as SPSS and R use it in almost any linear model. In this way jamovi keeps up with SPSS definition of "Simple" but gives an extra option of contrasting which is well-known and expected?

2) May I have access to the new code of contrasts so that I can use it in the modules I'm developing?
thanks
User avatar
Ravi
Posts: 194
Joined: Sat Jan 28, 2017 11:18 am

Re: Suggestion for ANOVA/ANCOVA modules in jmv

Post by Ravi »

1) I don't think I'm opposed to the idea of adding a dummy contrast. So SPSS uses the dummy contrast as the simple contrast?
2) You can find the changes I made in this pull request: https://github.com/jamovi/jmv/pull/124/ ... 62882b18d7

Cheers
User avatar
mcfanda@gmail.com
Posts: 457
Joined: Thu Mar 23, 2017 9:24 pm

Re: Suggestion for ANOVA/ANCOVA modules in jmv

Post by mcfanda@gmail.com »

1) I mean that SPSS uses it to code the categorical variables when it estimates the model coefficients in GLM, MIxed and generalized linear models.
2) Great, thanks
pao
Posts: 9
Joined: Thu Oct 05, 2017 7:03 pm

Re: Suggestion for ANOVA/ANCOVA modules in jmv

Post by pao »

Ravi wrote: I used this website to check the coding of the contrasts.
Difference forward coding on site above is named "repeated" in Jamovi, but both are wrong :-(

This contrast compare in that way:
a - b,c,d
a,b - c,d
a,b,c - d

NOT that:
a - b
b - c
c - d
User avatar
Ravi
Posts: 194
Joined: Sat Jan 28, 2017 11:18 am

Re: Suggestion for ANOVA/ANCOVA modules in jmv

Post by Ravi »

I think the contrasts for forward difference coding described on that website are correct. If you check the difference between means manually you see that they correspond to the estimate you get when using those contrasts.
Post Reply