jamovi

Hi everyone,

I have conducted a MANCOVA analysis using 4 DVs, 1 IV (of 3 groups), and 4 covariates. The overall result of the IV effect that I get on jamovi is F(8,1220) = 3.34, Wilk’s Λ = .958, p < .001.

However, a student decided to run the same analysis on SPSS and her result was different. The overall MANCOVA IV effect was F(8,1220) = 0.836, Wilk’s Λ = .989, p = .571. So the interpretation here is the opposite.

I talked with a statistician who decided to run this on R. She got the same result as jamovi and SPSS, depending on the order of the variables! Her two models were something like this:

1) result1 <- manova(cbind(DV1, DV2, DV3, DV4) ~ CoV1 + CoV2 + IV, data= FileName)
summary (results)

2) result1 <- manova(cbind(DV1, DV2, DV3, DV4) ~ IV + CoV1 + CoV2, data= FileName)
summary (results)

*IV = independent variable, DV = Dependent variables, CoV = Covariates

The first code uses covariates in front of the IV, and the second code uses the IV before the covariates, that is the only difference. However, the first code gives the results of SPSS and the second code give the result of jamovi. Why are we observing this difference? What is the correct result?

Thank you in advance for your help.

hi,

i hope someone else knows, because i don't!

jonathon

Hi jonathon,

thank you for your response. Maybe it is a problem in R...?

Best,
Gabriel

i wouldn't assume it's a problem. changing the order of things, can often change the results of an analysis.

i think that can happen with an ANOVA if you use a particular sum of squares.

jonathon

Hi Jonathon,

I think we figured why this is happening. First, we ran the same analysis in stata. We got the same result as SPSS.

Then we searched over the internet, and we found out that the order of variables matters in R. Covariates should go first in R code for ANCOVAs, please see this link: https://www.datanovia.com/en/lessons/ancova-in-r/#:~:text=The%20orders%20of%20variables%20matters,you%20will%20get%20different%20results.

Because when we use the IV before the covariates in R we get the same results as jamovi, we believe that maybe the order of variables is incorrect in jamovi for the MANCOVA test, since it is based on R? Please let me know what you think.

I hope this can contribute to further develop jamovi

Cheers,
Gabriel

Gabriel Tiraboschi wrote:...we believe that maybe the order of variables is incorrect in jamovi for the MANCOVA test, since it is based on R? Please let me know what you think.

Hey Gabriel,
thanks for your contribution here.

If the code used in jmv for MANCOVA had been Python with the statsmodels library there would have been differences with respect to R, SPSS, Stata etc.
Now, your model, if there were no covariates, would be a one-way MANOVA with no interaction, as there is only one IV (factor) present.
Under the hood of jamovi is code (R v4.1.2) with its own libraries and other contributors following the rules for passing variables for the generated model terms.
In the analysis module (jmv) installed with jamovi, MANCOVA is the daughter of the stats package of R, which makes available the manova() function, used to obtain a MANCOVA.
Here in the jmv module, you can see the source code for MANCOVA (file mancova.b.R):
https://github.com/jamovi/jmv/blob/master/R/mancova.b.R

As you can see on line 98, there is a call to the function, whose first argument must be a formula to use to get a MANOVA or a MANCOVA.
This formula has a left part which is the DV binding and a right part which represents the model terms without covariates (MANOVA) or with covariates (MANCOVA), but the first term must be (as a rule) for the IVs (factors) and if there are covariates (covs), they must be subsequent to IV.
If more than one IV is present, there will also be interaction (*).
You can follow the above steps for building the formula with model terms from line 394 and finally look at line 445 where the formula is completed with the two left and right parts, which will be the first argument of the stats::manova( ) to be a MANOVA or a MANCOVA.

What leads to the differences we find in the methodology used to divide the variation, since R has predefined Sums of Squares of Type I, Python of Type II, SPSS of Type III (They do not give the same result in case of unbalanced data) .

The sums of Type I squares are sequential, so the order of the variables in the models makes a difference.

Different software: different results?
There is debate as to which type of sums of squares to use.

Cheers,
Maurizio

Hi MAgojam,

thank you so much for your answer. I have to discuss this with our statistician and I will get back to you.
For us, it is really important to understand this because this is the difference between rejecting the null hypothesis or not, which completely changes the interpretation of our study.

Cheers,
Gabriel

Hi Maurizio,

again, thank you so much for your kind explanation.

I spoke with my lab statistician, and it seems that indeed the difference is between the sums of squares. Because MANCOVA results were so contradicting, we decided to run another analysis.

If allow us to offer a modest suggestion: We think that a cool feature to implement in Jamovi for the future would be an option to select different types of sums of squares for MANCOVA, specially because type I is argued to not be the most recommended for unbalanced data that we usually have in social sciences.

Keep up the amazing work that is Jamovi
Gabriel

Gabriel Tiraboschi wrote:If allow us to offer a modest suggestion: We think that a cool feature to implement in Jamovi for the future would be an option to select different types of sums of squares for MANCOVA

Thanks Gabriel,
I think it's a good suggestion to consider for possible better usability of MANOVA/MANCOVA.

Maurizio