Testing Normality of Residuals in Repeated and Mixed ANOVA

[Wake]

Hi,

My understanding is that the SPSS method of saving and testing a residual for each level of the repeated measures variable(s) is incorrect. However, at the moment jamovi doesn't offer an alternative.

I've just written slides recommending that our students compute an average and a difference of the repeated measures, i.e. MEAN(L1,L2,L3) and L1-MEAN(L2,L3), and then run one-way ANOVA or t-tests on those scores in order to run normality tests. For the difference score I feel like I should add a second orthogonal contrast (e.g. L2-L3) but equally, since there is only one within-subjects residuals term surely the is a single set of residuals? The degrees of freedom for the residuals suggest levels-1 residuals per participant, but does it matter which orthogonal contrasts are chosen. Of course, ideally I would like to test the normality of two difference columns simultaneously without having to copy and paste them into extra rows -- which is beyond what I'm happy to ask my students to do.

Best wishes,

Wakefield

[jonathon]

what *should* we do here?

jonathon

[Wake]

Hi,

I think that the correct answer is to allow the user to ask for normality tests and Q-Q plots which are then computed separately for each residual term.

For the Between Subjects Effects the residuals are based on the independent ANOVA of the between subject IVs and covariates on the mean of the repeated columns.

For the Within Subjects effects I think that the residuals should be based on the independent ANOVA of the between subject IVs and covariates on a set of orthogonal contrasts of the repeated columns (i.e. L1-L2 for two levels; L1-MEAN(L2,L3) and L2-L3 for three levels; L1-MEAN(L2,L3,L4) and L2-MEAN(L3,L4) and L3-L4 for four levels, etc (where L1 is the reference level). Ideally this would be a single test/Q-Q plot per residual so that the user isn't faced with multiple plots/tests for a single normality assumption.

It may be that this procedure is flawed, for example if the selection of which contrasts get used is critical. However, I'm sure that it is better than the SPSS residuals (which are demonstrably false since they don't simplify to the same as a paired t-test for a single repeated IV with two levels).

This is the closest I've found on the subject so far https://psych.wisc.edu/Brauer/BrauerLab/wp-content/uploads/2014/04/Murrar-Brauer-2018-MM-ANOVA.pdf

Cheers,

Wake

[jonathon]

yeah, i find it remarkably difficult to find answers to these "bread-and-butter" questions in statistics.

i really feel like the stats community have done a really poor job at empowering consumers of statistics. (exhibit A, jamovi is being written by psychologists!)

anyway, that's my rant.

jonathon

[coledavis]

I think that trouble (or virtue?) is that statistical tests have been developed in diverse ways. Various tests were created by psychologists, not mathematicians. Others, such as survival analysis, have different names because of applications in various academic fields (where it becomes event or duration analysis etc). Which is probably why lots of statistical books are either very different in their presentations, or are books designed to look similar (although superficially different) from other books.

In my investigations of statistical analysis for my books, it is amazing how many different terms are used for the same phenomena. Given the antipathy or maybe indifference between mathematically focused statisticians and applied statistics users, I can't see this changing anytime soon.

[mcfanda@gmail.com]

HI
I think that testing the normality of the residuals of the differences contrasts has its merit,, but I would suggest to implement it with a more general approach in jamovi. The reason is that testing each individual contrast residual separately does not guarantees a full test of normality. In RM ANOVA, in fact, the assumption is on the multivariate normality of the residuals, whereas the mentioned procedure tests marginal normality. The latter is a necessary condition of the former, but not a sufficient one.

Because there are tests of multivariate normality available, I'd suggest to implement one of those (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3927875/). The multivariate tests can be applied to the residual of the contrasts. Which contrast scheme one uses is immaterial, because if multivariate normality holds for one linear combination of the original variates, it holds for any linear combination of them.

Consider, however, that the contrasts approach may get a bit complex as the anova design expands. A simpler solution would be to test the multivariate normality of the residuals, computed as the original dependent variates minus their means (that are the estimated marginal means of the model).