[RESOLVED] Normality assumption

Discuss the jamovi platform, possible improvements, etc.
Manuel
Posts: 6
Joined: Fri Nov 02, 2018 12:46 pm

[RESOLVED] Normality assumption

Post by Manuel »

Dear Jamovi team:

First, I think Jamovi is a very nice statistical piece of software, very useful for teaching (I am thinking of using it).

Suggestions for improving:

- In t-test and ANOVA the normality assumptions are not carried out by group.
- LSD and Scheffe tests in post hoc analysis would be very useful.

Regards
Manuel
User avatar
Ravi
Posts: 194
Joined: Sat Jan 28, 2017 11:18 am

Re: Normality assumption

Post by Ravi »

Hi Manuel,

Thanks for the kind words. About the suggestions:
- in general linear models (e.g., t-tests, and ANOVA's) the assumption is that the residuals are normally distributed, and every model just has one residual term. So that's why they are not carried out by group.
- Have you looked at the regular ANOVA? There you can already compute Scheffe post-hoc tests (and I think you get the LSD test by picking "no correction".

Cheers,
Ravi
User avatar
jonathon
Posts: 2613
Joined: Fri Jan 27, 2017 10:04 am

Re: Normality assumption

Post by jonathon »

In t-test and ANOVA the normality assumptions are not carried out by group.
you can do this with descriptives though, if you "split by" the group.

jonathon
Manuel
Posts: 6
Joined: Fri Nov 02, 2018 12:46 pm

Re: Normality assumption

Post by Manuel »

Thank you for you prompt reply. I thought it was the prior assumption of normality. Sorted!
adaren
Posts: 20
Joined: Tue Jan 03, 2023 12:25 pm

Re: [RESOLVED] Normality assumption

Post by adaren »

Hello,

First of all, thanks for jamovi - it is a really accessible package with great features and capability! It is so friendly for beginners ;)

That said, I am a bit apprehensive of this single normality test for residuals of various groups together. Could you refer me to some source you know explaining such practice?

(I am aware of this other thread too: https://forum.jamovi.org/viewtopic.php?f=6&t=1350&hilit=Shapiro#p4741, Jonathon had said the practice is accepted, but I only found it in jamovi)

One reason is that normality may be fine in separate groups, while violated in the single normality test in jamovi (e.g. t-test for independent samples).
E.g. testing two samples of normally distributed X data (different means and sds) - generated as per r code below - will result in p < 0.001 for Shapiro-Wilk in the independent samples t-test module, whereas the values for the two groups separately are well above 0.05 (0.328 and 0.776).

data:

set.seed(555)
a <- rnorm(100, 10, 4)
as <- (a-mean(a))/sd(a)
b <- rnorm(100, 20, 8 )
bs <- (b-mean(b))/sd(b)
X <- c(a,b)
Z <- c(as,bs)
group <- c(rep(1,100), rep(2,100))
abba <- data.frame(X,Z,group)
#export abba to csv
User avatar
reason180
Posts: 274
Joined: Mon Jul 24, 2017 4:56 pm

Re: [RESOLVED] Normality assumption

Post by reason180 »

If you have different variances across groups, then you're already violating the assumptions of ANOVA (except Welch's OneWay ANOVA). But if you still want a normality test by group then (among already-mentioned options), you can just do a one-sample t test on each group.
adaren
Posts: 20
Joined: Tue Jan 03, 2023 12:25 pm

Re: [RESOLVED] Normality assumption

Post by adaren »

You probably misunderstood my question.
I would like to get to know the rationale for using the normality testing based on residuals grouped together. (if it's common practice, I suppose it has some reasonable grounding/literature - because as you can see in my example two methods give different results).


To relate to what you wrote:
Of course, I am considering a practical situation, where you can have two groups coming from different populations and with different variances.
Of course, you need to use Welch's t-test to analyze that.

But here I am asking about practical issues related to testing normality for such a case. When testing in two groups separately, the results are different (and the data comes from normal distributions for both groups, as you can see from the code) than when testing using the "assumptions" within the t-test module.

I know I have the other option in exploration. But why not use that also in t-test module if results may differ?

So, do you know any sources recommending/ giving the basis for the method used in jamovi's t-test module (and other)?
User avatar
reason180
Posts: 274
Joined: Mon Jul 24, 2017 4:56 pm

Re: [RESOLVED] Normality assumption

Post by reason180 »

FYI:

(1) For the 'Regular' (equal-variance) ANOVA:

"Normality of residuals: The errors used for the estimation of the error term(s) (MSE) are normally distributed. This can be inferred using performance::check_normality()."
https://cran.r-project.org/web/packages/afex/vignettes/assumptions_of_ANOVAs.html

Thus, the assumption concerns the residuals that pertain to the ANOVA model component--MSE--rather than residuals pertaining to a one or more one-group t tests. This makes sense because the regular ANOVA assumes equal variances across groups. Therefore, individual-group normality tests shouldn't be included in the output for a regular ANOVA.

(2) Problematic Comparison of p Values

Being a null-hypothesis significance test, the Shapiro-Wilk test has less power to reject the null hypothesis (i.e., that the residuals are normally distributed) as the sample size, N, gets smaller. If you do such a test on the residuals for the whole model you have a relatively large sample and thus relatively high power. If you do it separately for each group then you have relatively small Ns and thus relatively low power. So even if all groups were to similarly approximate normality, Shapiro-Wilk p values would tend to be higher for the individual groups than for the entire data set.

(3) For Welch's ANOVA:

I see this as a different sort of problem. It seems to me that one might need to use low power normality tests for each group separately in order to get a valid result. I'm not even sure how an ANOVA-model-based Shapiro-Wilk Test could be done in this case. I see that when the jamovi one-way ANOVA outputs Shapiro-Wilk, it's always identical regardless of whether one chooses the Fisher's (i.e., equal variance) version of ANOVA or Welch's version. (Is the normality test being done only for Fisher's only?)

I don't know if jamovi (or anyone else) does this, but if it were up to me I might always do the following when conducting a Shapiro-Wilk normality test of ANOVA--especially Welch's ANOVA: Step 1 would be to standardize the scores within each group, thereby making all of the variances (and all of the means) equal. Step 2 would be to perform the standard Shapiro-Wilk test of normality on the transformed data. (This would be for the model, not for the individual groups.)
adaren
Posts: 20
Joined: Tue Jan 03, 2023 12:25 pm

Re: [RESOLVED] Normality assumption

Post by adaren »

Thanks for pointing (1) - yet it does not state the residuals need to be combined from all of the groups together,
agreed on (2),
agreed on the problematic case of (3) - that's what made me question whether combined residulals method is the best for normality testing in general,
I've tested normality with Z-standardized values (present in my R-code example) - gives the same result as the jamovi t-test method.

I have never seen it stated in the literature, that testing residuals combined across the groups is the method to use. I suppose testing each group individually is the more "basic" method of checking this assumption.
I made this R code example so that the groups are not small (n=100) and still Shapiro-Wilk is far from 0.05 and Q-Q plots look fine. They should as the numbers were rnorm() generated ;D. Yet n=200 gives this highly significant result.

In general, if there really were two different populations our sample came from, I would generally expect both means and sds to differ.
So I would place my bets on testing grouped data/residuals separately. This is why I wonder how come no literature states it directly that combined residuals may be tested, whereas many say it should be tested for each group (e.g. Andy Field's "Discovering Statistics ... " for psychology students).
User avatar
reason180
Posts: 274
Joined: Mon Jul 24, 2017 4:56 pm

Re: [RESOLVED] Normality assumption

Post by reason180 »

In ANOVA, there is an MSE for the model. There is no MSE for each group.
Post Reply