To pool or not to pool, that is the question: Change to Anova RM post-hoc tests in version 2.7.6

[Mik]

I noticed the change in the way post hoc test are carried in v.2.7.6 ("2.7.6 – 25 August 2025, Fix to Post-hoc in RM ANOVA,
https://www.jamovi.org/release-notes.html). Now jamovi assumes that variances are unequal and therefore, the standard error is not pooled anymore. I question this change, because the default assumption of the RM ANOVA test is that variances are equal. This was also explained here: viewtopic.php?t=114

The repeated measure ANOVA can of course be adjusted when the sphericity test is significant. This should however not be the default behaviour of the procedure to compute post hoc tests with unequal standard error.

Would it be possible to have an option to check where the user could ask jamovi to use non-pooled error terms for the post hoc tests ? I think that would be a better choice, because the use of non-pooled error terms might conflict with the default assumption of the RM ANOVA. That idea is also supported here: https://github.com/jamovi/jamovi/issues/660

[jonathon]

Hi,

The post-hocs in RM ANOVA were fixed in the 2.7.6, but this did not change the way they were calculated.

If you're getting different numbers between 2.6 and 2.7, then this is something we need to investigate. could you confirm that you're getting different numbers between the 2.6 and the 2.7?

with thanks

jonathon

[Mik]

Hi,

I do not know exactly when this change was implemented in jamovi because I was using an earlier release of the program. What I did notice however, is that the default jamovi behaviour is assuming that homogeneity of variance (HOV) is violated, which goes against the basic assumption of the RM ANOVA. If HOV is present, then the same standard error (SE) should be used for the post hoc comparisons. If you look at the first post here (https://github.com/jamovi/jamovi/issues/660) and also on earlier discussions on the jamovi forum and compare it to the image below, you will notice that jamovi now uses different SE for the marginal means.

jamovi EMM RM ANOVA.jpg (24.28 KiB) Viewed 2001 times

What I put forward is that presenting the results based on different SE should not be the common output. In order to make an informed choice, I suggest that the choice need to be made by the user, according to the behaviour displayed by the data.

Regards,

[reason180]

I don't know that the issue ever concerned post-hoc tests. Rather, it has to do with the error variances for the the estimated marginal means.

[Mik]

I don't know that the issue ever concerned post-hoc tests. Rather, it has to do with the error variances for the the estimated marginal means.

Hi, yes it was partially related to the issue of post hoc tests,

"... lsmeans to calculate the post-hoc tests which uses the pooled variance from all groups that are being compared. Therefore, all standard errors have the same value. Using the pooled variance has some benefits such as a higher power, and it's a bit more robust when there are only a few counts in certain cells."

(viewtopic.php?t=114)

see also: https://forum.cogsci.nl/discussion/5794 ... thoc-tests where it is stated

"That's how the emmeans works: it operates based on the model you specify. In the ANOVA model, there is the assumption of equal variances, so that influences how emmeans handles the pairwise comparisons. We offer a workaround, where (for balanced designs only) in the RM ANOVA you can choose to use the non-pooled error term for RM factors."

[Ravi]

Hi!

I did a little digging and found the commit where this change was implemented (4 years ago!): https://github.com/jamovi/jmv/commit/a9 ... c60e5b2fd2. So instead of using a univariate model to calculate the emmeans, a multivariate model is used. This change was implemented back then because of recommendations made by the author of the package that we use to perform the RM ANOVA (see also this topic: viewtopic.php?t=1924).

So this change has been around for quite a while, but of course we can discuss whether an option to use the univariate model instead would be useful.

Cheers,
Ravi