Hi Ravi and Jonathon,

I have been working on this myself in R. As you show, you get the same result using afex and emmeans in R, so it's not specific to jamovi. However, I think the jamovi developers might want to consider making a change here, because I'm not sure what is being reported is useful or makes sense.

My R script is below. If you run it in R, what you see is that calling emmeans on the afex model produces a warning:

Warning: EMMs are biased unless design is perfectly balanced

Running emmeans on the mixed model output, which produces what I think are the correct marginal means, does not give that error. So currently jamovi is reporting marginal means that emmeans itself calls "biased", but jamovi users are not seeing this warning.

I investigated a little further and figured out what emmeans is reporting here.

If we take the mean of the four cell means, we get 0.682. This is also the intercept from the mixed model (with deviation coded contrasts). This is the unbiased prediction for the overall mean for a sample that had equal numbers of young and old participants.

The mean of all observations in the DV in this sample is 0.450. It is different than the grand mean calculated above because there are twice as many young participants as old. In other words, this unweighted mean of all observations is biased toward the value for young participants.

The difference between the marginal means you get from the ANOVA and the mixed model is that all marginal means (main effects and interactions) are 0.232 less when calculated from the ANOVA. This is just 0.682 – 0.450.

In other words, both models produce the exact same pattern of marginal means (i.e., all differences between cells are the same), but the marginal means from the ANOVA are shifted so that the average of the cell means is the unweighted average of the DV—and therefore biased toward the expectation for young participants.

But it’s not just that the grand mean is biased toward the young participants. The way this gets applied to the cell means results in all the marginal means, for both young and old, being biased too low.

So these marginal means are *not* the model prediction for each cell. Both models (afex ANOVA and lmer) make the same predictions for the cell means, and these predictions are what are reported as marginal means from the mixed model. The marginal means reported for the ANOVA here are also not unbiased estimates of any useful population value. Perhaps there is some situation in which these emms would be useful, but I can’t think of what it is.

Unless there is something I’m missing here, I really don't think this is what jamovi should be reporting.

R code (the .csv file loaded in the code is attached):

- Code: Select all
`library(psych)`

library(afex)

library(lme4)

library(emmeans)

#Default to deviation contrasts

options(contrasts = rep("contr.sum", 2))

#Get data and make categorical variables factors

data <- read.csv("example.csv")

data$group <- factor(data$group)

data$cond <- factor(data$cond)

#Calculate afex ANOVA and report ANOVA and emmeans

aov.m <- aov_car(Y ~ cond*group + Error(sub_id/cond), data=data)

print(aov.m)

print(emmeans(aov.m, ~group*cond))

#Calculate lme4 lmer and report ANOVA and emmeans

lmer.m <- lmer(Y ~ cond*group + (1|sub_id), data=data)

print(anova(lmer.m, type=3))

print(emmeans(lmer.m, ~group*cond))

#Get the afex model prediction for a Young-A observation

#and the EMM for Young-A

#Then find the difference in these

ya_pred <- aov.m$lm$fitted.values[1, "A"]

ya_emm <- summary(emmeans(aov.m, ~group*cond))[2, "emmean"]

diff <- ya_pred - ya_emm

#Get the difference between the intercept

#(the expected mean in the DV if group ns were equal)

#and the mean of all observations

wm_diff <- summary(lmer.m)$coefficients["(Intercept)", "Estimate"] - mean(data$Y)

#These two values are the same (within floating point error)

(diff - wm_diff) < 1e-12