Post hoc testing after two-way ANOVA

Discuss statistics related things

by Bobafett » Thu Jul 16, 2020 7:22 pm

Apologies if this has been posted previously, but I have a query about the number of post hoc tests calculated in a two-way ANOVA. jamovi appears to calculate all the possible combinations of condition pairings, rather than those that are required. For instance, in a simple (2*2) repeated measures design, whilst it is possible to calculate 6 paired post hoc tests, only 4 are required.

Let me give an example with a 2*2 factorial design (doesn't matter if it is between, repeated or mixed). I'm interested in the effect of a drug (ibuprofen vs. aspirin) and food (greasy fry-up vs. muesli) on hangover reduction. The interaction of these two IVs was significant and so I run post hocs.

I'm interested in four comparisons:
1. Ibuprofen & greasy food vs. Ibuprofen & muesli
2. Aspirin & greasy food vs. Aspirin & muesli
3. Greasy food & Ibuprofen vs. Greasy food & Aspirin
4. Muesli & Ibuprofen vs. Muesli & Aspirin

...however jamovi will also compare:
5. Greasy food & Aspirin vs. Muesli & Ibuprofen
6. Greasy food & Ibuprofen vs. Muesli & Aspirin

In my mind it makes no sense to run #5 & 6 - there is no common link between the combination of conditions i.e. different food type and different drugs are being compared. To me it's rather like comparing apples and oranges.

I could just pick and choose the 4 tests that I need from the post hoc table, but any corrections applied would be against all 6 tests and not the 4 I need.

This is just a long-winded way to ask is there a way to only run the post hoc tests that I need?

Cheers for any help.

p.s. great work on getting jamovi to run in a browser!
Posts: 7
Joined: Thu Jul 18, 2019 11:33 am

by jonathon » Fri Jul 17, 2020 5:02 am


so the principle of post-hoc testing is that you have no principled reason for thinking one comparison more important than another. so i wonder if you don't want more of a 'planned comparisons' approach.

you could run the post-hocs without a correction, and then copy the 4 p-values into a correction procedure (but i'm not 100% sure this is allowed).

let's see if someone else can provide any insights.

User avatar
Posts: 1379
Joined: Fri Jan 27, 2017 10:04 am

Return to Statistics