jamovi

Hiya,
Apologies if this has been posted previously, but I have a query about the number of post hoc tests calculated in a two-way ANOVA. jamovi appears to calculate all the possible combinations of condition pairings, rather than those that are required. For instance, in a simple (2*2) repeated measures design, whilst it is possible to calculate 6 paired post hoc tests, only 4 are required.

Let me give an example with a 2*2 factorial design (doesn't matter if it is between, repeated or mixed). I'm interested in the effect of a drug (ibuprofen vs. aspirin) and food (greasy fry-up vs. muesli) on hangover reduction. The interaction of these two IVs was significant and so I run post hocs.

I'm interested in four comparisons:
1. Ibuprofen & greasy food vs. Ibuprofen & muesli
2. Aspirin & greasy food vs. Aspirin & muesli
3. Greasy food & Ibuprofen vs. Greasy food & Aspirin
4. Muesli & Ibuprofen vs. Muesli & Aspirin

...however jamovi will also compare:
5. Greasy food & Aspirin vs. Muesli & Ibuprofen
6. Greasy food & Ibuprofen vs. Muesli & Aspirin

In my mind it makes no sense to run #5 & 6 - there is no common link between the combination of conditions i.e. different food type and different drugs are being compared. To me it's rather like comparing apples and oranges.

I could just pick and choose the 4 tests that I need from the post hoc table, but any corrections applied would be against all 6 tests and not the 4 I need.

This is just a long-winded way to ask is there a way to only run the post hoc tests that I need?

Cheers for any help.

p.s. great work on getting jamovi to run in a browser!

hey,

so the principle of post-hoc testing is that you have no principled reason for thinking one comparison more important than another. so i wonder if you don't want more of a 'planned comparisons' approach.

you could run the post-hocs without a correction, and then copy the 4 p-values into a correction procedure (but i'm not 100% sure this is allowed).

let's see if someone else can provide any insights.

jonathon

I think it is not a planned compared approach meaning I "manipulate" the procedure, but I plan to compare what is meaningful. There may be comparisons that in real life or relative to the research question have no sense or interest. Hence, the possibility to tell the software what comparisons will be required to answer the research question can be, in places, appropriate.

Regarding "you could run the post-hocs without a correction, and then copy the 4 p-values into a correction procedure (but i'm not 100% sure this is allowed)" I too am unsure if it's "allowed."

However, a straightforward way to do these planned comparisons, with adjustment for familywise error, would be to conduct such comparisons completely outside the ANOVA framework. After running the ANOVA, run four separate t tests and then use R's p.adjust function to correct the resulting four p values:
(I seem to recall that Jonathon started writing a 'Many-t' module to save the tedium of having to to run many separate t tests [4 is OK but 10 would be tedious]. Did that project ever get finished?)

> started writing a 'Many-t' module to save the tedium of having to to run many separate t tests

no, but it's probably not a bad project for a beginner if anyone wanted to run with it.

jonathon

Yeah, I started it a while ago but never finished it for some reason (See, https://github.com/raviselker/manytee). But in this module, all comparisons are done and you cannot just select the ones you are interested in.

Hey Ravi,
I took a look at your manytee module and it does well what it needs to do.
Why don't you bring it into the availability of jamovi's library?
I think it might be of interest to many.
TA, cheers
Maurizio