Post-hoc planned and unplanned comparisons

Discuss the jamovi platform, possible improvements, etc.
Post Reply
CBrownstein
Posts: 1
Joined: Thu Dec 02, 2021 8:39 am

Post-hoc planned and unplanned comparisons

Post by CBrownstein »

I am running a two-way repeated measures ANOVA (exercise intensity x time), in which I have 3 intensities and 5 time points.

I have a signifcant intensity x time interaction, and so ran the post-hoc using Tukey corrections.

However, for the corrections, there are many comparisons which I not interested in. For example, let's say the intesities are 100, 80 and 60% of peak power output, and the time-points are baseline, 25, 50, 75 and 100% task completion. I am only interested in whether within each intensity, there is a change over time relative to baseline (e.g. does my dependent variable change over time for the 100% peak power output trial), and whether at a specific time points (e.g. 50% task completion), there is a difference between the 3 intensities.

Yet, the post-hoc corrects for a number of unplanned comparisons (e.g. it will compare 25% task completion at 100% peak power output vs. 75% task completion from 80% peak power output).

Is there any way that I can specify only the planned comparisons which I am interested in, so that the p value is not adjusted for all these unplanned comparisons?
User avatar
jonathon
Posts: 2613
Joined: Fri Jan 27, 2017 10:04 am

Re: Post-hoc planned and unplanned comparisons

Post by jonathon »

hi,

no this isn't available in the RM ANOVA. probably your best bet is to use no-correction, and then pluck out the p-values you are interested in, and correct these.

the p.adjust() method you could run from the rj editor.

cheers

jonathon
pkj
Posts: 2
Joined: Mon Oct 24, 2022 1:21 pm

Re: Post-hoc planned and unplanned comparisons

Post by pkj »

Hello,

I am wondering if the same solution could be applied to post-hoc done in GLM models?

In my specific case, I have a significant 3-way interaction that I would like to further explore by doing a post-hoc comparison. However, I am not interested in all the comparisons, but only the ones that are relevant to the research question. Still, when doing a Bonferroni correction in Jamovi, it would correct for all the possible comparisons, making the end result too strict and risk not detecting true significance?

If I understood this previous answer right, would the solution then be to look at the non-corrected p-values and then to correct them manually? So for example, instead of taking the automatic Bonferroni correction for the total of 24 comparisons, I would take the usual alpha=0.05 and divide it by the number of comparisons I am interested in (e.g. 0.05/6) and then check the non-corrected p-values to see which ones fall below that cut off point? Would that be a recommended solution to this problem or is there another preferred way?

Thank you, any advice or input would be much appreciated!
Post Reply