Lettering based on Tukey test

Discuss statistics related things

by Shahid » Sun May 14, 2017 11:43 am

I have just started using jamovi on my MacBook. I have one question in mind.

Does jamovi give lettering (abc...) and value of Tukey's HSD in post-hoc test (Tukey's HSD test)? I am unable to find it. Most of the times we present our results as means followed by lettering (lettering show mean comparisons based on some post-hoc test). So, we need lettering.

Waiting for help.
Shahid
 
Posts: 2
Joined: Sun May 14, 2017 11:36 am

by Ravi » Sun May 14, 2017 10:23 pm

Hi Shahid,

I'm not sure that I completely get what you mean with 'lettering'. Could you elaborate on that?

Cheers
User avatar
Ravi
 
Posts: 154
Joined: Sat Jan 28, 2017 11:18 am

by Shahid » Sat May 20, 2017 6:41 am

If significant based on ANOVA, we run Tukey test to rank means. For example we have four treatments with treatment averages = T1 23 ; T2 25; T3 19; T4 5. After mean ranking by post-hoc test (Tukey test), output might be:
T2 25 A
T1 23 AB
T3 19 B
T4 5 C
The ranking is designated by letters. I am referring to these as lettering. Will it be possible for you to produce results of Tukey test in such a format?
Shahid
 
Posts: 2
Joined: Sun May 14, 2017 11:36 am

by Ravi » Mon May 22, 2017 2:38 pm

We like to keep our main analyses as simple and clean as possible. I think the output most people are looking for here are the Tukey corrected p-values, and adding more results might confuse a lot of people. However, we encourage people to make modules for jamovi so if this is functionality a lot of people want someone (you?) will hopefully come along and start implementing this.
User avatar
Ravi
 
Posts: 154
Joined: Sat Jan 28, 2017 11:18 am

by Webolive » Fri Sep 25, 2020 8:51 am

I am surprised not to find more requests about this. As far as I know, lettering (a.k.a compact letter display) is very common in many fields of research and is a standard in many statistical packages. But I also think I understand why it is not implemented in Jamovi:

Jamovi's ANOVA post-hoc analysis is done with the emmeans package. Its author (Russel Lenth) has removed the CLD function (which produced the letter grouping) from the package. He explains the reason as follows:

"Another way to depict comparisons is by compact letter displays, whereby two EMMs sharing one or more grouping symbols are not “significantly” different. These may be generated by the CLD() function (or equivalently by multcomp::cld()). I really recommend against this kind of display, though, and decline to illustrate it. CLD displays promote visually the idea that two means that are “not significantly different” are to be judged as being equal; and that is a very wrong interpretation. In addition, they draw an artificial “bright line” between P values on either side of alpha, even ones that are very close.
In response to ever stronger recommendations from professional societies against the use of “significance” criteria and language, the CLD() function is now being deprecated and will be removed entirely from emmeans at a future date." (Source: https://cran.r-project.org/web/packages/emmeans/vignettes/comparisons.html)

This is confusing because it forces us to rethink the way we present our results. In my recent work I continued to use letters because they are handy to have a quick look at the data in figures. I even sarted to provide different groupings with different threshold values e.g. 0.05, 0.01, 0.001, and tried my best not to use the term "statistically significant", but rather referring to p-values being smaller or larger than a given threshold while taking care not to dismiss any difference in means just because it is below a certain threshold. We need to be careful how we discuss the results anyway, more so if we use the letter display. Good references on that are: doi: 10.1038/d41586-019-00857-9 and doi: https://doi.org/10.1080/00031305.2019.1583913

To add a module in Jamovi we would need to use the cld function from the multcomp package. In R I usually do it in 2 steps, first I save the emmeans result as an object, e.g.
Code: Select all
myemmeans <- emmeans(model,~Treatment)

and then I execute the cld function on that object, which I transform to a dataframe along the way, e.g.:
Code: Select all
mycld <- as.data.frame(multcomp::cld(myemmeans, sort = T, Letters = letters, reversed=F))

I then use this dataframe to plot my data

But I am not sure if I can manage that myself (would have to look into the module creation tutorial and commit the time; I never made a package in R, I am just a user). Moreover, I am even more unsure if we should dare to do it, since the grouping is slowly being abandoned and kind of banned by an increasing number of researchers.
Webolive
 
Posts: 1
Joined: Fri Sep 25, 2020 7:41 am


Return to Statistics