Reliability analysis

Discuss statistics related things

by vincep » Mon May 15, 2017 2:49 pm

Hi, I'm trying to get my head around the jamovi reliability analysis, and in particular using omega as a reliability measure.
My understanding of this is based on reading William Revelle:
http://www.personality-project.org/r/book/Chapter7.pdf
http://personality-project.org/r/psych/HowTo/factor.pdf

Here's my questions:
1. It seems like there are two omega values - omegaH and OmegaT. Which does jamovi report?

2. Is it possible to calculate omega reliabilities using polychoric correlations (ie the equivalent of setting poly=TRUE in the omega function in R). Is this necessary for working out reliabilities on data from Likert scales?

3. What is the equivalent R procedure (without using jmv) to produce a list of 'if item dropped' omega values?

4. The Revelle references explain omega calculations as involving an EFA plus some other steps. It seems like the default option for the psych omega function is to calculate an EFA with 3 factors. Is that also what jamovi does? If so, is that going to be ok for all datasets? Are there any situations where we would need to specifiy a different number of factors?

5. A slight bug: From the Reliability Analysis screen, if I clear all variables out of the 'items' selection box, this does not automatically clear items from the 'Reverse Scaled Items' box and causes an error.

Thanks for any clarity on these points.
vincep
 
Posts: 3
Joined: Sat May 06, 2017 11:37 am

by Ravi » Mon May 15, 2017 3:49 pm

Thanks for your questions. To calculate McDonald's omega we use the procedure described over here: http://journal.frontiersin.org/article/ ... 9/full#h12. In Appendix I you can see the R code used to calculate omega.

More specific answers:
1. jamovi reports omegaT

2. It's not possible yet to calculate omega using polychoric correlations in jamovi, I'll look into this a bit more. Generally, you can assume that Likert scale data behave fine for these analyses when there are 5 or more categories (See https://pdfs.semanticscholar.org/3833/3 ... 0524cb.pdf for a simulation study).

3. You can see the specific code we use over here: https://github.com/raviselker/Rjamovi/b ... .R#L43-L48. Basically, we calculate the omegaT without including the specific item.

4. We assume 1 factor. See the paper I linked to at the start of this post for more information.

5. Thanks for reporting this. Could you also report this on github (https://github.com/jamovi/jamovi/issues)? This way you'll get an update once this is fixed + we can see clearly which bugs we still need to fix.

Cheers
User avatar
Ravi
 
Posts: 141
Joined: Sat Jan 28, 2017 11:18 am

by vincep » Wed May 17, 2017 5:58 am

Thanks so much for the quick reply and those links - very helpful!
I've added the github bug report you requested.

As a general suggestion - it would be really helpful to record the underlying R syntax for each procedure somewhere in the documentation. It's great that novices can quickly and easily use jamovi without too much expertise, but it would also be very nice to let curious users understand what is happening behind the scenes.
vincep
 
Posts: 3
Joined: Sat May 06, 2017 11:37 am

by Ravi » Mon May 22, 2017 3:24 pm

For now, the curious reader can already go to our github and check out the precise procedure that we used (https://github.com/jamovi/jmv/tree/master/R contains all our R scripts for jmv. However, it would be nice to make it easy for people to see which packages are used for a particular analysis, also to give credit to the maintainers of the packages we are using.
User avatar
Ravi
 
Posts: 141
Joined: Sat Jan 28, 2017 11:18 am

by snfraser » Thu May 25, 2017 7:52 pm

Ravi,

You can use the userfriendlyscience package in R to calculate Likert Omega and Alpha reliability.

I have a description, links, and example here: http://shawnsstats.blogspot.ca/2016/11/ ... omega.html

-shawn
snfraser
 
Posts: 7
Joined: Thu May 25, 2017 7:48 pm

by Raymond89 » Tue Nov 14, 2017 12:20 pm

snfraser wrote:Ravi,

You can use the userfriendlyscience package in R to calculate Likert Omega and Alpha reliability.

I have a description, links, and example here: http://shawnsstats.blogspot.ca/2016/11/ ... omega.html

-shawn


Thanks for the link!
Raymond89
 
Posts: 1
Joined: Tue Nov 14, 2017 12:18 pm


Return to Statistics

cron