jamovi

Hi

There is a large discrepancy between the correlations I get using factor -> reliability -> 'correlations heatmap' and the ones I get using the standard menu (regression -> correlation matrix). See screenshots below. I have also attached the data file.

Is this a bug, or is there something I don't understand about the correlations heatmap in the reliability menu? My assumption is that both methods should produce the same simple item-item correlations.

I am using Jamovi v. 2.6.19.

This is the heatmap: Notice, for instance, the several negative correlations for variable i10.
For example the correlation btw. i10 and 20 is reported as -0.64

When I calculate that same correlation pair using regression -> correlation matrix I get a positive pearson's r of 0.48, which seems reasonable given the scatter plot: This is just an example. The discrepancies are all over the place, check out the full correlation matrix from the regression menu in my attached .omv file.

Best regards
Anders

Note: I downgraded to 2.3.38 with the same result.

Update:
It seems that there is also something wrong with the alpha calculations?

Notice for instance 'alpha when item dropped' for i04 in this screenshot: it says 0.64, while alpha for the whole scale is 0.74
However, if I remove i04 from the scale, alpha rises to 0.86

Hi!

I found the issue: it’s due to how each analysis handles missing values. The correlation matrix uses pairwise deletion, while the reliability analysis applies listwise deletion. Since your dataset has a lot of missing data, listwise deletion removes far more cases than pairwise deletion. When I apply a filter in jamovi to emulate listwise deletion, the results of the correlation heatmap and matrix align (see attached file).

This also explains the "alpha when item dropped" issue. Here, alpha is calculated using the same dataset (with all missing data removed listwise) but without the item. However, dropping an item can alter the dataset, as cases excluded before may now be included, which affects the alpha calculation.

I think it makes sense to use listwise deletion for reliability analysis, as alpha requires complete cases. That said, it might be helpful to inform users about missing value handling and the number of cases omitted. I’ll consider how we can improve this.

Thanks for your questions!

Cheers,
Ravi

Thanks a lot, Ravi, for looking into this. Glad that you could pinpointed the issue.
Yes, a warning text in the output about omitted cases would be very helpful.
Thanks again