EFA results different than SPSS

Discuss statistics related things

by KCo2er » Thu Sep 03, 2020 4:49 pm

Hi everyone!
I am conducting a scale development study for my dissertation and I have run into quite a barrier. When I run the analysis in Jamovi, 6 factors are extracted. When I run SPSS, 14 are extracted. I have to figure out which program is correct (or at least better for the purpose of scale development) before I can proceed. I found other people talking about this in a couple different places but I haven't found any answers or solutions. I have 159 participants, no missing data, 126 items, and I am running Principle Axis Factoring with Oblimin rotation and using eigenvalues > 1 as my cutoff criteria. I ran the analysis in R as well to compare, and while it is much closer to Jamovi compared to SPSS, it is still a little bit different.

My overall question is why this is so drastically different, and which is best to use, but I understand it is not as simple as asking that and getting an answer. Here are more questions to help me figure this out. I am sure I will have even more but this will be a good start.

1. I ran an EFA (PAF, eigenvalues > 1) with no rotation so I can compare it to SPSS. Why are Jamovi’s initial eigenvalues smaller than the SS loading eigenvalues? From what I understand, initial should be larger because they are using all the variance, and the other is using common variance.

2. Also, relating to eigenvalues, should the number of eigenvalues > 1 ever change in total from the initial values to the total after extraction? For example, in SPSS it gives me 14 initial eigenvalues > 1, but then under “extraction sums of squared loadings” there are 8 eigenvalues > 1, and the eigenvalues got smaller (which is alright from what I learned about common variance vs all variance mentioned above). Which is appropriate in SPSS at that point? Do I pay more attention to initial or after extraction?

3. Is there a way I can get the covariance/correlation matrix used in the EFA out of Jamovi so I can compare it to SPSS?

4. From what I have seen I can’t use a matrix from SPSS as input for Jamovi’s EFA. If you can confirm that it is not possible to do that it would be helpful.

5. Since Jamovi is built on top of R, I have an R-related question too. When running the analysis in R, I have to specify the number of factors (say, 4 through 16 since Jamovi is telling me 6 and SPSS is telling me 14) and then compare across models. Is that right? That is what all tutorials I have found say, and I am just puzzled why R wouldn’t have a method where it tells you how many factors it extracts. I am using fa from the psych package. (I specified 14 factors, ran it with oblique rotation, then learned about Kaiser normalization so I ran it instead with no rotation, and then applied the Kaiser normalization (since that was developed specifically because results differ from SPSS) and things are still not matching SPSS with regard to Eigenvalues. R is telling me 8 factors based on eigenvalues > 1.)

6. Is there a way to apply Kaiser normalization in Jamovi? (and do you think that could be the issue? I don’t think it is because I ran SPSS without Kaiser normalization and it barely changed the results at all. Definitely did not match Jamovi.

Let me know what else might be helpful to include.

Ultimately I would prefer to use Jamovi because I love everything this platform represents. Thanks in advance for your help in building my understanding.

Edit: Also, when applying oblimin rotation, my analysis took 74 iterations for the rotation. Does jamovi have a cap? (this doesn't explain why there are differences with no rotation though)
KCo2er
 
Posts: 1
Joined: Thu Sep 03, 2020 2:00 pm

by jonathon » Thu Sep 03, 2020 11:41 pm

hey,

yeah, it's often difficult to figure out why software produces different results. in our case, we end up using and trusting the upstream R packages. i'll attend ravi to this, as he might have some ideas, but you may be better off asking the R package authors.

however, sometimes you ask, and the response will be something like "this is what my package does. i don't know [or care] what spss does, so i don't know why they are different".

we've spent a lot of time researching analyses over the years, and if you find it difficult to find answers to these sorts of questions, it's because it is :/

jonathon
User avatar
jonathon
 
Posts: 1379
Joined: Fri Jan 27, 2017 10:04 am

by Ravi » Tue Sep 08, 2020 2:53 pm

Hi,

So it's hard to answer all your questions cause I'm not an expert on EFA. We use the psych R package for the actual underlying calculations (see https://github.com/jamovi/jmv/blob/master/R/pca.b.R#L52-L55), so for the technical details it's best to read the psych documentation. I can answer questions 3, 4, and 6 with a simple "no" though :).

About question 5, I think there's a way of estimating the number of factors in R as well, but this is how jamovi does it: https://github.com/jamovi/jmv/blob/master/R/pca.b.R#L411-L435.

Sorry I cannot be more helpful. If you can provide us with a clear example in R and a jamovi file containing the analysis you're doing it might be a little easier to see how the differences in results appear.

Cheers,
Ravi
User avatar
Ravi
 
Posts: 151
Joined: Sat Jan 28, 2017 11:18 am


Return to Statistics