Issues with WLSMV Estimator in SEM (Ordinal Data, Likert Scale)

[ASD]

First of all, thank you for the great work on Jamovi. it's a fantastic tool for applied statistical analysis.

I'm currently working on Structural Equation Modeling (SEM) using ordinal data (5-point Likert scale), and I've encountered some issues when using the WLSMV estimator for CFA, which is typically recommended for ordinal data.

Here are the specific problems I've run into:

• Some of the estimated coefficients come out negative, even though,based on theory and the measurement model they should not be.

• The path diagram fails to load; it gets stuck in a loading state and never completes.

Interestingly, when I run the same model using the Maximum Likelihood (ML) estimator, the estimation results appear reasonable (e.g., all coefficients are positive), and the path diagram displays correctly without issue.

My questions are:

• Is this a known issue with the WLSMV estimator in Jamovi?

• Are there any recommended workarounds or settings I should check?

Here is a comparison of the results using ML and WLSMV:



Any insights or suggestions would be greatly appreciated. Thank you in advance!

[mcfanda@gmail.com]

could you share an example file so we can check what is going on? thanks

[ASD]

Thank you very much for your response and attention to this issue.

I have encountered this problem across several different datasets. To illustrate, I’m attaching two Jamovi files containing SEM analyses where the issues occurred (WLSMV_Issues.omv and First_Order_CFA.omv)

In the first_order_CFA.omv file, in addition to the negative estimates and factor loadings, I’m also unable to display the path diagram. This issue tends to arise when the SEM analysis involves multiple latent variables, the diagram loading process never completes.

I hope these files help in identifying the cause of the problem.
Thank you again for your assistance.

[mcfanda@gmail.com]

Alright, finally I've uncovered the mystery. The issue is not related with SEMLj, but with the way the ordinal variables are encoded. In fact, when SEMLj gets a ordinal variables, it makes sure it is indeed ordinal and then passed to the estimator (lavaan::sem()) with the ordinal options (you can see that in the first table footnote).
Now, in your file (wlsmv) you have PASS2 like this

q1.png (19.92 KiB) Viewed 5965 times

and PASS3 like this

q2.png (19.69 KiB) Viewed 5965 times

So, when they are coerced to ordinal by the estimator you get opposite score and thus negative correlations.

This is how your two variables are internally defined

Code: Select all

 PASS1: Ord.factor w/ 5 levels "5"<"4"<"3"<"2"<..: 1 1 1 2 1 2 2 3 2 1 ...

Code: Select all

 PASS3: Ord.factor w/ 5 levels "1"<"2"<"3"<"4"<..: 3 4 4 3 4 3 3 2 3 4 ...

So their scores are opposite.

To fix it, you need to redefine the levels of your ordered variables in the dataset. In jamovi datasheet, select PAS1 , go to variable setup and reorder the levels

q5.png (53.75 KiB) Viewed 5963 times

At the end, they should look like this

q5.png (53.75 KiB) Viewed 5963 times

Rerun the analysis (a new analysis) and you should get the expected results

[mcfanda@gmail.com]

I am sorry, in the last message the last picture was not intended. Here it is
the correct order for PAS1 (and PAS2) should look like this

q6.png (19.95 KiB) Viewed 5959 times

the the results (after re-run) are

[ASD]

Hi, thank you so much for the explanation, it really helped me understand the problem!

If I may, I have a couple of follow-up questions:

• Would it be possible to have Jamovi automatically sort the levels from highest to lowest when changing a variable to ordinal? That small change would make things a lot easier, especially when working with Likert-scale data.

• After reordering the levels, the path diagram still doesn’t seem to load, it just keeps spinning (tested using the file First_Order_CFA.omv). The model contains three latent variables, each with several indicators. I’m wondering if this issue is related to the complexity of the model, or if something else might be going on?

Really appreciate all the support, thank you again for your time and help!

[jonathon]

> Would it be possible to have Jamovi automatically sort the levels from highest to lowest when changing a variable to ordinal?

wait, what?

[ASD]

Hi Jonathon,
Apologies if my earlier question was unclear, let me try to explain it a bit better.

When i open an Excel file in Jamovi, all the data is initially recognized as nominal. In this case, the levels are sorted automatically from the lowest to the highest value for all variables (e.g., 1 to 5) , as shown in the screenshot.


However, if i change the variable type to continuous (for any reason), the level information is removed as expected.


Now, when i switch the variable back from continuous to ordinal, the order of the levels can change from the original one. It turns out that this change in level order was the root cause of the issue I encountered in my SEM analysis using WLSMV (also shown in the screenshots).


Of course, this is a small issue and can be fixed manually by reordering the levels. But if Jamovi could apply a consistent default order (e.g., always from lowest to highest) when converting continuous data types back to ordinal or nominal, it would help avoid these kinds of problems in future analyses.

Thanks again for your time and all the amazing work you do on Jamovi!

[jonathon]

i will investigate!

jonathon

[mcfanda@gmail.com]

As for the path diagram, so far the module had some trouble in saving and showing large complex diagrams. We sorted out the issue, so in the next version (coming out Monday), these diagrams should be ok.