Interpreting Mixed Model Analysis (Fixed and Simple Effects Table)

[burhanbaglar]

Hello Jamovi Community,

My name is Burhan, and I am currently pursuing a master's degree in cognitive psychology and conducting my first research project.

I decided to analyze my experiment using a mixed model with 2 (between-subjects variables: condition 0 or 1) * 4 (disruption 0, 1, 2, or 3) * (3: time 0 early, 1 mid, 2 late) within-subjects.

Briefly about the experiment: participants are assigned to one of two conditions (0 or 1) based on a specific pre-phase condition. They then see some stimuli on the screen, some of which are coded as disruption 0-1-2 or 3. While participants watch these stimuli, they provide key-press data (0 no press or 1 press). The duration from the start to the end of the experiment is divided into three parts: early, mid, and late (0-1-2). We aim to see how these keypresses differ between the control group and the experimental condition at the relevant disruption points and whether this effect changes over time.

When setting up my mixed model, I assign subjects as random effects and the response as the dependent variable. I then enter the nominal variables condition, disruption, and time as factors.

Here, I share the drive link for my model analysis jamovi output PDF: https://drive.google.com/file/d/1TG8De7 ... sp=sharing

As the omnibus test output shows, I have significant effects for condition, disruption, and the interaction effect of conditiondisruptiontime category.

Later, since I am interested in group response comparison at specific manipulation points (and its relative change according to time), I look at the simple effects table. But there, although I see the effects that are observable on my plot, I do not know if I need to use a p-value correction for them? Because all those simple effect tests are valuable to me, and if I would do corrections, my effects are lost...

General questions about my analysis:
1-) Is my model structure correct for what I intend to test?

2-) Is it appropriate to look at the simple effects to see the changes in the comparisons I want according to the interaction results in the omnibus test?

3-) Are the comparisons in the simple effect corrected for p-value?

4-) If the simple effect comparisons are not corrected and I need to correct them, my effects may be lost due to conservativeness. In this case, is it possible for me to get the comparisons I want from the fixed effects parameter estimates table?

5-) Can I not see the effect size of my model for comparisons? Or which effect size should I report?

Thank you in advance for your patience and answers. If my work gets published one day, I would like to thank you again as well.

[burhanbaglar]

Oh sorry my output PDF did not show the all tables Here is the link for the all page:

https://drive.google.com/file/d/1bMjk38 ... sp=sharing

[mcfanda@gmail.com]

HI
your model seems ok as long as you do not have repetitions (trials) within the conditions. If you have, you need to add to the model the random effects of the repeated measure factors (I guess you only have intercepts as random coefficients now).
As for "corrected p-value", is not clear what you want to correct for. I guess you mean correction for multiplicity (multiple tests). In general, if you find a significant interaction you can probe it with simple effects, so you see at which levels of the moderator a variable has an effect or not. Simple effects pvalues do not need correction (assuming you are talking about correction for multiple tests).
However, if you want to compare every means with any other mean (brute force comparisons), you need to adjust for multiple tests. In this case, you can use "posthoc tests", that are adjusted for multiplicity.

[burhanbaglar]

First of all, thank you very much for your interest and response. Seeing such support is very important for researchers like me who are at the beginning of their career path.

Regarding More Cluster Variables or Adding Random Slopes of Within Measures:
Yes, in the analysis file you reviewed, only subjects were specified as clusters and added to the model as intercepts. The ICC value of this was around 0.09. Besides this, I had not added any cluster variables or random slopes for any of my factors to the model.

In my experimental design, the condition shows whether the participant is in the control group (coded with 0) or the experimental group (coded with 1). Since this is a between-subjects variable, I am not adding it. Among my within-subjects variables, there is disruption. All four types of disruption (0, 1, 2, 3) are repeatedly shown to participants within the experiment on the same or different stimuli, and response data is collected. Another within-subject variable shows the time elapsed during the experiment (0 early, 1 mid, 2 late).

I am unsure which within-subject variable I should add as a random slope. As far as I know, adding a time measure as a random slope within the participant cluster is important for showing changes over time within participants. I can add this as a random slope if you think it is appropriate. I am not sure if I should also want the effects of disruption types to vary randomly within the participant cluster.

Multiple T-Test Usages and Corrections:
Yes, my other question was about the use of multiple tests and type 1 error correction due to multiple comparisons. In this sense, it was very important for me to report my simple effect results for approximately 12 tests without correction. Thank you very much for this.

[mcfanda@gmail.com]

In mixed models one should clearly distinguish between clustering variables and random coefficients. Your experiment clearly requires "participant" as clustering variable, so for that you are ok. As regards which random coefficients to add, you correctly added the intercepts. As regards the effect of time, it depends on the number of repetitions you have within each time. If there is only one repetition, you cannot add time random slope. If there are more, you can add it and check if the model converges. If no issue arises, keep the random slope, otherwise remove it.

One may wonder if you also need "stimuli" as clustering variable. This depends on the design. If every participant sees the same set of stimuli, and you have several stimuli, you may need to add "stimuli" as a clustering variable. See here for an example https://gamlj.github.io/mixed_example3.html

[burhanbaglar]

Thank you very much for taking the time to answer my questions.

While solving my current challenges, I also gained very valuable information.

Best regards
Burhan Baglar