jamovi

Hi all,
I am looking for help. We received reviews for an article where we ran LMMs. The weakness of the study was a small sample, N=21 participants. We ran rather simple LMMs with a (continuous) rating of stimuli as a dependent variable, two continuous predictors as covariates, and stimuli and rater IDs as random effects in Jamovi. The models (based on Fs and associated p values) were not stat significant, though with R2C around 0.5, R2M ~ 0.02. Some covariates had estimates' slopes with 95%CI non-including zero and we discussed them in our results section.

The reviewer wants us to run a post hoc power analysis for all models.

The analysis is rather exploratory, we are aware of post hoc power pitfalls and we do not consider running it.

Is there a way how to run sensitivity analysis you could recommend or even provide help on how to perform? None of us is versed in R.

We also came across a preprint Murayama, Kou, Satoshi Usami, and Michiko Sakaki. “Summary-Statistics-Based Power Analysis: A New and Practical Method to Determine Sample Size for Mixed-Effects Modelling.” Preprint. Open Science Framework, May 11, 2020. https://doi.org/10.31219/osf.io/6cer3.

To our understanding, they suggest that simple LMMs power calculations based on simple tests (e.g. t-test) available in GPower are a good approximation. We tried running sensitive for linear regression, for alpha 0.05, beta 0.8, N=21, 2 predictors and got f2=0.545 which is equal to R2=0.35. If compared to the observed R2C, we should have enough sensitivity for R2C we observed. This makes no sense, as our models were non-significant.


What would you suggest we should do or reply?

Update:

so far I came across the simr R package that seems to do the job of listing observed power and allowing for simulations (extending sample with simulated data along or within chosen variable) and drawing power curves.