Feature request: Sensitivity analysis reported with tests

Everything related to development of the jamovi platform
Post Reply
Vit
Posts: 65
Joined: Fri Apr 19, 2019 11:51 am

Feature request: Sensitivity analysis reported with tests

Post by Vit »

As planning and reporting statisical power became standard practice, it is still rather cumbersome to use external tools. Recently I noticed StatsCloud.app reports power in results.

Would it be possible to add sensitivity analysis at least for base jamovi test (and moretest) modules? When a test is performed, jamovi does have all the necessary info to run a sensitive analysis alongside the main analysis. If we would get the results and, in addition, report on what sensitivity we had to observe what effects, it would highly impact results understanding and reporting.

The user could specify alpha and beta levels or see a plot, like in power.

Having this alongside t-test. ANOVAs, correlation and regressions would be godsent.
User avatar
jonathon
Posts: 2613
Joined: Fri Jan 27, 2017 10:04 am

Re: Feature request: Sensitivity analysis reported with test

Post by jonathon »

yeah, i think this would go into a moretests type module. i'd be happy to assist anyone wanting to develop it.

i think it's bad practice to report power in results, because power doesn't make sense after you've collected your data. but whatevs.

jonathon
Vit
Posts: 65
Joined: Fri Apr 19, 2019 11:51 am

Re: Feature request: Sensitivity analysis reported with test

Post by Vit »

Thank you, Jonathon, for your reply!

I agree that concerns about posthoc power are futile. But there are still cases where predicting effects for apriori power is complicated or not possible, and sensitive analysis is informative.

My idea was that the jamovi results section for, let's say, independent sample t-test could report that we have observed effects of XX with p-value YY. But based on the given sample size, we had sensitive to observing effects of (only) ZZZ with such and such Alpha and Beta levels... Thus, it should be known and considered if the test reports an effect smaller than what sensitivity shows.
User avatar
jonathon
Posts: 2613
Joined: Fri Jan 27, 2017 10:04 am

Re: Feature request: Sensitivity analysis reported with test

Post by jonathon »

oh yup. yeah, i'm happy to help someone with a module, irrespective of whether i agree with the approach or not :P

jonathon
Post Reply