Jamovi is treating my data in a very weird way

[BlackFox]

Hi !
Today I was doing a paired sample t-test, and the results were... strange. For a study, participants completed a questionnaire at two times : pre-test and post-test. When I look at my data, I see participants improved their BMI, and decreased their level of stress. There is no mistake here, I triple checked. But the t-test gives a negative effect size when data increased in post-test, and a positive effect size when data dicrease post-test...

Can you help me understand these "reversed" results ? Thank you !

Here is a screenshot of Jamovi, and the translation in english

Capture d’écran 2025-03-14 172514.png (109.68 KiB) Viewed 46963 times

Capture d’écran 2025-03-14 172258.png (19.36 KiB) Viewed 46963 times

[MAgojam]

Hey @BlackFox ,
The "error" is in the order of the variables.

The t-test calculates Post_test - Pre_test.

Put Post_test on the left and Pre_test on the right.
If you switch them, the sign of the effect reverses.

Cheers,
Maurizio

[BlackFox]

Thanks for your answer !
If I understand correctly, I always have to assume Jamovi is asking the first data (on the left in the box) to be the post-test, and the second data (on the right in the box) to be the pre-test ? It seems very counterintuitive, do you know why it'works that way ?

[MAgojam]

It seems very counterintuitive, do you know why it'works that way ?

It is related to how the t.test() function of the R stats package works, used in the jmv analysis module of jamovi.
You can intercept it at this link: https://github.com/jamovi/jmv/blob/mast ... ps.b.R#L76

Why Should the Post-Test Group Be Listed First in a Paired t-Test?

In a paired t-test (or dependent samples t-test), the order of the groups affects the sign of the difference but not the statistical significance of the test. However, it is common practice to list the post-test group first and then the pre-test group for the following reasons:

1. Consistency in Calculating Differences
- The paired t-test compares two measurements on the same subjects by computing the difference between the two conditions:
D = X_post - X_pre
- If the post-test is listed first, a positive mean difference (D̄ > 0) indicates an improvement, while a negative mean difference (D̄ < 0) indicates a decline.
- This makes interpretation more intuitive.

2. Expectation of Improvement or Change
- In many scenarios (e.g., clinical trials, educational experiments), we aim to assess whether a treatment or intervention had an effect.
- If the treatment is effective, the post-test value is expected to be higher than the pre-test value (D > 0).
- Listing the post-test group first ensures that the sign of the differences aligns with this natural expectation.

3. Uniformity with Other Statistical Analyses
- In regression and other statistical tests, change is often expressed as "after - before" to assess the effect of an intervention.
- Following this convention reduces the risk of confusion and facilitates comparisons across studies.

Conclusion
The order of the groups does not affect the significance of the test, but listing the post-test first makes the interpretation of the results more intuitive.

Cheers,
Maurizio
https://www.jamovi.org/about.html

[reason180]

I want to point out that paired-samples t tests are employed not only in pre-post situations, but also when the distinction between the two conditions isn't about time (e.g., left-hand performance versus right-hand performance). In addition, for some common dependent variables, good performance equates to a small number (reaction-time, for existence), rather than a large number. So overall I don't think there's a most-natural or least-confusing way to have jamovi to the subtraction.

[BlackFox]

Ah, the more you know! Now I understand much better, thank you both for your explanations, it's much clearer for me!