Confidence intervals for two-sample t-test

Discuss the jamovi platform, possible improvements, etc.
Post Reply
PeteD
Posts: 25
Joined: Tue Jun 04, 2019 11:46 pm

Confidence intervals for two-sample t-test

Post by PeteD »

Hi

We have been using jamovi as a trial at my university (first years stats, compulsory course; 400-ish students; all enrolled in health and science) alongside SPSS (hoping to ditch SPSS in the future).

My students do a free-choice project (with some boundaries) and one group measured ruler bending deflections for wooden and plastic rulers.

As part of their analysis, they were seeking a CI for the difference between the means of the two groups. The CI confused them: The CI given in jamovi is from -Inf to -60.1.

The reason is their choice of alternative hypothesis: as shown on their output (image attached), they used Ha Wooden < Plastic.

Changing the alternative hypothesis to the two-tailed alternative gives CI as -69.2 to -59.2 (so both limits have changed).

The same data in SPSS give -69.2 to -59.2, so the SPSS output agrees with jamovi's when the two-tailed alternative is selected. (Of course, SPSS always does two-tailed tests by default, at least in my experience.)

Now I see where jamovi is going here. But to get a CI for the difference between the means of the two groups, one has to select a two-tailed test, whether a two-tailed test is really what one wants to test for (ignoring all the discussion about deciding on one- or two-tailed tests and their merits...).

So my students want to obtain two things from jamovi:
  1. A CI for the difference between the means, when they need to set the alternative hypotheses as two-tailed
  2. A one-tailed test to compare the means, when they need to select a one-tailed alternative hypothesis.
To me this is very confusing for students, especially at this level.

Is this intentional? It seems to me to be a poor choice (IMHO)... Thoughts? Opinions?

P.
Attachments
Screen Shot 2019-09-26 at 10.18.17 am.png
Screen Shot 2019-09-26 at 10.18.17 am.png (69.64 KiB) Viewed 4494 times
User avatar
jonathon
Posts: 2613
Joined: Fri Jan 27, 2017 10:04 am

Re: Confidence intervals for two-sample t-test

Post by jonathon »

hi,

> The CI confused them: The CI given in jamovi is from -Inf to -60.1

is the issue here that you don't think the CI is correct? that is how CI's work with one-tailed tests.

kind regards

jonathon
PeteD
Posts: 25
Joined: Tue Jun 04, 2019 11:46 pm

Re: Confidence intervals for two-sample t-test

Post by PeteD »

No, the issue is not that it is wrong; as I said (but could have said more clearly), "Now I see where jamovi is going here", and I know what is happening. I know jamovi isn't wrong, per se.

But if my students, who do not know all this, just want a CI for the difference between the means (say, as a description of that difference), they need to select a two-tailed alternative hypothesis... for the hypothesis test (even if they don't want to do a test!).

If they then also want a one-tailed test, they need to change to a one-tailed test.

SPSS circumvents the issue by issuing just two-tailed tests and CIs. (I don't even know if one can get one-tailed CIs in SPSS for this.)

So it seems jamovi is confusing my students: I don't think any basic, first year stats class would be talking about one-tailed CIs in these situations; certainly none of the textbooks I have ever seen (and I have seen a lot...) do so.

The one-tailed alternative hypothesis sounds like it should apply to the test, not the CI. Maybe the CI could also have an option for a two-sided CI, or one-sided CI?

It just is confusing (not wrong): Select a one-tailed test, which changes the CI to something never (hardly ever?) discussed in any intro stats textbooks (a similar target for the software) and which they may not want anyway. (Yes, we could discuss the rights and wrongs of that approach... but off the topic).

P.
User avatar
jonathon
Posts: 2613
Joined: Fri Jan 27, 2017 10:04 am

Re: Confidence intervals for two-sample t-test

Post by jonathon »

i think this might be a "damned if we do, damned if we don't" situation.

i think the CI needs to match the p-value.

i know that a lot of people/teachers teach the "if the CI contains zero, then the result will not be significant" thing, so disconnecting the CI from the p-value would confuse these people.

i'd suggest teaching it as follows:

"so when we use a one-tailed test of a > b, we no longer care about the situation in which a < b ... if a < b we'll get a very large p-value. in the same way, with confidence intervals, can you see that we no longer care about the lower bound of b - a? and so that's why it's gone to negative infinity. the plus side to all of this is that when a > b, our p-value will be more 'accurate', as will the upper bound of our confidence interval."

jonathon
PeteD
Posts: 25
Joined: Tue Jun 04, 2019 11:46 pm

Re: Confidence intervals for two-sample t-test

Post by PeteD »

Fair call. I can't disagree with your philosophy...

But I still challenge you to find an intro stats book where this is discussed in this way, so that jamovi could be useful for teaching.

Damned both ways indeed!

P.
User avatar
jonathon
Posts: 2613
Joined: Fri Jan 27, 2017 10:04 am

Re: Confidence intervals for two-sample t-test

Post by jonathon »

what if i bundle this change into the addon i described in the other thread?

i'll call it "just for pete"

i'm sufficiently amused by this idea that i'll probably do it.

jonathon
PeteD
Posts: 25
Joined: Tue Jun 04, 2019 11:46 pm

Re: Confidence intervals for two-sample t-test

Post by PeteD »

That's amusing indeed, and super ;->

But how about this idea, even more generally:

One issue with teaching stats is the inconsistency of language and notation, and intro stats textbooks have different language and notation too.

So, for some common inconsistent language and notation, have a dialog, or options box, or package where one select the notation/language used, so that the jamovi language/notation is consistent with the textbook one is using?

As examples (perhaps poor as I am taking these from the top of my greying head): in regression the x variable has heaps of names: predictor, explanatory, independent variable to name but three. Imagine a dialog, option, package where the name could be set so it was consistent throughout, and with the textbook, to avoid confusing students? Likewise, "stemplot" vs "stem-and-leaf plot' (admittedly less confusing), "least-squares regression line" vs "line of best fit", etc. And, of course, "Mean difference" vs "Difference: Means" :->

Two cents...

P.
Post Reply