Comparing a mix of paired and unpaired data?

[Biochemist]

Hello,

I have four groups (datasets) that I want to compare statistically for a significant difference using an hypothesis test. The data are ordinal data, which is why I need to use a nonparametric test.

The problem that I now have is this: Two of the datasets/groups are paired (group 1 & 2), two are not (group 3 & 4).

If none of the groups were paired/matched, I would use the Kruskall-Wallis test. If all of the groups were matched, I would use the Friedman test. Both in combination with a post-hoc test for individual comparisons.

What can I do in this case? Should or can I just use the test for non-matched data because not all of the data are matched?

[Biochemist]

The individual comparisons that I am interested in are group 1 vs group 2 (paired), group 3 vs group 4 (paired), group 2 vs group 3 (unpaired), and group 3 vs group 4 (unpaired).

[reason180]

I have some trouble understanding the data set. Could you produce and share a small, mock data set as an example?

[Biochemist]

Sure, I can describe the data set and produce a small, mock data set as an example.

The experimental setup looks like this: A cell line, i.e. identical cells that can be propagated indefinitely, is cultivated alone and in combination with primary cells from specific donors. The focus is on the combination, i.e. the co-culture, of the two cell types and the effect of substance X.
So, the four groups are:
1) cell line + primary cells
2) cell line + primary cells + substance X
3) cell line alone
4) cell line alone + substance X

Groups 1) and 2) with the primary cells are paired because each data pair contains the same primary cells from the same donor. The groups with the cell line alone are unpaired because they are all the same cells in the cultures and hence creating data pairs is not possible.

The determined parameter (dependent variable) is a score from 1 to 10 that is the mean of the values from several scorers in order to make the score less subjective. So, the dependent variable is ordinal.

The research questions are:
Is there a statistically significant difference between 1) and 2) (= paired data), i.e. does substance X have an effect when the two cell types are co-cultured?
Is there a statistically significant difference between 1) and 3) (= unpaired data), i.e. do the primary cells affect the determined parameter when co-cultured with the cell line?
Is there a statistically significant difference between 3) and 4) (= unpaired data), i.e. does substance X have an effect on the cell line alone?

I think a two-way ANOVA would be the best choice here but a) the data are ordinal and b) there is this combination of paired and unpaired data. I don't think there is any universally accepted non-parametric equivalent to a two-way ANOVA. That is why I am not sure what the best way to analyze these data and answers these research questions would be.

[Biochemist]

Here is an example data set:

Group 1: Cell line + primary cells
7
7,5
6,75
7,25
7
7,25
7,75
6,5
7
7,5

Group 2: Cell line + primary cells + substance X
4
4,5
3,75
4
4
5
4,25
3,5
4
4,25

Group 3: Cell line alone
10
9,5
10
9,25
9
9,5
10
8,5
9,5
10

Group 4: Cell line + substance X
9,5
9,5
9,75
10
8,5
9
9,5
9,75
10
10

[Biochemist]

I read about a method called ART (Aligned Rank Transformation), which might be used, but so far I have never seen it applied in any research paper in my field. And the fact that it has not been implemented in Jamovi or any other more common statistical software gives me some doubt too.

[reason180]

Within the context of traditional, well-known statistical analyses, I don't think a "mix of paired and unpaired data" is analyzable in any streamlined way. You have three groups rather than four, and you need to do a set of t tests--most of which will be independent-samples t tests, and one of which will be be a paired samples t test.

123.png (37.77 KiB) Viewed 292 times

[Biochemist]

Thanks for your reply. So, multiple two-sample comparisons would be the correct choice here.

What about multiple comparison adjustments such as the Bonferroni correction? Should I apply them here?

And what about the quantified parameter being not strictly continuous (i.e. interval or ratio scale) because it is a score? Would t-tests still be OK or should I rather use their nonparametric equivalents?

[reason180]

The quantified parameter needs to be a 'close-enough' approximation to interval/ratio. Most analysts think that the kind of scores you have are close enough.

You can adjust for multiple comparisons. You'll need to use something like jamovi's Rj module (or R). See the example code, below:

Code: Select all

# This code computes the Holm-corrected p values for a vector of 
# uncorrected p values. (Options besides "holm" are: 
# "hochberg", "hommel", "bonferroni", "BH", "BY", and "fdr".)

p.adjust(c(.027, .141, .018, .053), "holm")  

# output: [1] 0.081 0.141 0.072 0.106