After running a Friedman's test (non-parametric repeated measures ANOVA), there's the post-hoc test.
It's called the Durbin-Conover test.
From my best understanding:
- This is NOT the same as the "Durbin" test
- Based on the results shown in JASP for the "Conover" post-hoc comparison test, the Durbin-Conover test is NOT the same as the "Conover" test
- The Durbin-Conover test Does NOT appear in W.J. Conover's textbook Practical Nonparametric Statistics - it talks about the Durbin test, but does not talk about Durbin-Conover.
- Google scholar searches reveals that there are NO research article that uses this test - until about 2019.... which is around the time Jamovi came to be.
- The articles that feature this test, NEVER cites any article or textbook or handbook chapter, or anything at all that lets me track down where this test is from
- PMCMRplus documentation (https://cran.r-project.org/web/packages ... MRplus.pdf) states, "a Friedman-test for one-way ANOVA with repeated measures on ranks (CRBD) and Skillings-Mack test for unbalanced CRBD is provided with consequent all-pairs tests (Nemenyi test, Siegel test, Miller test, Conover test, Exact test)" --> There is no mention of Durbin-Conover test
Does this test really exist? Where is the source of this test?
I doubt this is something that's just invented by Jamovi - that makes 0 sense to me.
Perhaps my understanding is wrong? Is Conover the Durbin-Conover test? If so, why would JASP and JAMOVI yield different results for the post-hoc tests but the same for the Friedman's test?
Is the Durbin-Conover test real? Or is this something that was made up somewhere along the line and nobody actually checked where the source is from?
Did Jamovi invent a test that doesn't exist: Durbin-Conover --> is this a fake test? Does it really exist?
Re: Did Jamovi invent a test that doesn't exist: Durbin-Conover --> is this a fake test? Does it really exist?
Ok so to answer my own question: It MIGHT exist, but I think Jamovi is mislabelling things. I think Jamovi should just call this the Durbin's test, or at least provide a clear explanation why they use the term "Durbin-Conover".
The term "Durbin-Conover" might make sense because in Conover's textbook in 1999, Conover talks about Durbin's equations and mentions an alternative equation. Conover himself never explicitly uses the term "Durbin-Conover" test, nor does anyone else on this planet from what I can tell except for people who used Jamovi for non-parametric repeated measures ANOVA.
DETAILS ON REACHING THIS CONCLUSION:
Jamovi's anovarmnp.b.R file states:
See the part "result <- PMCMR::posthoc.durbin.test(mat, p.adjust='none')"?
Ok, so it uses PMCMR's function.
You can view the documentation for this function here:
https://cran.r-project.org/web/packages/PMCMR/PMCMR.pdf
But let's look at the R code from PMCMR and see what the function actually does.
The code's documentation references Conover's 1999 textbook. The textbook cites Durbin (1951) with the test statistic stated as:
T_1 = ((t - 1) * (Σ(j=1 to t) * (R_j)ˆ2 - r * C)) / A - C
where
A = Σ(i=1 to b) * Σ(j=1 to t) (R(X_ij))ˆ2
C = (b * k * (k + 1)ˆ2) / 4
...
Which matches what the PMCMR code does. This is indeed the same thing.
Which means:
* Jamovi uses PMCMR to calculate pairwise comparison for Friedman's test
* PMCMR uses Durbin (1951)'s equation, which is featured in Conover's 1999 textbook
* Therefore, Jamovi uses Durbin's equation.
Conover in his textbook, after introducing the Durbin's equation (T_1 = ((t - 1) * (Σ(j=1 to t) * (R_j)ˆ2 - r * C)) / A - C),
states:
T_2 = (T_1 / (t - 1)) / ((b (k - 1) - T_1) / (b * k - b - t + 1))
... So T_2 is the "preferred statistic", but Jamovi does NOT use this equation.
Im summary:
* T_1 is the Durbin's test
* T_2 would arguably be this mythical, supposed, Durbin-Conover's test (since Conover seems to have made a contribution to this alternative derived from Durbin's test). (But tbh not even since others seem to be using the alternative equation and just calling it Durbin just fine)
* Jamovi uses T_1 - the Durbin's test
* Jamovi does NOT use T_2 - the supposed Durbin-Conover test
So the question is:
Why does Jamovi use the term "Durbin-Conover" for their pairwise comparison for Friedman's test?
The term "Durbin-Conover" might make sense because in Conover's textbook in 1999, Conover talks about Durbin's equations and mentions an alternative equation. Conover himself never explicitly uses the term "Durbin-Conover" test, nor does anyone else on this planet from what I can tell except for people who used Jamovi for non-parametric repeated measures ANOVA.
DETAILS ON REACHING THIS CONCLUSION:
Jamovi's anovarmnp.b.R file states:
Code: Select all
if (self$options$get('pairs')) {
table <- self$results$get('comp')
result <- PMCMR::posthoc.durbin.test(mat, p.adjust='none')
n <- length(measureNames)-1
rowNo <- 1
for (j in 1:n) {
for (k in j:n) {
table$setRow(rowNo=rowNo, list(
stat=result$statistic[k,j],
p=result$p.value[k,j]
))
rowNo <- rowNo + 1
}
}
}
Ok, so it uses PMCMR's function.
You can view the documentation for this function here:
https://cran.r-project.org/web/packages/PMCMR/PMCMR.pdf
But let's look at the R code from PMCMR and see what the function actually does.
Code: Select all
## Need to ensure consistent order.
o <- order(blocks, groups)
y <- y[o]
groups <- groups[o]
blocks <- blocks[o]
p.adjust.method = match.arg(p.adjust.method)
t <- nlevels(groups)
b <- nlevels(blocks)
r <- unique(table(groups))
k <- unique(table(blocks))
rij <- unlist(tapply(y, blocks, rank))
Rj <- tapply(rij, groups, sum)
## Taken from NIST
A <- sum(rij^2)
C <- (b * k * (k + 1)^2) / 4
D <- sum(Rj^2) - r * C
T1 <- (t - 1) / (A - C) * D
denom <- sqrt(((A - C) * 2 * r) / (b * k - b - t + 1) *
(1 - T1 / (b * (k -1))))
df <- b * k - b - t + 1
# Pairwise comparisons
compare.stats <- function(i,j) {
dif <- abs(Rj[i] - Rj[j])
tval <- dif / denom
return(tval)
}
PSTAT <- pairwise.table(compare.stats,levels(groups),
p.adjust.method="none" )
compare.levels <- function(i,j) {
dif <- abs(Rj[i] - Rj[j])
tval <- dif / denom
pval <- 2 * pt(q=abs(tval), df=df, lower.tail=FALSE)
return(pval)
}
T_1 = ((t - 1) * (Σ(j=1 to t) * (R_j)ˆ2 - r * C)) / A - C
where
A = Σ(i=1 to b) * Σ(j=1 to t) (R(X_ij))ˆ2
C = (b * k * (k + 1)ˆ2) / 4
...
Which matches what the PMCMR code does. This is indeed the same thing.
Which means:
* Jamovi uses PMCMR to calculate pairwise comparison for Friedman's test
* PMCMR uses Durbin (1951)'s equation, which is featured in Conover's 1999 textbook
* Therefore, Jamovi uses Durbin's equation.
Conover in his textbook, after introducing the Durbin's equation (T_1 = ((t - 1) * (Σ(j=1 to t) * (R_j)ˆ2 - r * C)) / A - C),
states:
And then gives us the T_2 equation:An alternative procedure, equivalent to this one, is to use the ordinary analysis of variance procedure on the ranks and average ranks. This results in the following statistic T_2, which is merely a function of T_1. Current research indicates the approximate quantiles for T_2 are slightly more accurate than the approximate quantiles for T_1, making T_2 the preferred statistic.
T_2 = (T_1 / (t - 1)) / ((b (k - 1) - T_1) / (b * k - b - t + 1))
... So T_2 is the "preferred statistic", but Jamovi does NOT use this equation.
Im summary:
* T_1 is the Durbin's test
* T_2 would arguably be this mythical, supposed, Durbin-Conover's test (since Conover seems to have made a contribution to this alternative derived from Durbin's test). (But tbh not even since others seem to be using the alternative equation and just calling it Durbin just fine)
* Jamovi uses T_1 - the Durbin's test
* Jamovi does NOT use T_2 - the supposed Durbin-Conover test
So the question is:
Why does Jamovi use the term "Durbin-Conover" for their pairwise comparison for Friedman's test?
Re: Did Jamovi invent a test that doesn't exist: Durbin-Conover --> is this a fake test? Does it really exist?
Hey @abhtony,
Maybe I can help you by starting from here:
As you can see we are on the jmv source line (anovarmnp.b.R) where there is the call to the PMCMR::posthoc.durbin.test() function, which is deprecated, because it was replaced with PMCMRplus::frdAllPairsConoverTest().
take a look here too (an old answer of mine) so I don't reload the screenshot:
viewtopic.php?p=4758#p4758
As you can see in the old PMCMR package there is posthoc.friedman.conover.test(), another deprecated function, which returns the same statistic as posthoc.durbin.test().
Let's see them together in this screenshot: So enclosing "Durbin-Conover" in parentheses neither in the check-box label nor in the table title refers to a new test.
If the PMCMRplus::frdAllPairsConoverTest() function is used as it probably is for Jasp, there will be something like this last screenshot: Cheers,
Maurizio
Maybe I can help you by starting from here:
As you can see we are on the jmv source line (anovarmnp.b.R) where there is the call to the PMCMR::posthoc.durbin.test() function, which is deprecated, because it was replaced with PMCMRplus::frdAllPairsConoverTest().
take a look here too (an old answer of mine) so I don't reload the screenshot:
viewtopic.php?p=4758#p4758
As you can see in the old PMCMR package there is posthoc.friedman.conover.test(), another deprecated function, which returns the same statistic as posthoc.durbin.test().
Let's see them together in this screenshot: So enclosing "Durbin-Conover" in parentheses neither in the check-box label nor in the table title refers to a new test.
If the PMCMRplus::frdAllPairsConoverTest() function is used as it probably is for Jasp, there will be something like this last screenshot: Cheers,
Maurizio