Did Jamovi invent a test that doesn't exist: Durbin-Conover --> is this a fake test? Does it really exist?

[abhtony]

After running a Friedman's test (non-parametric repeated measures ANOVA), there's the post-hoc test.

It's called the Durbin-Conover test.

From my best understanding:
- This is NOT the same as the "Durbin" test
- Based on the results shown in JASP for the "Conover" post-hoc comparison test, the Durbin-Conover test is NOT the same as the "Conover" test
- The Durbin-Conover test Does NOT appear in W.J. Conover's textbook Practical Nonparametric Statistics - it talks about the Durbin test, but does not talk about Durbin-Conover.
- Google scholar searches reveals that there are NO research article that uses this test - until about 2019.... which is around the time Jamovi came to be.
- The articles that feature this test, NEVER cites any article or textbook or handbook chapter, or anything at all that lets me track down where this test is from
- PMCMRplus documentation (https://cran.r-project.org/web/packages ... MRplus.pdf) states, "a Friedman-test for one-way ANOVA with repeated measures on ranks (CRBD) and Skillings-Mack test for unbalanced CRBD is provided with consequent all-pairs tests (Nemenyi test, Siegel test, Miller test, Conover test, Exact test)" --> There is no mention of Durbin-Conover test

Does this test really exist? Where is the source of this test?

I doubt this is something that's just invented by Jamovi - that makes 0 sense to me.

Perhaps my understanding is wrong? Is Conover the Durbin-Conover test? If so, why would JASP and JAMOVI yield different results for the post-hoc tests but the same for the Friedman's test?

Is the Durbin-Conover test real? Or is this something that was made up somewhere along the line and nobody actually checked where the source is from?

[abhtony]

Ok so to answer my own question: It MIGHT exist, but I think Jamovi is mislabelling things. I think Jamovi should just call this the Durbin's test, or at least provide a clear explanation why they use the term "Durbin-Conover".

The term "Durbin-Conover" might make sense because in Conover's textbook in 1999, Conover talks about Durbin's equations and mentions an alternative equation. Conover himself never explicitly uses the term "Durbin-Conover" test, nor does anyone else on this planet from what I can tell except for people who used Jamovi for non-parametric repeated measures ANOVA.


DETAILS ON REACHING THIS CONCLUSION:
Jamovi's anovarmnp.b.R file states:

Code: Select all

        
        if (self$options$get('pairs')) {

            table  <- self$results$get('comp')
            result <- PMCMR::posthoc.durbin.test(mat, p.adjust='none')

            n <- length(measureNames)-1
            rowNo <- 1
            for (j in 1:n) {
                for (k in j:n) {
                    table$setRow(rowNo=rowNo, list(
                        stat=result$statistic[k,j],
                        p=result$p.value[k,j]
                    ))
                    rowNo <- rowNo + 1
                }
            }
        }

See the part "result <- PMCMR::posthoc.durbin.test(mat, p.adjust='none')"?
Ok, so it uses PMCMR's function.

You can view the documentation for this function here:
https://cran.r-project.org/web/packages/PMCMR/PMCMR.pdf

But let's look at the R code from PMCMR and see what the function actually does.

Code: Select all

    
    ## Need to ensure consistent order.
    o <- order(blocks, groups)
    y <- y[o]
    groups <- groups[o]
    blocks <- blocks[o]

    p.adjust.method = match.arg(p.adjust.method)
    t <- nlevels(groups)
    b <- nlevels(blocks)
    r <- unique(table(groups))
    k <- unique(table(blocks))
    rij <- unlist(tapply(y, blocks, rank))
    Rj <- tapply(rij, groups, sum)
    ## Taken from NIST
    A <- sum(rij^2)
    C <- (b * k * (k + 1)^2) / 4
    D <- sum(Rj^2) - r * C
    T1 <- (t - 1) / (A - C) * D

    denom <- sqrt(((A - C) * 2 * r) / (b * k - b - t + 1) *
                      (1 - T1 / (b * (k -1))))
    df <- b * k - b - t + 1
   # Pairwise comparisons
    compare.stats <- function(i,j) {
        dif <- abs(Rj[i] - Rj[j])
        tval <- dif / denom
        return(tval)
    }
    PSTAT <- pairwise.table(compare.stats,levels(groups),
                            p.adjust.method="none" )

    compare.levels <- function(i,j) {
        dif <- abs(Rj[i] - Rj[j])
        tval <- dif / denom
        pval <- 2 * pt(q=abs(tval), df=df, lower.tail=FALSE)
        return(pval)
    }

The code's documentation references Conover's 1999 textbook. The textbook cites Durbin (1951) with the test statistic stated as:

T_1 = ((t - 1) * (Σ(j=1 to t) * (R_j)ˆ2 - r * C)) / A - C
where
A = Σ(i=1 to b) * Σ(j=1 to t) (R(X_ij))ˆ2
C = (b * k * (k + 1)ˆ2) / 4

...

Which matches what the PMCMR code does. This is indeed the same thing.

Which means:
* Jamovi uses PMCMR to calculate pairwise comparison for Friedman's test
* PMCMR uses Durbin (1951)'s equation, which is featured in Conover's 1999 textbook
* Therefore, Jamovi uses Durbin's equation.

Conover in his textbook, after introducing the Durbin's equation (T_1 = ((t - 1) * (Σ(j=1 to t) * (R_j)ˆ2 - r * C)) / A - C),
states:

An alternative procedure, equivalent to this one, is to use the ordinary analysis of variance procedure on the ranks and average ranks. This results in the following statistic T_2, which is merely a function of T_1. Current research indicates the approximate quantiles for T_2 are slightly more accurate than the approximate quantiles for T_1, making T_2 the preferred statistic.

And then gives us the T_2 equation:

T_2 = (T_1 / (t - 1)) / ((b (k - 1) - T_1) / (b * k - b - t + 1))

... So T_2 is the "preferred statistic", but Jamovi does NOT use this equation.

Im summary:
* T_1 is the Durbin's test
* T_2 would arguably be this mythical, supposed, Durbin-Conover's test (since Conover seems to have made a contribution to this alternative derived from Durbin's test). (But tbh not even since others seem to be using the alternative equation and just calling it Durbin just fine)
* Jamovi uses T_1 - the Durbin's test
* Jamovi does NOT use T_2 - the supposed Durbin-Conover test

So the question is:
Why does Jamovi use the term "Durbin-Conover" for their pairwise comparison for Friedman's test?

[MAgojam]

Hey @abhtony,
Maybe I can help you by starting from here:
As you can see we are on the jmv source line (anovarmnp.b.R) where there is the call to the PMCMR::posthoc.durbin.test() function, which is deprecated, because it was replaced with PMCMRplus::frdAllPairsConoverTest().

take a look here too (an old answer of mine) so I don't reload the screenshot:
viewtopic.php?p=4758#p4758

As you can see in the old PMCMR package there is posthoc.friedman.conover.test(), another deprecated function, which returns the same statistic as posthoc.durbin.test().

Let's see them together in this screenshot:

ScreenShot_20230918233319.jpeg (377.53 KiB) Viewed 1276 times

So enclosing "Durbin-Conover" in parentheses neither in the check-box label nor in the table title refers to a new test.
If the PMCMRplus::frdAllPairsConoverTest() function is used as it probably is for Jasp, there will be something like this last screenshot:

ScreenShot_20230919005422.jpeg (203.25 KiB) Viewed 1276 times

Cheers,
Maurizio