Page 1 of 3

Improving ROC curves - Youden index

Posted: Sat Mar 24, 2018 12:05 pm
by dr_Primus
First of all. Thanks for including ROC curves into jamovi... It is really handy and easy to use.
Here are some suggestions:

- display multiple curves on the same graph (i dabbled with pROC in R and made a script which I adjust as needed - if you would like me to I can upload it here)

-it would be nice to have the option to have curves in same colour but different shape (full, dotted, dashed, dot-dash-dot and so on)

-calculate the so called Youden index which will show the optimal cutoff value (best combination of specificity and sensitivity)

Thanks :)

Re: Improving ROC curves - Youden index

Posted: Thu Sep 05, 2019 3:01 pm
by drrajeshb
Dr Primus
Good morning
I saw this and was extremely interested to learn more about multiple curves on a single ROC curve. Could you be kind enough to provide me with the script you developed for this.
I shall justifiably acknowledge your contribution in my scientific publication.
Besides, your idea of having the curves in different shapes is phenomenal.
I am a physician in Miami, USA.

Re: Improving ROC curves - Youden index

Posted: Thu Sep 05, 2019 6:23 pm
by ljfriese
Hi Dr. Primus,

Thanks! I'm the developer of this particular module. I would be happy to incorporate the features you're talking about. I've been a bit busy lately, but I'm planning to push an update in the next month or two. It would be great if you could provide a link to the script you mentioned.

Feel free to email me directly at lucasjfriesen@gmail.com

Cheers!

Re: Improving ROC curves - Youden index

Posted: Sat Sep 07, 2019 12:37 pm
by dr_Primus
Hello guys
This is the code that I used:

To clarify:

Volumen = name of the dataset that I used
Responder = name of the response variable (0 or 1)
CVP, PPV, PCWP, SVV = predictor variables (continuous)

rocCVP <- roc (Volumen$Responder, Volumen$CVP, percent = TRUE)
rocPPV <- roc (Volumen$Responder, Volumen$PPV, percent = TRUE)
rocPCWP <- roc (Volumen$Responder, Volumen$PCWP, percent = TRUE)
rocSVV <- roc (Volumen$Responder, Volumen$SVV, percent = TRUE)
ciCVP <- ci(rocCVP, of = "se", sp = seq(0, 100, 5), boot.n=10000)
ciPPV <- ci(rocPPV, of = "se", sp = seq(0, 100, 5), boot.n=10000)
ciPCWP <- ci(rocPPV, of = "se", sp = seq(0, 100, 5), boot.n=10000)
ciSVV <- ci(rocPPV, of = "se", sp = seq(0, 100, 5), boot.n=10000)
rtest <- roc.test(rocCVP, rocPPV, boot.n=10000)
rtest1 <- roc.test(rocCVP, rocPCWP, boot.n=10000)
rtest2 <- roc.test(rocCVP, rocSVV, boot.n=10000)
rtest3 <- roc.test(rocPPV, rocPCWP, boot.n=10000)
pdf("Volume responder.pdf", width=10, height=10)
plot(rocCVP, col="#FF0000", lty=1, lwd=2, print.auc=TRUE, print.auc.y=56)
plot(rocPPV, col="#0000FF", add=TRUE, lty=2, lwd=2, print.auc=TRUE, print.auc.y=50)
plot(rocPCWP, col="#00FF00", add=TRUE, lty=3, lwd=2, print.auc=TRUE, print.auc.y=44)
plot(rocSVV, col="#FFAA00", add=TRUE, lty=4, lwd=2, print.auc=TRUE, print.auc.y=38)
text(18, 50, sprintf("CVP/PPV p = %.3f", rtest$p.value), adj=c(0,1))
text(18, 48, sprintf("CVP/PCWP p = %.3f", rtest1$p.value), adj=c(0,1))
text(18, 46, sprintf("CVP/SVV p = %.3f", rtest2$p.value), adj=c(0,1))
text(18, 44, sprintf("PPV/PCWP p = %.3f", rtest3$p.value), adj=c(0,1))
lines(c(20, 20), c(50, 41), lwd=1.5)
legend("bottomright", legend=c("CVP", "PPV", "PCWP", "SVV"), col=c("#FF0000", "#0000FF", "#00FF00", "#FFFF00"), lwd=2)
dev.off()


The resulting plot looks like this:
Slika zaslona 2019-09-07 u 14.36.05.png
Slika zaslona 2019-09-07 u 14.36.05.png (108 KiB) Viewed 12925 times

Re: Improving ROC curves - Youden index

Posted: Sat Sep 07, 2019 12:44 pm
by dr_Primus
Of course... if you change col to ="#000000" lines will be black

Re: Improving ROC curves - Youden index

Posted: Sat Sep 07, 2019 12:47 pm
by dr_Primus
And here is the website where I "steal" R code I need to get ROC curves I need


https://rpubs.com/Wangzf/pROC

Re: Improving ROC curves - Youden index

Posted: Sun Sep 08, 2019 10:37 pm
by ljfriese
Youden's index and combined ROC curves.
testROC.png
testROC.png (137.93 KiB) Viewed 12895 times
I have a few other tweaks to make before pushing this*, but would this meet your needs? Jamovi comes with the ability to switch to black/white by default.

*Adding DeLong's Test of difference between ROC curves and fixing up the UI

Re: Improving ROC curves - Youden index

Posted: Mon Sep 09, 2019 7:18 pm
by dr_Primus
Wow! That looks great!
Thanks!

Re: Improving ROC curves - Youden index

Posted: Sat Sep 14, 2019 7:31 am
by MAgojam
dr_Primus wrote:This is the code that I used:
Hi, dr_Primus.
Thanks for sharing your script, very useful.

Cheers.
Maurizio

Re: Improving ROC curves - Youden index

Posted: Thu Sep 19, 2019 12:26 am
by drrajeshb
hey lucas

when should we expect this frature to come in the doenload able module in jamovi? i am anxiously waiting

thanks