Improving ROC curves - Youden index

Discuss the jamovi platform, possible improvements, etc.
dr_Primus
Posts: 29
Joined: Wed Dec 20, 2017 12:07 pm

Improving ROC curves - Youden index

Post by dr_Primus »

First of all. Thanks for including ROC curves into jamovi... It is really handy and easy to use.
Here are some suggestions:

- display multiple curves on the same graph (i dabbled with pROC in R and made a script which I adjust as needed - if you would like me to I can upload it here)

-it would be nice to have the option to have curves in same colour but different shape (full, dotted, dashed, dot-dash-dot and so on)

-calculate the so called Youden index which will show the optimal cutoff value (best combination of specificity and sensitivity)

Thanks :)
drrajeshb
Posts: 2
Joined: Thu Sep 05, 2019 2:56 pm

Re: Improving ROC curves - Youden index

Post by drrajeshb »

Dr Primus
Good morning
I saw this and was extremely interested to learn more about multiple curves on a single ROC curve. Could you be kind enough to provide me with the script you developed for this.
I shall justifiably acknowledge your contribution in my scientific publication.
Besides, your idea of having the curves in different shapes is phenomenal.
I am a physician in Miami, USA.
ljfriese
Posts: 16
Joined: Wed Sep 19, 2018 9:07 pm

Re: Improving ROC curves - Youden index

Post by ljfriese »

Hi Dr. Primus,

Thanks! I'm the developer of this particular module. I would be happy to incorporate the features you're talking about. I've been a bit busy lately, but I'm planning to push an update in the next month or two. It would be great if you could provide a link to the script you mentioned.

Feel free to email me directly at lucasjfriesen@gmail.com

Cheers!
dr_Primus
Posts: 29
Joined: Wed Dec 20, 2017 12:07 pm

Re: Improving ROC curves - Youden index

Post by dr_Primus »

Hello guys
This is the code that I used:

To clarify:

Volumen = name of the dataset that I used
Responder = name of the response variable (0 or 1)
CVP, PPV, PCWP, SVV = predictor variables (continuous)

rocCVP <- roc (Volumen$Responder, Volumen$CVP, percent = TRUE)
rocPPV <- roc (Volumen$Responder, Volumen$PPV, percent = TRUE)
rocPCWP <- roc (Volumen$Responder, Volumen$PCWP, percent = TRUE)
rocSVV <- roc (Volumen$Responder, Volumen$SVV, percent = TRUE)
ciCVP <- ci(rocCVP, of = "se", sp = seq(0, 100, 5), boot.n=10000)
ciPPV <- ci(rocPPV, of = "se", sp = seq(0, 100, 5), boot.n=10000)
ciPCWP <- ci(rocPPV, of = "se", sp = seq(0, 100, 5), boot.n=10000)
ciSVV <- ci(rocPPV, of = "se", sp = seq(0, 100, 5), boot.n=10000)
rtest <- roc.test(rocCVP, rocPPV, boot.n=10000)
rtest1 <- roc.test(rocCVP, rocPCWP, boot.n=10000)
rtest2 <- roc.test(rocCVP, rocSVV, boot.n=10000)
rtest3 <- roc.test(rocPPV, rocPCWP, boot.n=10000)
pdf("Volume responder.pdf", width=10, height=10)
plot(rocCVP, col="#FF0000", lty=1, lwd=2, print.auc=TRUE, print.auc.y=56)
plot(rocPPV, col="#0000FF", add=TRUE, lty=2, lwd=2, print.auc=TRUE, print.auc.y=50)
plot(rocPCWP, col="#00FF00", add=TRUE, lty=3, lwd=2, print.auc=TRUE, print.auc.y=44)
plot(rocSVV, col="#FFAA00", add=TRUE, lty=4, lwd=2, print.auc=TRUE, print.auc.y=38)
text(18, 50, sprintf("CVP/PPV p = %.3f", rtest$p.value), adj=c(0,1))
text(18, 48, sprintf("CVP/PCWP p = %.3f", rtest1$p.value), adj=c(0,1))
text(18, 46, sprintf("CVP/SVV p = %.3f", rtest2$p.value), adj=c(0,1))
text(18, 44, sprintf("PPV/PCWP p = %.3f", rtest3$p.value), adj=c(0,1))
lines(c(20, 20), c(50, 41), lwd=1.5)
legend("bottomright", legend=c("CVP", "PPV", "PCWP", "SVV"), col=c("#FF0000", "#0000FF", "#00FF00", "#FFFF00"), lwd=2)
dev.off()


The resulting plot looks like this:
Slika zaslona 2019-09-07 u 14.36.05.png
Slika zaslona 2019-09-07 u 14.36.05.png (108 KiB) Viewed 12600 times
Last edited by dr_Primus on Sat Sep 07, 2019 12:46 pm, edited 1 time in total.
dr_Primus
Posts: 29
Joined: Wed Dec 20, 2017 12:07 pm

Re: Improving ROC curves - Youden index

Post by dr_Primus »

Of course... if you change col to ="#000000" lines will be black
dr_Primus
Posts: 29
Joined: Wed Dec 20, 2017 12:07 pm

Re: Improving ROC curves - Youden index

Post by dr_Primus »

And here is the website where I "steal" R code I need to get ROC curves I need


https://rpubs.com/Wangzf/pROC
ljfriese
Posts: 16
Joined: Wed Sep 19, 2018 9:07 pm

Re: Improving ROC curves - Youden index

Post by ljfriese »

Youden's index and combined ROC curves.
testROC.png
testROC.png (137.93 KiB) Viewed 12570 times
I have a few other tweaks to make before pushing this*, but would this meet your needs? Jamovi comes with the ability to switch to black/white by default.

*Adding DeLong's Test of difference between ROC curves and fixing up the UI
dr_Primus
Posts: 29
Joined: Wed Dec 20, 2017 12:07 pm

Re: Improving ROC curves - Youden index

Post by dr_Primus »

Wow! That looks great!
Thanks!
User avatar
MAgojam
Posts: 421
Joined: Thu Jun 08, 2017 2:33 pm
Location: Parma (Italy)

Re: Improving ROC curves - Youden index

Post by MAgojam »

dr_Primus wrote:This is the code that I used:
Hi, dr_Primus.
Thanks for sharing your script, very useful.

Cheers.
Maurizio
drrajeshb
Posts: 2
Joined: Thu Sep 05, 2019 2:56 pm

Re: Improving ROC curves - Youden index

Post by drrajeshb »

hey lucas

when should we expect this frature to come in the doenload able module in jamovi? i am anxiously waiting

thanks
Post Reply