multiple ROC curves, logistic regression

[anagrammarian]

Hi,

Is there a way to generate multiple ROC curves on the same plot in the logistic regression module? For e.g. I have two different methods generating volume predictors for Alzheimer's disease and want to characterize each method using AUC for AD prediction as a metric. I can generate an ROC for each method separately (the data is in separate sheets but I can merge them). Alternatively, is there a way to save the data used to generate the curve as pairs so I can use another program to plot multiple curves?

And a related question. Does the logistic regression I used to generate the ROC use an internal test train split or do I have to do that manually? I did not see any documentation on how that is done.

Thanks,

manoj

[seol]

Hi

This is a R programming for multiple ROC curves.
https://github.com/cardiomoon/multipleROC

Hope this helps you

Best Regards
Seol

[seol]

Oh

Have you try to run 'psychoPDA' module for Multiple ROC ?

I think it's possible to draw Multiple ROC with pyschoPDA module.

Best Regards
Seol

[anagrammarian]

Thanks Seol. I have but I am trying to use the results from jamovi's logistic regression and not able to get the data out to plot. We coded it in R.

[seol]

Hi anagrammarian

I installed Multiple ROC within snowCluster module and pushed it to github yesterday.
It will be available today in jamovi library. I will let you know it's available.

Best Regards
Seol

[seol]

Hi

You can update snowCluster module in jamovi library. You should check the latest version of jamovi(current version).

Best Regards
Seol

[anagrammarian]

This is great. I will give it a spin and let you know. Thanks much !

[anagrammarian]

I just tried it (had to redownload jamovi) and seems to work very well. Thank you for adding this so quickly. Some followup qs/comments-
a. The text legends like model, auc overlap on the curve. Maybe it should be printed as text instead of on the image
b. I tried to predict y based on N xs. The multiple ROC curves options plots y vs x1, y vs x2.. and so on. What I had in mind when I said multiple ROIs was multiple *different* models like y based on 2 xs or 3xs or Nxs. This is how the original https://github.com/cardiomoon/multipleROC envisions also. I guess this will be much more complex in jamovi to implement.
c. The last q (and I think I know the answer but wanted to check) is does this do a test train split? I think not but that might be useful.

[seol]

Hi anagrammarian
Thanks for your questions.

a, Try resizing the picture by entering a number in size option.
b. To do it the way you suggest requires some tricks. In this case, it might be more accurate to use the stepwise method in logistic regression rather than ROC analysis.
c. You might be try to run it snowCluster >Machine Learning in jamovi

Best Regards
Seol

[anagrammarian]

Thanks.

a. I can do that but in general some people may not want to show it in the curve.
b. I am mainly doing it to compare 3 or 4 different methods (which generate different xs) as opposed to different number of xs (which might require stepwise analysis as you suggest. So imagine for the iris case we had 3 different ways of generating petal and sepal lengths and widths. I want to see which measurement method is more accurate (using AUC of iris prediction as a surrogate). Here the xs (i.e. the 4 variables) are the same and y is species. It will be nice to show all 3 ROC curves. But I think jamovi is not the tool for that. I can do this in R (i just need to dust off my manual, haven't done it in a while).
c. I need to generate ROC curves for the test train splits which I have already done so thanks.