How to get more point [X,Y] in output of perfcurve function??

Hello,

I have a problem with using perfcurve, at the outputs [X, Y], applying the "perfcurve" I only get 3 points, I want to increase the output point numbers for a good tracking curve. how can I do??

Thanks!!

Answer 1

[the cyclist]

Short answer:

You do not have information for an ROC curve. You have one contingency table, specifying one point in ROC space. That point is FPR = 0.35, and TPR = 0.75.

Long answer:

First, and most importantly, you need to understand a couple things about an ROC curve (if you don't already). The curve is generally the output of a family of models, related to each other by some parameter -- typically a threshold score for which one will assign the "positive" label. Each point on that curve corresponds to a value of that parameter. For a good illustration of this, take a look at the figure in the "ROC space" section of the Wikipedia page for Receiver Operating Characteristic. (Each blue point is one model. Varying the threshold typically leads to a continuous curve connecting the points, as in the very first figure on that page.)

In MATLAB, the scores (the second input to perfcurve, representing probability from 0 to 1) give information that allow it to calculate the TPR and FPR to make the curve.

BUT ... you have not entered scores. You entered labels. I think that MATLAB is interpreting 1 as 100% (correctly) and -1 as 0%. [I am not certain of that, but it is consistent with the output.]

You have effectively told MATLAB the following: Your SVM model assigned some points 100% probability of being positive class, and some points a 0% probability of being positive class. Sort of coincidentally, this is actually what you intended to specify, even though you did not really understand the syntax (it seems).

So, how does MATLAB construct the ROC curve from these inputs? Well, you have two unique thresholds for determining positive class: 0% and 100%. (There are no other thresholds in your data.) You have 20 positive cases. If you assign positive class only to those points with 100% chance, then you classify 15 correctly (15/20 = 0.75 TPR). You have 20 negative cases. You incorrectly assign true to 7 of those cases (7/20 = 0.35 FPR).

The only other things you could be to assign positive class to all cases, or don't assign positive to any cases. These correspond to the lower-left and upper-right corners of ROC space, which are also on the MATLAB output. By default, MATLAB's plot command simply draws a line between those, but they are not calculated points in ROC space. The only accurately calculated points are

• (0,0)
• (0.35,0.75)
• (1,1)

which are the (X,Y) output of perfcurve.

There is no more information in your inputs. There are no "in between" points possible, because you only have one model.

Answer 2

[the cyclist]

The output vectors of perfcurve will have N+1 points when the input vectors have N points.

If you want more outputs, you need to somehow calculate the labeling for more inputs. Alternatively, you could presumably use some kind of interpolation on the outputs. See, for example, the interp1 function.

[abdelhamid bourouhou]

hi, fisrt, thank you for the answer, but my aim is to find something to modify in perfcurve function t get more point in output. the interpolation can not be useful in my case because I have juste three point in output and the plot (X,Y) present something like in the figure!

I think if I can get more point, the curve in plot will be more expressive.

[Walter Roberson]

How many input points do you have?

[the cyclist]

abdelhamid, I am not sure what you mean by "more expressive", but the only way you can get more information out of perfcurve is by putting more information into it.

If you do not have any more input data, with known labeling, then how do you expect MATLAB to calculate any more points along the perfcurve? It can't!

[abdelhamid bourouhou]

Mr.walter Roberson, I have 40 labels and 40 scores in input of percurve. Mr. the cyclist, I don't know where is the problem I think that I have suffisant labels and scores input to get more point in output than three!!

[the cyclist]

Can you post your input data (perhaps by attaching a *.MAT file) and your code here, so that we can take a look at the problem, rather than guessing?

[the cyclist]

One possibility is that you have more than three points on that plot, but they are perfectly aligned. You could try plotting like this

plot(X,Y,'.-')

using the '.-', to see where the points actually are.

[abdelhamid bourouhou]

Mr.the cyclist, I'm sure that the output are only three point, and you can verify it. I attached the *.mat file, where I have the inputs are test_label, and predicted_label, with the posclass is '1'. If we excute this few line command, we can obtain to vectors X,Y in output. if we see their values, we find only three values, and after ploting we have three points in the plan, and I can't work with only three points!!!

[X,Y] = perfcurve(test_label,predicted_label,'1');

plot(X,Y)

xlabel('False positive rate'); ylabel('True positive rate')

title(['ROC curve of (AUC = ' num2str(AUC) ' )']);

Answer 3

[the cyclist]

I'm not an expert at perfcurve, but I'm pretty sure your second input into perfcurve is now what MATLAB is expecting. For each test label (e.g. the first one, "1"), MATLAB is expecting the probability that your model (e.g. a logistic model) predicted a positive response.

MATLAB is interpreting your second input NOT as the predicted label being "1" or "-1". It is interpreting it as probabilities 100% and -100%. Frankly, I'm surprised it did not spin off an error.

How did you come up with your predicted responses? Did you have a probability model, and then simulate to get the actual 1 or -1?

[abdelhamid bourouhou]

yes, I have SVM classifier, and I compute the predicted_label with the model of svm classifier, and "1", "-1" or "0" is samething, because the classifier can recognise the label. And I dont think that "-1" create the problem!

[abdelhamid bourouhou]

Thanks, is not a solution but your remarks are very useful.