How to get prediction scores from exported Classification Learner SVM model

Hello, I am using the Classification Learner tool to train a binary classifier, and I am wondering how to obtain scores for the predictions the trained classifier makes on new test data. I have been using the below command to test the classifiers:

yfit = C.predictFcn(T)

But I would like to know how to calculate/access the scores as well after exporting the model to the workspace from Classification Learner. The command

[labels,score] = predict(___)

gives the following error:

Error using predict (line 84)
Systems of struct class cannot be used with the "predict" command. Convert the system to an identified model
first, such as by using the "idss" command.

I am new to this, so if anyone could explain what I am doing incorrectly, and how to get scores for each prediction using an exported classifier, I would greatly appreciate the help.

Thanks!

Answer 1

[Swarooph]

Hello Ashley,

The predict function expects an input classifier of type such as ClassificationKNN or ClassificationTree etc. In this case you are passing a variable T which is if type struct. That is the reason for the error.

Type the following at the command prompt to see the contents of the struct:

>> T %Variable name of the exported classifier struct

Here one of the fields will be the classification object. For e.g. for my exported model referred by struct variable trainedClassifier, I see the following.

>> trainedClassifier

trainedClassifier = 

           predictFcn: [function_handle]
    RequiredVariables: {1x11 cell}
    ClassificationKNN: [1x1 ClassificationKNN]
                About: 'This struct is a tra…'
         HowToPredict: 'To make predictions …'

Note here that one of the fields is the actual classification object, ClassificationKNN. Note that, based on the type of trained classifier this might differ. To access this we can use the dot operator.

kNNMdl = trainedClassifier.ClassificationKNN;

Now use this as input to the predict function to get scores:

[labels,score] = predict(kNNMdl,ValData);

[Ashley Stephenson]

Got it working! Thanks so much for your help!

[Aditi Vedalankar]

if possible will u share the code for prediction of the test data

[SK]

Swaroop what does ValData correspond to? where did you get that from original question? ValData isnt Matlab funtion. PLease help. Thanks!

[xuan fan]

whats mean of this score matrix?

what if I wish to output that probability

Answer 2

[Miguel sanchez]

Hello, i have the same problem. I try mi trainingClassifier on a complete new data but i cant get the score values. 'ValData' correspond to what???

Answer 3

[Aditi Vedalankar]

Dear swaooph, I have trained the data using linear SVM. And from classification learner, I have exported the trained model to work space. I have written the following code for test data prediction as follows.. function Test_svm1() d = dir('*.xlsx'); fileName = d.name; %// Get the file name filename = 'DBModified_DTCWT_4Level_TST.xlsx'; sheet = 1; trainingData = xlsread(fileName,sheet); [trainedClassifier, validationAccuracy] = trainClassifier(trainingData);

filename = 'DBModified_DTCWT_4Level_TST.xlsx'; T = xlsread(filename); yfit=predict(trainedModel,T(:,trainedModel.predictFcn)); end But I get the following error Function 'subsindex' is not defined for values of class 'cell'. Error in trainClassifier (line 48) predictors = inputTable(:, predictorNames);

Error in Test_svm1 (line 7) [trainedClassifier, validationAccuracy] = trainClassifier(trainingData); Pl suggest me where i have done mistake Regards Aditi Vedalankar

Answer 4

[Felix Ajayi]

struct with fields:

predictFcn: @(x)ensemblePredictFcn(predictorExtractionFcn(x))

RegressionEnsemble: [1×1 classreg.learning.regr.RegressionBaggedEnsemble]

About: 'This struct is a trained model exported from Regression Learner R2018b.'

HowToPredict: 'To make predictions on a new predictor column matrix, X, use: ↵ yfit = c.predictFcn(X) ↵replacing 'c' with the name of the variable that is this struct, e.g. 'trainedModel'. ↵ ↵X must contain exactly 499 columns because this model was trained using 499 predictors. ↵X must contain only predictor columns in exactly the same order and format as your training ↵data. Do not include the response column or any columns you did not import into the app. ↵ ↵For more information, see How to predict using an exported model.'

yfit = C.predictFcn(T)

Please experts in the house how do how to get prediction scores from exported Classification Regression Learner SVM model. When I run the code the scores are not generated. I would really appreciate your swift reponse. Thank you.

Answer 5

[Felix Ajayi]

Experts in the house. Please I still need your help. I count on you guys.

Thank you in advance as I look forward for your reponses.

[SK]

I assume you are using Ensemble algos: either BAG or LS Boost. Plug in your trained model, either of those two where it says yourBagModel.mat

Load yourBagModel.mat;

yourBagModel.RegressionEnsemble

Names = yourBagModel.RegressionEnsemble.PredictorNames

x = cell(2,499) % this means starting row 2 (assuming in row 1 are your predictor lables), and 499 columns for number of predictors

x(1,:)= Names

x(2,:)= num2cell(predictorImportance(yourBagModel.RegressionEnsemble))

x = x;

Answer 6

[Savita Hanchinal]

clc

close all;

close all;

cd Database

DF=[]

for i=1:45

i

str1=int2str(i);

str2=strcat(str1,'.jpg');

nor=imread(str2);

%color

cmap=rgb2hsv(nor);

H=cmap(:,:,1);

S=cmap(:,:,2);

V=cmap(:,:,3);

Hmean=mean(mean(H));

Hst=std2(H);

Smean=mean(mean(S));

Sst=std2(S);

Vmean=mean(mean(V));

Vst=std2(V);

Hsk=sum(skewness(H));

Ssk=sum(skewness(S));

Vsk=sum(skewness(V));

Hmin=min(imhist(H));

Hmax=max(imhist(H));

Smin=min(imhist(S));

Smax=max(imhist(S));

Vmin=min(imhist(V));

Vmax=max(imhist(V));

%texture

I=rgb2gray(nor);

glcm=graycomatrix(I,'offset',[2 0;0 2]);

stats1=graycoprops(glcm,{'contrast','homogeneity'});

stats2=graycoprops(glcm,{'correlation','energy'});

t1=stats1.Contrast;

t2=stats1.Homogeneity;

t3=stats2.Correlation;

t4=stats2.Energy;

%Shape

FEAT=horzcat(1,[Hmean Hst Smean Sst Vmean Vst Hsk Vsk Ssk Hmin Hmax Smin Smax Vmin Vmax t1 t2 t3 t4]);

DF=[DF;FEAT];

end

cd ..

[fname,path]=uigetfile('.jpg','Provide currency for testing');

filename=strcat(path,fname);

nor=imread(filename);

%color

cmap=rgb2hsv(nor);

H=cmap(:,:,1);

S=cmap(:,:,2);

V=cmap(:,:,3);

Hmean=mean(mean(H));

Hst=std2(H);

Smean=mean(mean(S));

Sst=std2(S);

Vmean=mean(mean(V));

Vst=std2(V);

Hsk=sum(skewness(H));

Ssk=sum(skewness(S));

Vsk=sum(skewness(V));

Hmin=min(imhist(H));

Hmax=max(imhist(H));

Smin=min(imhist(S));

Smax=max(imhist(S));

Vmin=min(imhist(V));

Vmax=max(imhist(V));

%texture

I=rgb2gray(nor);

glcm=graycomatrix(I,'offset',[2 0;0 2]);

stats1=graycoprops(glcm,{'contrast','homogeneity'});

stats2=graycoprops(glcm,{'correlation','energy'});

t1=stats1.Contrast;

t2=stats1.Homogeneity;

t3=stats2.Correlation;

t4=stats2.Energy;

%Shape

QF=horzcat(1,[Hmean Hst Smean Sst Vmean Vst Hsk Vsk Ssk Hmin Hmax Smin Smax Vmin Vmax t1 t2 t3 t4]);

%Multi svm

Trainigset=[DF(1,:);DF(2,:);DF(3,:);DF(4,:);DF(5,:);DF(6,:);DF(7,:);DF(8,:);DF(9,:);DF(10,:);DF(11,:);DF(12,:);DF(13,:);DF(14,:);DF(15,:);DF(16,:);DF(17,:);DF(18,:);DF(19,:);DF(20,:);DF(21,:);DF(22,:);DF(23,:);DF(24,:);DF(25,:);DF(26,:);DF(27,:);DF(28,:);DF(29,:);DF(30,:);DF(31,:);DF(32,:);DF(33,:);DF(34,:);DF(35,:);DF(36,:);DF(37,:);DF(38,:);DF(39,:);DF(40,:);DF(41,:);DF(42,:);DF(43,:);DF(44,:);DF(45,:)];

GroupTrain={'1' '1' '1' '1' '1' '1' '1' '1' '1' '2' '2' '2' '2' '2' '2' '2' '2' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '4' '4' '4' '4' '4' '4' '4' '4' '4' '5' '5' '5' '5' '5' '5' '5' '5' '5' };

TestSet=QF;

SVMModels=cell(5,1);

y=GroupTrain

classes=unique(y);

rng(1);

for j=1:numel(classes)

indx=strcmp(y',classes(j));

SVMMOdels{j}=fitcsvm(DF,indx,'ClassNames',[false true],'Standardize',true,'KernelFunction','rbf','BoxConstraint',1);

end

xGrid=QF;

for j=1:numel(classes)

[~,score]=predict(SVMModels{j},xGrid)

Scores(:,j)=score(:,2);

end

[~,maxScore]=max(score,[],2);

result=maxScore;

figure.imshow(nor)

title('input')

if result==1

msgbox('10')

elseif result==2

msgbox('20')

elseif result==3

msgbox('50')

elseif result==4

msgbox('100')

elseif result==5

msgbox('500')

end

Iam getting error from thtis line please help me

for j=1:numel(classes)

[~,score]=predict(SVMModels{j},xGrid)

Scores(:,j)=score(:,2);