Posterior probability for cross-validated fitcecoc model

I'm trying to get posterior probabilities, rather than prediction scores from a cross-validated fitcecoc model using the following

t = templateSVM('Standardize',true);

Modelall = fitcecoc(X,Y,'Learners',t,'ClassNames', unique(stimNum),'Options',opts,'FitPosterior', true);

[label,~,~,Posterior] = resubPredict(Modelall);

However, when cross-validation is turned on

Modelall = fitcecoc(X,Y,'Learners',t,'ClassNames', unique(stimNum),'Options',opts,'FitPosterior', true, 'CrossVal','on','KFold' ,10);

I get the following error:

"Incorrect number or types of inputs or outputs for function 'resubPredict'."

is there any way to get the prediction probability of a multiclass cross-validated classifier analogous to a binary SVM (using 'fitSVMPosterior')?

Thanks in advance

Answer 1

[Aditya]

Hi Noam,

When you have a cross-validated error-correcting output codes (ECOC) model created with “fitcecoc” and the 'CrossVal','on' option, you cannot use “resubPredict” to get the predictions or posterior probabilities. Instead, you should use “kfoldPredict” to obtain the cross-validated predictions and posterior probabilities. Here's how you can do it:

% Cross-validated ECOC model 
opts = statset('UseParallel', false); % Set 'UseParallel' to true if you want to use parallel computing 
Modelall = fitcecoc(X, Y, 'Learners', t, 'ClassNames', unique(Y), ... 
    'Options', opts, 'FitPosterior', true, 'CrossVal', 'on', 'KFold', 10); 

% Get the predicted labels and posterior probabilities for each fold 
[label,~,~,Posterior] = kfoldPredict(Modelall); 

The “kfoldPredict” function will return the cross-validated predicted labels in label and the posterior probabilities in Posterior. The syntax “label,~,~,Posterior“is used to ignore the second and third output arguments from “kfoldPredict” and only obtain the labels and posterior probabilities.

For additional examples and documentation, you can refer to the official MathWorks documentation page for “fitcecoc” function, which includes an example of using “kfoldPredict” with a cross-validated ECOC model.

• https://www.mathworks.com/help/stats/fitcecoc.html#:~:text=a%20ClassificationECOC%20model.-,Cross%2DValidation,-Cross%2Dvalidate%20the
• https://www.mathworks.com/help/stats/classreg.learning.partition.classificationpartitionedmodel.kfoldpredict.html

Hope this helps you to resolve the issue!

[Noam]

Awesome, thank you @Aditya