I have been experiementing with training different machine learning methods for a classification problem. A typical dataset might have 40,000 samples representing four different classes, but highly imbalanced where maybe 98.5% of the samples are one class and each of the other three classes have about 0.5% representation. In such a domain, which is typical of the datasets I deal with in my domain, an AI object can achieve a high degree of accuracy by always returning the one class. This is the least important class in the overall scheme. The other three are of much greater interest. (For those familiar with the classic credit card dataset, this is like finding the few incidents of fraud among the ocean of proper transactions.)
When using something like fitcensemble, I can specify a 'cost' or 'prior' to modify the misclassification penalty or prior probabilities, respectively, to help deal with the imbalanced data. I would like to do something similar with patternnet, but don't see how to do this. The default performance function for patternnet is 'crossentropy'. When used on its own, crossentropy has an optional parameter perfWeights which looks like it might act very similar to cost in fitcensemble. However, there doesn't seem to be any way to access perfWeights from within patternnet. I've tried below, but it returns the message: "Warning: 'perfWeights' is not a legal parameter."
oNet = patternnet( nNeurons, 'trainrp' );
oNet.performFcn = 'crossentropy';
oNet.performParam.perfWeights = nWeights;
Scouting around patternnet documentation and the object itself, I don't see anywhere to specify or modify penalty weights for the various classes. Please tell me I've missed something super obvious, I'm pulling my hair out.
Thanks in advance!