How do I view Responses from a Network built with the Neural Network Pattern Recognition App?

I trained a Network in the Pattern Recognition App (included in the Deep Learning Toolbox) with 14 features and 1 Response with 2 classes.

I then tested the Network on an additional Feature set (using the Test Icon on the ribbon). There doesn't appear to be any way to view the specific responses for the test set. There are statistics regarding the performance, generation of confusion matricies, etc., but I can't find any place to view the actual responses predicted.

I then exported the network to the work space, and tried to use the "predict" command on the test feature set, but only got an error message:

The exported network is "DiamondResults"

The Out-Of-Sample Features are in the Numeric Matrix array: "DiamondPredictOneHot_Matrix" - which is in the exact same format as the network training file ("DiamondTrainOneHot").

I tried using both a Matrix format and a Table format of the data. I tried using classify. I tried using sim. I tried using DiamondResuts.Network. Every permutation I could think of that was mentioned in the documentation, or forum posts I could find.

All of the Pattern Net documentation and scripts only appear to deal with training a network and getting performance statistics. I can't find any description of the process to actually view the predicted responses.

In a nutshell, this is what I'm looking to do:

Can anyone point me in the right direction?

Answer 1

[Harsha Vardhan]

Hi,

I understand that you are trying to view the predicted responses on additional test inputs/observations (feature set) from a network built with the Neural Network Pattern Recognition App.

I will provide solutions based on this document -https://www.mathworks.com/help/deeplearning/gs/pattern-recognition-with-a-shallow-neural-network.html.

To view the actual predicted responses by the trained network, I suggest 2 methods.

1st method using the ‘Export Model’ feature of the App:

The App requires that the number of observations in the predictor data must be greater than 10. So, for the predictors of training, I created a random sample of 11 observations, each with 14 features. And for responses of training, I created a random sample of 11 responses each with 2 classification classes.

Make sure that in the imported data, each row of the predictors matrix (here inputs matrix) corresponds to one feature and each column to one observation. So, in your case, the training set with 14 features and ‘n’ observations should be of the dimension 14 x ‘n’ matrix.

Similarly, the corresponding responses matrix (here targets matrix) must be of the dimension 2 x ‘n’ matrix. Here, each row corresponds to one classification class and each column corresponds to the desired/correct response of one input.

After the training is completed, click on ‘Export Model’ --> ‘Export to Workspace’. This will export a structured array containing the trained network and results to the workspace. I named that structured array as ‘DiamondResults’.

This variable ‘DiamondResults’ is of the type ‘struct’. It has the ‘Network’ variable which is the trained network. We can use this ‘Network’ to pass new observations and view the results of the trained network.

Let us create a random matrix (14 x 4) containing 4 more new observations as below.

moreObservations = randi([-21,34],14,4);

We can pass these new observations to the trained network as below and get the responses.

predictedResponses = DiamondResults.Network(moreObservations)

predictedResponses =

    0.1781    0.5932    0.8955    0.7840
    0.8219    0.4068    0.1045    0.2160

2nd method using the ‘Generate Code’ feature of the App:

Refer to this section of the above document - https://www.mathworks.com/help/deeplearning/gs/pattern-recognition-with-a-shallow-neural-network.html#ClassifyPatternsUsingTheNeuralNetPatternRecognitionAppExample-5

After the network is trained in the App, click on ‘Generate Code’ --> ‘Generate Simple Training Script’. This will create MATLAB code/script to reproduce the training steps of the App. Generally, creating MATLAB code from the App can be helpful to customize the training process. Save this generated MATLAB code as ‘train.m’. A few lines from the generated code is shown below.

% This script assumes these variables are defined:
%
%   inputs - input data.
%   targets - target data.

x = inputs;
t = targets;    

Before executing the generated code, we should load ‘inputs’ (14 x 11) and ‘targets’ (2 x 11) into the workspace and make these variables accessible to the generated MATLAB code.

Execute the MATLAB file (train.m) containing the generated code from App. This will reproduce the entire training steps done earlier through the UI of the App. And the variables created during the training get loaded into the workspace.

Among these variables loaded into the workspace, ‘net’ variable stores the trained network. As seen in the generated code, this ‘net’ variable can be used to test on new input observations and get the actual responses from the network.

% Train the Network
[net,tr] = train(net,x,t);

% Test the Network
y = net(x);

Like the 1st method, create a random matrix (14 x 4) containing 4 more observations. Pass this matrix to the ‘net’ variable to receive the responses of the trained network.

predictedResponses = net(moreObservations)

predictedResponses =

    0.8933    0.5895    0.8724    0.8906
    0.1067    0.4105    0.1276    0.1094

References:

1. Pattern Recognition with a Shallow Neural Network - MATLAB & Simulink: https://www.mathworks.com/help/deeplearning/gs/pattern-recognition-with-a-shallow-neural-network.html

[Lawrence]

Harsa,

Thank you for your detailed response. The steps you describe are exactly correct, and they are the same as the steps I was originally trying.

The error messages were a result of something I failed to mention in my post - but at the time didn't think it mattered: When I initially imported the data in the Pattern App, it was in Row Format, and I clicked the Row Format radio button. The data imported correctly, and I was able to generate a Network.

What I subsequently discovered is that this feature is just a courtesy to convert the input internally to Column Format, and that after the Network is exported it expects the new observations to be in Column format as well.

I discovered this purely by trial and error. When I went through the whole process - with all of my tables in Column format, everything worked. There were no errors.

There very well may be a command somewhere that instructs the predict function to accept Row vs. Column format, but for now everything I'm working with is in Column format.