can I see testing accuracy and loss graph in Neural network, like training graph?

In classify() function can i set parameters to plot graph for testing accuracy and loss?

also what if I have not provided any validation data ie i have done two partions only training and test. Is there any problem?

Answer 1

[Raunak Gupta]

Hi Krishna,

I assume by graph of the testing accuracy and loss; you mean epoch wise plot of the parameters for testing data. I think if you want to get the values for the testing data it is required to pass the data while training itself so that prediction can be made at every epoch and accordingly mini-batch accuracy and loss can be updated.

So essentially you need to pass testing data as validation data for calculating the accuracy and loss epoch wise.

For second question, it is completely fine to skip the validation data.

Hope this clarifies.

[krishna Chauhan]

Thank You so much Raunak,

plz tell me how to specify testing data in classifiy function.

The code I am using and the training progress graph is attached. How to observed the testing progress here?

rng('default')

trainedNet = trainNetwork(XTrain,YTrain,layers,options);

[Ypredicted,~] = classify(trainedNet,XTest,'ExecutionEnvironment','GPU');

cnnAccuracy = sum(Ypredicted==YTest)/numel(YTest)*100;

[Raunak Gupta]

Hi,

The testing data you are passing in classify function is mentioned correctly. For passing testing data as validation data to see its performance during training you can modify the options variable you passed to trainNetwork function. Let's say the options looks something like this

options = trainingOptions('sgdm', ...
    'MaxEpochs',15, ...
    'Shuffle','every-epoch', ...
    'Plots','training-progress', ...
    'Verbose',false);

So you can add the Name-Value pair of Validation data as follows:

options = trainingOptions('sgdm', ...
    'MaxEpochs',15, ...
    'Shuffle','every-epoch', ...
    'Plots','training-progress', ...
    'Verbose',false, ...
    'ValidationData',{XTest,YTest});

Just modify the options like this and this should enable you to see the performance of test data while training.

Hope this clarifies!

[krishna Chauhan]

Hi thanks its working , So now the validation accuracy is my final accuracy , if I passed test data in training options, right?

But could you plz tell me why its giving diffrent validation accuracy every time and how can I observe the confusion matrix now?

[Raunak Gupta]

Hi,

The different validation accuracy is due to the network being intialized with random weights everytime you start the training from scratch. If you want to get same result every time you can fix the seed of random number generation using

rng default

Include this command on the top of the training code and this will fix the seed and thus you will have consistency between different training runs.

For plotting confusion matrix you can use plotconfusion. You may find the example given the documentation helpful for implementing the same.

[krishna Chauhan]

Did that too sir as below

rng('default')

trainedNet = trainNetwork(XTrain,YTrain,layers,options);

But still its giving different accuracy everytime.

[Raunak Gupta]

Hi,

What is the typical difference you are seeing between different runs? If the difference is small, it may be due to the shuffling of the training data that happens between every epoch or at the very start of the training.

You may set the 'Shuffle' Name-value pair in trainingOptions to 'never' if this is the issue.

[krishna Chauhan]

oh! I just did that

shuffle-never

and it drop down the classification accuacy almost 20%

:(

and about ur first part of question yes it differes by 1-2 %

thats I think high

Can u suggest anything ?

and what i can understand shuffling the data in every epoch is good. no?