Do Train and Test have to be in the same file - Neural Network Matlab Toolbox

Hello,

I am using Neural Network toolbox for pattern recognition. I split my data to train and test. I was able to train the network with train data. Now I want to run my test data through the network and obtain some results.

My questions are:

Is there a way to disable the training on the test data and just run the test by using the previous training results? Or do I need to combine my test and train data as one file and adjust the ratio of testing/training/validation?

I was hoping to be able to train on one set of data and then run different sets of test data without having to go through training again. Is such a thing possible?

Thank you in advance for your help.

Answer 1

[Greg Heath]

The training algorithm does not estimate weights using validation or test data.

 data   = design + test;
 design = train + val
          train: Estimates weights and biases
          val  : Stops training when performance on the non-training val data is optimized 
          test : Obtains an UNBIASED estimate of performance on unseen non-training data

You should be able to train with any combination of val and test data. Test data does not affect training. However val data provides one of several stopping options that is explicitly designed to prevent poor performance on non-training data.

Regardless of the combination used when training, the net can be saved and used on unseen data

 save net
 delete net  % from working space
 load net
 ynew = net(xnew)

Hope this helps

Thank you for formally accepting my answer

Greg .

[H]

Thank you Greg. That made it clear.

I use:

 save net
 delete net  % from working space
 load net
 ynew = net(xnew)

As you suggested and to see the test result I use:

figure, plotconfusion(targets,ynew)

Is this the correct way? This seems to give me all confusion matrix not the test confusion matrix.

Also I want to use a front end processing such as PCA on my data to get better results, but seems like applying a front end process (feature extraction) makes the result worse. Any thought on this?

I really appreciate your help.

Thanks

Answer 2

[Greg Heath]

No. it is not the correct way.

The inputs to confusion and plotconfusion must have the same dimensions because they should be an estimated output and the corresponding true(target) output.

Generally, PCA is not appropriate for classification because it maximizes mixture variance instead of class separation. The equivalent for classification is

 help plsregress
 doc plsregress

Other functions that can be useful for classifier input dimensionality reduction are

 stepwise and stepwise fit (Generalized Linear (e.g., Polynomial models))

 sequentialfs (More general. However, I have not used this yet)

Hope this helps

Thank you for formally accepting my answer

Greg

[H]

Thanks for your help Greg. I will look into plsregress.

Sorry for my stupid question, but neural network is a new subject to me and I am trying to figure out how to work with it. Would you be able to tell me how I should visualize the result (if I want to see what the percentage for recognition is) after ynew=net(xnew)?

Thanks again!