Validation check = 0 for traingdm

15 views (last 30 days)
Linh Pham
Linh Pham on 10 Mar 2019
Edited: Greg Heath on 10 Mar 2019
I am trying to run traingdm for my neural net. But I keep getting validation checks = 0. I have tried the following:
  • varying the number of hidden layer size, learning rates and momentum.
  • Changing the stopping criteria for trainParam.goal
  • Changing the number of max epochs.
  • Trying crossentropy instead of mse for performance measure
Please find my code below. Thank you,
x = input';
t = target';
%grid search
[p,q,r,s] = ndgrid(hidlaysize,learning_rates,momentum);
pairs = [p(:) q(:) r(:)];
% Tune hyper-parameters:
hidlaysize = [50 15 20];
learning_rates = [0.9 0.03 0.003];
momentum = [0.9 0.5 0.98];
% 'traingdm'
trainFcn = 'traingdm';
% mse or crossentropy:
performFcn = 'mse';
% Set up training for each set of parameters:
for a = 1 % try 1 loop for testing but results dont change that much for more loops. The real condition is a = 1:size(pairs,1)
MLP_run = a;
MLP_net = patternnet([pairs(a,1) pairs(a,1)], trainFcn, performFcn);
if trainFcn == 'traingdm'
MLP_net.trainParam.lr = pairs(a,2);
MLP_net.trainParam.mc = pairs(a,3);
else
break
end;
%Preprocess inputs
MLP_net.input.processFcns = {'removeconstantrows','mapminmax','mapstd'};
% Stopping criteria:
MLP_net.trainParam.epochs = 3000;
%Reset the minimum performance goal:
MLP_net.trainParam.goal = 0.01; %?? % Not working for either 0.15 or 0.0025
% Set the maximum number of iterations to assess the validation error
MLP_net.trainParam.max_fail = 20;
% List of all plot functions type:
MLP_net.plotFcns = {'plotperform','plottrainstate','ploterrhist', ...
'plotconfusion', 'plotroc'};
end;
% Training:
MLP_net = init(MLP_net);
[MLP_net,tr] = train(MLP_net,x,t);
This is my result:
Screen Shot 2019-03-10 at 09.41.49.png

Answers (1)

Greg Heath
Greg Heath on 10 Mar 2019
Edited: Greg Heath on 10 Mar 2019
What you are worrying about is irrelevant. Your data is so good you don't even need a validation subset.The main purpose of a validation subset is to prevent you from continuing to improve performance on training data when it will not significantly improve performance on nontraining dat.
Typically, it is sufficient for the mean-square-error to be at least 100 times smaller than the average target variance. For example, I use
MSEgoal = 0.01*mean(var(target',1) )
Lower values do not significantly improve performance.
I have written several zillion tutorials in BOTH the NEWSGROUP and ANSWERS. Just include
Greg
as a search word.
Hope this helps.
THANK YOU FOR FORMALLY ACCEPTING MY ANSWER
Greg

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!