How i can to choose a good structure of neural networks( number of nodes in hidden layer)

I’am new to neural networks (for prediction) and I'm not sure how to go about trying to achieve better test error on my dataset. i have 1200*7 inputs matrix ( 1200 lines and 7 columns)and targets (1200*7) for training my neural network. I tested my neural network by differents datasets (not using in trianing). i got bad MSE about 0,10084 with 19 nodes in hidden layer!!!!. Always My program displays the following error message :

Error using .* Matrix dimensions must agree.

Error in neuralNetworktest (line 95) traintargetsTesting = targetsTesting .*tr.trainMask{1};

please advise my about this problem. thanks .

• * * this is my program : * * *

clear all 
A = load('C:\Users\Omar\Desktop\les données\inputs.txt');
B = load('C:\Users\Omar\Desktop\les données\targets.txt');
C=  load('C:\Users\Omar\Desktop\les données\inputs_Testing.txt');
D= load('C:\Users\Omar\Desktop\les données\targets_Testing.txt');
inputs = A';
targets = B';
inputs_Testing=C';
targets_Testing=D';

% Create a Fitting Network

% Choose Input and Output Pre/Post-Processing Functions
% For a list of all processing functions type: help nnprocess
net.inputs{1}.processFcns = {'removeconstantrows','mapminmax'};
net.outputs{2}.processFcns = {'removeconstantrows','mapminmax'};

% Setup Division of Data for Training, Validation, Testing
% For a list of all data division functions type: help nndivide
net.divideFcn = 'dividerand';  % Divide data randomly
net.divideMode = 'sample';  % Divide up every sample
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;

% For help on training function 'trainlm' type: help trainlm
% For a list of all training functions type: help nntrain

net.trainF = 'trainlm';  % Levenberg-Marquardt

% Choose a Performance Function
% For a list of all performance functions type: help nnperformance

net.performFcn = 'mse'  % Mean squared error

n=10;% number of iteration 

fileoutmse=fopen('mse.xls','w');% open XLS file  to save MSE and number of nodes  

fprintf(fileoutmse,'%s\t%s\n','numnodes','MSE');

          for NNHL=1:20 % NNHL number of nodes 

        fprintf('number of nodes in hidden layer :%d\n',NNHL);
          net = fitnet(NNHL);

      for j=1:n % n= number of iteration 

                   [net,tr] = train(net,inputs,targets);
                   outputs = net(inputs_Testing);

perf = mse(net,targets_Testing,outputs);

fprintf(fileoutmse,'%d\t %.5f\r\n',NNHL,perf);

end 
   end    

     fclose(fileoutmse);    

% Choose Plot Functions
% For a list of all plot functions type: help nnplot

net.plotFcns = {'plotperform','plottrainstate','ploterrhist', ...
  'plotregression', 'plotfit'};
% Train the Network
% Test the Network

errors = gsubtract(targets_Testing,outputs);
performance = perform(net,targets_Testing,outputs)

% Recalculate Training, Validation and Test Performance

traintargets_Testing = targets_Testing .*tr.trainMask{1};
valtargets_Testing = targets_Testing .*tr.valMask{1};
testtargets_Testing = targets_Testing .*tr.testMask{1};
trainPerformance = perform(net,targets_Testing,outputs)
valPerformance = perform(net,valtargets_Testing,outputs);
testPerformance = perform(net,testtargets_Testing,outputs);

% View the Network

view(net)
fileout=fopen('outputs.xls','w'); % open Excel file to save outputs
fprintf(fileout,'%.2f\r\n',outputs);
fclose(fileout);

% Plots
% Uncomment these lines to enable various plots.
%figure, plotperform(tr)
%figure, plottrainstate(tr)
%figure, plotfit(net,inputs,targets)
%figure, plotregression(targets,outputs)
%figure, ploterrhist(errors)
                                -----------------------------

Answer 1

[Greg Heath]

 [ I N ]  = size(input)    % [7 1200]
 [ O N ]  = size(target)   % [ 7 1200 ]
 Ntrn     = N -2*(0.15*N)  % 840
 Ntrneq   = Ntrn*O         % 5880

 Nw = (I+1)*H+(H+1)*O      % 8*19+20*7=152+140 = 292

Since Ntrneq >> Nw, choose

 MSEgoal << mean(var(target',1))

MSE ~ 0.1 is not bad if the RHS >~ 10

For details see my posts

 greg fitnet MSEgoal

[omar belhaj]

 please Sir  I'm new in the field of neural networks.  Idon't understand what's  mean Ntrn, Ntrneq, Nw ?? if you can explain more thanks. Ihave juste [1 1200] targets not [7 1200]. I got Mse=0.1 but with 19 nodes in the hidden layer!!!.

Theoretically number of nodes in the hidden layer 4 or 5 for me(7 inputs parametres and 1 ouput parametrs). But 19 nodes in the hidden layer isn't According to these rules:

•The number of hidden neurons should be between the size of the input layer and the size of the output layer.

 •The number of hidden neurons should be 2/3 the size of the input layer, plus the size of the output layer.
  
•The number of hidden neurons should be less than twice the size of the input layer.

please Sir if you can change my program it will be better to understand.

thank you very much

[Greg Heath]

TOSS THOSE RULES IN THE GARBAGE!!!

Preferable to choose H so that the number of training equations Ntrneq is much larger than the number of unknown weights, Nw. This makes the solution robust w.r.t. noise, interference and measurement errors.

Otherwise, enable validation set stopping (a default) or regularization (trainbr or msereg). Unfortunately, trainbr does not allow validation stopping or a specified regularization fraction.

 [ I N ] = size(input)
 [ O N ] = size(target)
 Ntrneq  = N*O         % Number of training equations

For a single hidden layer MLP NN, the number of weights is

 Nw = (I+1)*H+(H+1)*O

The condition Ntrneq >> Nw is equivalent to

 H << Hub  % Upper bound for Ntrneq > Nw

where

 Hub = -1 + ceil( (Ntrneq-O) / (I+O+1) )

Reference mean-square-error values are obtained when, naively, the output is assumed to be the average of the target values

 MSE00  = mean(var(target',1)) % Biased (N denominator)
 MSE00a = mean(var(target',0)) % Degree of freedom adjusted (N-1 denominator)

If validation set stopping or regularization is not used, I use the following

 Minimize H subject to the condition 

 MSE <= 0.01*Ndof*MSE00a/Ntrneq

where

 Ndof = Ntrneq - Nw

is the number of degrees of freedom for estimating with training data after Nw weights are estimated.

If H has to be increased so that Ntrneq >> Nw does not hold, then use validation stopping or regularization.

In your case with a default division ratio

 [ I N ]  =  [ 7 1200 ]
 [ O N ]  =  [ 1 1200 ]
 Ntrn     = N -2*(0.15*N)  % 840
 Ntrneq   = Ntrn*1         % 840
 Hub      = -1 + ceil( (840 -1) /(7+1+1)) % 93

Therefore N=1200 is large enough so that your value of H = 19 is fine.

[Image Analyst]

omar's so called "Answer" moved here:

Thank you sir Greg Heath,i don't understand what's means these commands??? :

MSE00 = mean(var(target',1)) espcially Var !!!! MSE00a = mean(var(target',0))

and here normally: Ntrneq = N*O why in this line >>> Ntrneq = Ntrn*1 where: Ntrn = N -2*(0.15*N)

[ I N ] = [ 7 1200 ] [ O N ] = [ 1 1200 ] Ntrn = N -2*(0.15*N) % 840 Ntrneq = Ntrn*1 % 840 anohter question please it is necessary to reset the weight during aprrentissage??!! And how i can do?? (to reset the weight of neural neutwork each iteration).

[Greg Heath]

1. Please don't put comments and/or questions comments in the ANSWERS box. Use the COMMENT box.

% Thank you sir Greg Heath, i don't understand what's means these % commands??? : % % MSE00 = mean(var(target',1)) espcially Var !!!! % MSE00a = mean(var(target',0))

 help var
 doc var

They are reference MSEs obtained when the output is, NAIVELY, assumed to be a constant, independent of the input:

 y00 = repmat(mean(target,2),1,N);

 % and here normally: Ntrneq = N*O why in this line >>> Ntrneq = Ntrn*1 
 %where: Ntrn = N -2*(0.15*N)

Well, gee!! What could that possibly mean??? ( O = 1?)

 % [ I N ] = [ 7 1200 ] [ O N ] = [ 1 1200 ] 
 % Ntrn = N -2*(0.15*N) % 840 Ntrneq = Ntrn*1 % 840 
 % 
 % anohter question please it is necessary to reset the weight during 
 % aprrentissage??!!

I don't know that word.

 % And how i can do?? (to reset the weight of neural neutwork each iteration).

 help configure
 doc configure

Quite a few of these questions have been answered by me in the past. In the future, before asking a question, please make a search using

 greg searchword

Thanks,

Greg

[omar belhaj]

Hey Greg Can you help me again please. two lines following are used for normalized data inputs?? or can i using simple mathematical relationship In=(Inn-Imin)/(Imax-Imin) while In: normalized input ; Inn: No normalized input

   net.inputs{1}.processFcns = {'removeconstantrows','mapminmax'};
    net.outputs{2}.processFcns = {'removeconstantrows','mapminmax'};

best regards