How to choose number of hidden layers
5 views (last 30 days)
Show older comments
Meryeme BOUMAHDI
on 25 Jun 2017
Commented: Greg Heath
on 26 Jun 2017
Hi, I want to design a neural network with 3 input and 1 output. Samples are 21 Millions. For predictions and I don't know how many hidden layers and also the network parameters to use to get best results
0 Comments
Accepted Answer
Walter Roberson
on 25 Jun 2017
When you call patternnet, pass it a vector of the sizes of the hidden layers.
2 Comments
Greg Heath
on 26 Jun 2017
WHOOPS!! ATTENTION!!!
I moved MERYEME'S comment from an ANSWER box to a COMMENT box. I don't see it so I must have screwed it up.
SORRY!
The jist was: She is designing for regression, not pattern recognition.
Greg
Greg Heath
on 26 Jun 2017
Walter's comment is valid for REGRESSION, CLASSIFICAITON, TIME-SERIES, etc
If you add FITNET to my suggested search you will see how to implement the double loop approach to minimize the number of hidden nodes for REGRESSION
ANSWERS
Hmin:dH:Hmax Ntrials FITNET ==> 37 HITS
Hmin:dH:Hmax Ntrials PATTERNNET ==> 28 HITS
the NEWSGROUP doesn't yield much with that detail
NEWSGROUP
Hmin Hmax Ntrials FITNET ==> 12 HITS
Hmin Hmax Ntrials PATTERNNET ==> 4 HITS
Hmax Ntrials FITNET ==> 12 HITS
Hmax Ntrials PATTERNNET ==> 4 HITs
Hmax FITNET ==> 14 HITS
Hmax PATTERNNET ==> 4 HITS
Ntrials FITNET ==> 27 HITS
Ntrials PATTERNNET ==> 15 HITS
Hope this helps.
Greg
More Answers (1)
Greg Heath
on 25 Jun 2017
For 3 inputs and 1 output you only need 1 hidden layer. Minimize the number of hidden nodes subject to the maximum training error constraint.
mse(errortrn) <= 0.01*mean(var(targettrn',1))
then the training error Rsquare is 99%.
I use a double loop approach over
1. Number of hidden nodes h = Hmin:dH:Hmax
2. j = 1:Ntrials number of random assignments for initial
weights and trn/val/tst data divisions.
3. [ I N ] = size(input)% [ 3 N ]
[ O N ] = size(target)% [ 1 N ]
Ntrn ~ 0.7*N % default
Ntrneq = Ntrn*O % No of training equations
Nw = (I+1)*H+(H+1)*O = 5*H + 1 % No. of unknown weights
4. A reasonable choice to prevent overtraining an overfit net is
Ntrneq >= 10*Nw ==> Ntrn >~ 50*H
5. Obviously, a major task is to find a small ( << 21 millions!)
subset of data that can adequately represent the 21 million 3-dimensional
inputs. My experience is that for d-dimensional Gaussian distributions
Ntst >= 30*d random picks is usually sufficient.
Taking Nval = Ntst ~ 30*d and
Ntrn ~ (7/3)*(Nval+Ntst) ~ 140*d
Then, N ~ Ntrn + Nval + Ntst ~ 200*d = 600
6. There are zillions of examples of my double loop minimum H approach in
both the NEWSGROUP and ANSWERS. Just search on
Hmin:dH:Hmax Ntrials
Hope this helps.
Thank you for formally accepting my answer
Greg
2 Comments
Walter Roberson
on 26 Jun 2017
The GUI tool is not always flexible enough. You can have the GUI tool create a network with the default number of hidden layers, and then you can tell it to generate the code for the network. You can then edit the code for the network so that it initializes the sizes of the hidden layers the way you want.
See Also
Categories
Find more on Parallel and Cloud in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!