How many neurons in the hidden layer???

Number of neurons in the input layer of my feed-forward network is 77, number of neurons in output layer is 7, I want to use multiple hidden layers, How many neurons, Should I keep in each hidden layer from first to last between input and output layer

Answer 1

[Greg Heath]

For a given performance on training data, the fewer the number of weights, the better. Then the net will better generalize to validation, test and unseen data with the same statistical properties.

Since universal approximators can be made with a single hidden layer. Most default configurations have an I-H-O node topology for I dimensional inputs, O-dimensional outputs and H hidden nodes. The resulting number of unknown weights is

 Nw = (I+1)*H+(H+1)*O

With Ntrn input/target training pairs, the number of training equations is

 Ntrneq = Ntrn*O.

For the number of unknowns to not exceed the number of equations, Nw <= Ntrneq which yields the following upper bound on the number of hidden nodes

 H <= Hub = (Ntrneq-O)/(I+O+1)

If possible, increase stability, robustness and generalization with Nw << Ntrneq which yields.

   H << Hub

It is very doubtful that you need 77 inputs. Using more than necessary will increase the number of weights and, typically, destabilize your design. Therefore, you should first reduce the number of inputs before trying to reduce the number of hidden nodes.

The choice of variables obtained from the linear STEPWISE and STEPWISEFIT model functions are, typically relevant.

Hope this helps.

Thank you for formally accepting my answer

Greg

PS I should have a number of posts in the NEWSGROUP and ANSWERS which are relevant.

[Beulah Jasmine]

My project is on Speech Emotion recognition.... i have extracted MFCC feature of order 75 from speech signal. So, there is 76 cepstral coefficients including the zeroth order cepstral coefficient. Including the log energy value, there is 77 values for each frame of the speech signal. hence there is 77 input.... i cant reduce that....

[Greg Heath]

On the contrary. For the purposes of pattern-recognition, the number of inputs can be reduced using the method of PARTIAL LEAST SQUARES.

I think the appropriate MATLAB function is

             PLSREGRESS

Search in both the NEWSGROUP and ANSWERS. Hope this helps.

Greg

Answer 2

[Thorsten]

There are only rules of thumb on how to do this, I think.

http://stackoverflow.com/questions/3345079/estimating-the-number-of-neurons-and-number-of-layers-of-an-artificial-neural-ne