neural net performance function 'msereg' disabled in 2013B

I notice that 'msereg' neural net performance function disabled in 2013B. Any attempt to set net.performFcn='msereg' reverts back to 'mse'. For where in the code this happens, see ...\toolbox\nnet\nnet\@network\subsagn.m line 1901. Temporarily commenting out lines 1901-1903 allows to create a NN with net.performFcn='msereg' but training such a net results in an error.

Why was this functionality removed? Why does the documentation still mention is? Can this functionality be restored in any way?

   Richard Hodges

Answer 1

[Greg Heath]

It was probably discontinued because mse now has a regularization option.

 help mse

HTH

Thank you for formally accepting my answer

Greg

[Richard Hodges]

Thank you for the information. But there is very little documentation on the actual usage or tradeoffs.

Running some experiments, using trainbr on a multi-input multi-output time series net, I notice that when I set net.performParam.normalization to 'none', then TRAINBR gives a warning that net.performParam.regularization has been set to 0 (even though I set to .5)--does regularization REQUIRE a non-default normalization?

And when I set regularization to .5 and normalization to 'standard' the net training terminates with "maximum Mu reached" (Mu is 5e10), with a result in which the Error Histogram is several times broader (i.e. worse) than before, while the reported performance value is orders of magnitude smaller. The latter is expected, because my targets were in a range much larger than +-1. But what is the formula for what is minimized in the the regularized case? Is it such that the Sum Squared Param value is added to the performance value, or is it something else. Is Mu a factor in this term? In my test case, the SSP value is orders of magnitude larger than the final performance value.

Can you cite literature which this regularization is based on?

Thank you,

    Richard

[Greg Heath]

It would help immensely if you ran your code on one of MATLAB's NN data sets so that we could compare details.

 help nndatasets

Please post code, comments and error messages.

[Greg Heath]

For some reason

 net.trainFn    = 'trainbr';
 net.performFcn = 'mse'

does result in regularization where the weights alpha and beta are AUTOMATICALLY determined.

 To try to understand how the regularization is done, 

 type trainbr 

 find gamk
 find ssX

 I haven't figured the logic out yet.

Bottom line: Accept the default 'mse' and it will work fine. The time history of gamk and ssK can be obtained from the training history

 gamk  = tr.gamk
 ssX   = tr.ssX

Hope this helps.

Greg

P.S. Report this to MATLAB

[Mohd Khan]

I would really like to know that is this tr.gamk. It is reported as 29.1771 in my case. I understant that 29 is the number of parameters used by the model in current stage but I have not idea what is 1771. Kindly, help me out.